2010 Jan 29 - Fri

Migrating Bacula 2.x on Debian Etch to 3.x on Squeeze

Debian Etch, which is the current release, has the Bacula 2.x packages. I needed to upgrade to the Bacula 3.x packages, which are located in debian/testing, also known as the forthcoming Debian Squeeze release. In addition, since PostgreSQL 8.3 is packaged in Etch, and PostgreSQL 8.4 is packaged in Squeeze/testing, a database migration is also required.

I had attempted updating my sources.list file to testing and then running the apt-get dist-upgrade process. This broke some dependences, and also broke on a udev migration. I guess testing has more testing to do on the distribution upgrade process.

In the end, I built a new Bacula service on a freshly installed Debian testing server.

The special consideration for this configuration is that it needs to handle backing up servers across a WAN. As such, backups may travel through one or more firewalls. Through such a configuration, it is very difficult to get the firewall ports opened for the various necessary Bacula service ports. The better way to tackle this is through the use of ssh' port local and remote port forwarding capability. Port 22 is becomes the only necessary port to open on a firewall. The ssh-tunnel.sh script helps make this happen.

To build the server, when it came to package selection, I unselected all packages, and then chose just the database package which installed PostgreSQL.

After the basic server finished installing and rebooted, I manually installed the following packages:

apt-get install bacula-common-pgsql
apt-get install bacula-client
apt-get install bacula-director-common
apt-get install bacula-director-pgsql
apt-get install bacula-sd-pgsql
apt-get install bacula-server

If starting with a new database, then dbconfig-common can be used. If migrating an older database, don't use dbconfig-common, and use the manual methods I'll describe further on. There is further documentation in /usr/share/doc/bacula-director-pgsql.

During installation of the bacula packages, a new user of 'bacula' is created, as well as a group called 'tape'. The 'bacula' user has a home directory of /var/lib/bacula.

Into that directory, create a .ssh directory for any authorized_keys and known_hosts required. I also created a keys subdirectory to hold the public/private keys for ssh'ing into other servers for processing backups. I called the two files 'bacula' and 'bacula.pub'. These will be referenced in my customized ssh-tunnel.sh script.

Run

dpkg-reconfigure exim4-config

to reconfigure the mail system to allow outbound mail delivery.

My backups go onto a remote file share. I created an entry in /etc/fstab along the lines of:

10.1.1.1:/bu /mnt/nas nfs rw,hard,intr,async,nodev,nosuid 0 0

Ensure that the NFS client is installed through:

apt-get install nfs-common

In /etc/postgresql/8.4/main/pg_hba.conf, I have lines along:

host    bacula  bacula          127.0.0.1/32            trust
host    bacula  sysadmin        127.0.0.1/32            trust
local   bacula  bacula          trust
local   bacula  sysadmin        trust

As an aside, a useful command to find out database information is through the use of:

psql -l

When migrating the database to 8.4, there are modifcations to the pg_dump command required (which are required to prevent import errors along the lines of 'ERROR: invalid byte sequence for encoding "UTF8"', basically resolving the UTF-8 to SQL_ASCII issues in Bacula):

pg_dump -E SQL_ASCII -U bacula bacula > /var/lib/bacula/bacula.sql

On the new server, use the following to import the database:

dropdb bacula
su - postgres
psql
create role bacula;
create database bacula owner=bacula encoding='SQL_ASCII' template=template0;
/q
psql bacula </var/lib/bacula/bacula.sql

Basic instructions for updating the database from Bacula table version 10 to Bacula table version 11 is found in /usr/share/bacula-director/update_postgresql_tables:

BEGIN;
ALTER TABLE file ALTER fileid TYPE bigint ;
ALTER TABLE basefiles ALTER fileid TYPE bigint;
ALTER TABLE job ADD COLUMN readbytes bigint default 0;
ALTER TABLE media ADD COLUMN ActionOnPurge smallint default 0;
ALTER TABLE pool ADD COLUMN ActionOnPurge smallint default 0;

-- Create a table like Job for long term statistics
CREATE TABLE JobHisto (LIKE Job);
CREATE INDEX jobhisto_idx ON JobHisto ( starttime );

UPDATE Version SET VersionId=11;
COMMIT;

Once the configuration files for the director, storage manager, and file manager are ready, bacula can be managed through 'bconsole'.

My modified /etc/bacula/scripts/ssh-tunnel.sh looks like:

#!/bin/sh
# script for creating / stopping a ssh-tunnel to a backupclient
# Stephan Holl sholl@gmx.net
# Modified by Joshua Kugler joshua.kugler@uaf.edu
# Modified by Ray Burkholder ray@oneunified.net
#
#

# variables
USER=bacula
CLIENTADDR=$2
# CLIENTPORT is local end
CLIENTPORT=$3
#LOCAL=your.backup.server.host.name
# local is a local address and uses ssh's remote/local port forwarding
LOCAL=127.0.0.1
SSH=/usr/bin/ssh
SSHOPTIONS=-vfnCN2
LOG1=/var/lib/bacula/log1.log
LOG2=/var/lib/bacula/log2.log
#LOG1=/dev/null
#LOG2=/dev/null
# location of the public/private keys used with ssh to gain access to remote servers
KEY=/etc/bacula/keys/bacula  

case "$1" in
 start)
    # create ssh-tunnel
        echo "Starting SSH-tunnel to $CLIENTADDR..."
        $SSH $SSHOPTIONS -o PreferredAuthentications=publickey -i $KEY -l $USER \
            -R 9101:$LOCAL:9101 -R 9103:$LOCAL:9103 -L $CLIENTPORT:$LOCAL:9102 $CLIENTADDR \
            >> $LOG1 2>> $LOG2
        exit $?
        ;;

 stop)
        # remove tunnel
        echo "Stopping SSH-tunnel to $CLIENTADDR..."
        # find PID killem
        PID=`ps ax | grep "$SSH $SSHOPTIONS -o PreferredAuthentications=publickey -i $KEY" \
             | grep "$CLIENTADDR" | awk '{ print $1 }'`
        kill $PID
        exit $?
        ;;
 *)
        #  usage:
        echo "             "
        echo "      Start SSH-tunnel to client-host"
        echo "      to bacula-director and storage-daemon"
        echo "            "
        echo "      USAGE:"
        echo "      ssh-tunnel.sh {start|stop} client.fqdn"
        echo ""
        exit 1
        ;;
esac

The links I used for getting started with ssh-tunnels are found at:

• /usr/share/doc/bacula-common/examples/ssh-tunnel-README.txt.gz
• http://wiki.bacula.org/doku.php?id=sshtunnel

In /etc/hosts file, 127.0.0.1 should be the only line referring to the local server. The exteral port ip address should be commented out:

127.0.0.1      localhost  bu.example.com        bu
#10.10.10.1    bu.example.com        bu

In the bacula-dir.conf configuration file, a typical client configuration will look similar to:

Client {
  Name = mail-fd
  Address = 127.0.0.1
  FDPort = 9130  # specific port for this client, allows multiple simultaneous backups
  Catalog = MyCatalog
  Password = "xxxxxx"          # password for FileDaemon
  File Retention = 120 days         
  Job Retention = 4 months          
  AutoPrune = yes                     # Prune expired Jobs/Files
}

The special characteristic of the above configuration is the use of a unique port number for FDPort. Each client in the bacula-dir.conf should have a unique port number. This allows bacula to tunnel via ssh to remote clients and redirect them to the storage manager on the local server.

The definition of the storage device in bacula-dir.conf will have Address=127.0.0.1 and SDPort=9103.

The job description for each client should have something similar to:

Job {
  Name = "mail-fd"
  Client = mail-fd
  JobDefs = "DefaultJob"
  FileSet = "FileSet_mail"
  Storage = storageSshClients
  Write Bootstrap = "/var/lib/bacula/mail.bsr"
  Priority = 12
  Run Before Job = "/etc/bacula/scripts/ssh-tunnel.sh start mail.example.com 9130"
  Run After  Job = "/etc/bacula/scripts/ssh-tunnel.sh stop  mail.example.com 9130"
}

When using Bacula in console mode, a useful command to find out the meaning of the backup status codes:

*sqlquery
Entering SQL query mode.
Terminate each query with a semicolon.
Terminate query mode with a blank line.
Enter SQL query: select * from status;
+-----------+---------------------------------+
| jobstatus | jobstatuslong                   |
+-----------+---------------------------------+
| C         | Created, not yet running        |
| R         | Running                         |
| B         | Blocked                         |
| T         | Completed successfully          |
| E         | Terminated with errors          |
| e         | Non-fatal error                 |
| f         | Fatal error                     |
| D         | Verify found differences        |
| A         | Canceled by user                |
| F         | Waiting for Client              |
| S         | Waiting for Storage daemon      |
| m         | Waiting for new media           |
| M         | Waiting for media mount         |
| s         | Waiting for storage resource    |
| j         | Waiting for job resource        |
| c         | Waiting for client resource     |
| d         | Waiting on maximum jobs         |
| t         | Waiting on start time           |
| p         | Waiting on higher priority jobs |
+-----------+---------------------------------+
Enter SQL query:
End query mode.

For the bacula entry in /etc/passwd, change /bin/false to be /bin/sh.

For each server to which will be connected via ssh, within the context of the bacula user, use the following command to update ~/.ssh/known_hosts:

ssh -l bacula -i /etc/bacula/keys/bacula -v server.example.com

2009 Sep 29 - Tue

Upgrade to KDE4: Black Screen, Obsidian Cursor

Today when upgrading my Debian Lenny/KDE to the latest version, I started having problems with KDE.

On my first upgrade, I did a simple 'apt-get update', 'apt-get upgrade' sequence. A bunch of packages were held back. The end result was that I could log in to KDE, and could see a desktop, but I had no menu interface.

Considering that there were a bunch of packages being help back, I did a 'apt-get update', 'apt-get dist-upgrade' sequence. Upon logging into the KDE shell, all I saw was a black screen and a shiny obsidian cursor.

It looks like the transition from KDE 3.5 to KDE 4.0 is not seamless in this Debian (Lenny) point release. However, that isn't quite correct. In my /etc/apt/sources.list file I do have entries for testing and experimental. So..., I may now be downloading testing or experimental releases.

In any case, the resolution to the problem appears to be to drop into the console and run one of these three commands: 'apt-get install kde-standard', 'apt-get install kde-minimal', or 'apt-get install kde-full'.

2009 Sep 22 - Tue

Updating WebGUI

WebGUI's Update Page has links to the various updates.

Upgrade information can be found at Upgrading WebGUI.

To view the current upgrade history:

cd /data/WebGUI/sbin
perl upgrade.pl --history --doit
perl testEnvironment.pl

Stop Spectre:

cd /data/WebGUI/sbin
perl spectre.pl --shutdown

Make a backup of the files in /data/WebGUI/etc. The originals will be over-written, but the customized ones should be ok after the upgrade.

Decompress the new archive over the old files (with the current version as of this writing):

cd /data
wget http://update.webgui.org/7.x.x/webgui-7.7.20-stable.tar.gz
tar -zxvf  webgui-7.7.20-stable.tar.gz

Read the WebGUI/docs/gotcha.txt file.

Read the WebGUI/docs/changelog/7.x.x.txt to check out the latest changes.

Restart apache with '/etc/init.d/apache2 restart'.

Run the upgrade:

cd /data/WebGUI/sbin
perl upgrade.pl
perl upgrade.pl --doit --backupDir /data/bu/wg

Run testEnvironment.pl:

cd /data/WebGUI/sbin
perl testEnvironment.pl

Start Spectre:

cd /data/WebGUI/sbin
perl spectre.pl --daemon

Restart apache with '/etc/init.d/apache2 restart'.

2009 Aug 19 - Wed

Debian Dpkg Install

From the Debian Security Announce List, a little short-cut for installing .deb packages:

wget url
        will fetch the file for you
dpkg -i file.deb
        will install the referenced file.

2009 Jul 31 - Fri

Installing OpenLDAP on Debian Lenny

Here are a few basic apt-get commands for the OpenLDAP installation. I have to look into how TLS is actually implemented and configured.

apt-get install libsasl2-2 libgnutl26
apt-get install ldap-utils libsasl2-modules-ldap 
apt-get install  slapd libldap-2.4-2

Installing Asterisk 1.6.2.0 beta3 on Debian Lenny 5.0.2

Debian package manager has the Asterisk v1.4 flavour as a package, but I wanted the latest to try out. Here is the work flow to get the basics in place:

Here are some pre-requisites to install. I havn't figured out the 'lua' bit yet:

apt-get install build-essential
apt-get install openssl
apt-get install libssl-dev
apt-get install libldap2-dev
apt-get install libncurses5-dev
apt-get install festival-dev festival
apt-get install curl libcurl4-openssl-dev
apt-get install lua5.1
apt-get install uw-mailutils
apt-get install libgsm1
apt-get install libiksemel3
apt-get install libogg0
apt-get install libspeex1 libspeexdsp1
apt-get install libtonezone1
apt-get install libvorbis0a libvorbisenc2
apt-get install doxygen
apt-get install postgresql-server-dev-8.3 postgresql-client-8.3
apt-get install libnewt-dev
apt-get install linux-headers-2.6.26-2-686
apt-get install libogg-dev
apt-get install libvorbis-dev
apt-get install liblua5.1-posix-dev
apt-get install libgsm1-dev

The basic hardware layer for the kernel is next. This includes dummy timers for systems without additional telephony hardware.

d /usr/src
wget http://downloads.asterisk.org/pub/telephony/dahdi-linux/dahdi-linux-2.2.0.2.tar.gz
tar -zxvf dahdi-linux-2.2.0.2.tar.gz
cd dahdi-linux-2.2.0.2
make 
make install

User space Dahdi tools are then built:

d /usr/src
wget http://downloads.asterisk.org/pub/telephony/dahdi-tools/dahdi-tools-2.2.0.tar.gz
tar -zxvf dahdi-tools-2.2.0.tar.gz
cd dahdi-tools-2.2.0
./configure  \
   --sysconfdir=/etc/ \
    --libdir=/usr/lib \
   --localstatedir=/var/local \
   --datarootdir=/usr/share \
   --includedir=/usr/include 
make menuselect
make
make install
make config

This portion installs a recent beta releaes of the Asterisk engine:

cd /usr/src
wget http://downloads.asterisk.org/pub/telephony/asterisk/asterisk-1.6.2.0-beta3.tar.gz
tar -zxvf asterisk-1.6.2.0-beta3.tar.gz
cd asterisk-1.6.2.0-beta3
./configure  \
   --sysconfdir=/etc/ \
    --libdir=/usr/lib \
   --localstatedir=/var/local \
   --datarootdir=/usr/share \
   --includedir=/usr/include \
   --disable-xmldoc

Ensure you've got all the various libraries, modules, bits and pieces attached:

make menuselect

If you are installing a system from scratch, the run all these. If you already have configuration files, skip the 'make samples'.

make
make install
make samples
make progdocs

If you are using PostgreSQL, build the database tables with:

su - postgres
psql template1
> create database asterisk;
> quite;
psql asterisk < /usr/src/asterisk-1.6.2.0-beta3/contrib/scripts/realtime_pgsql.sql

Then edit /etc/asterisk/res_pgsql.conf to add connection information. Other files you may need to edit include:

sip.conf
dahdi-channels.conf
cdr_manager.conf
cdr_pgsql.conf
cdr.conf
extensions.conf
iax.conf

Get things started with:

/etc/init.d/dahdi start
safe_asterisk

2009 Jul 24 - Fri

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 6

I've spent the last articles writing about getting an open source email server up and running. So far so good. My email logs show that a tremendous amount of spam is being blocked. One begins to wonder if there any real email remaining any more.

During the building of this server, a number of web sites provided useful information for troubleshooting and for configuration. I'm listing them here for reference before I close them out.

• http://www.linuxweblog.com/blogs/sandip/20080206/sendmail-accessdb-example: provided useful explanations and examples of the interactions between the access database, the blacklist_recipients feature, the value part of the map, and how to use the delay_checks feature for negative or positive exception handling.
• ZEN Return Codes: The 127.0.0.x return codes. Basically, 127.0.0.2 is for direct UBE sources, spam services, and ROKSO spammers; 127.0.0.4-8 are for illegal 3rd party exploits, including proxies, worms and trojan exploits; and 127.0.0.10-11 are for non-MTA IP address ranges set by outbound mail policy
• : a good description of the sendmail.mc file, it's options, features, and ordering. It goes into some detail about special considerations of the VIRTUSER_DOMAIN_FILE. It also goes into uses and configuration examples of the access file.
• SPF Setup Wizard: I'm not sure if the Sender Policy Framework (SPF) is much in use, but this web size provides a wizard for it's DNS record creation.
• Sendmail Readme for Configuration: The original source for configuring Sendmail.
• Linux Home Server HowTo: Sendmail: another article on how to build a full-fledged email server. One key command for ensuring you havn't configured an open relay through a series of 19 tests: 'telnet relay-test.mail-abuse.org'. When run from the mail server, the server at relay-test.mail-abuse.org will connect back to your server on port 25 and run the series of tests.
• sendmail.mc: this is the best organized and best documented sample sendmail.mc file I've ever seen.
• xabean's sendmail.mc: example sendmail.mc with native macros and a milter, with hotlinks to relevant sections in the Sendmail Readme file.
• Hugo van der Kooij's sendmail.mc: looks like he no longer runs sendmail, but here is his sendmail.mc with some native macros.

In some follow-up, I came across MailWatch, which is a web-based front-end to MailScanner written in PHP, MySQL and JpGraph and is available for free under the terms of the GNU Public License.

2009 Jul 19 - Sun

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 5

A couple of articles ago, I started with a DoveCot Installation. I managed to download, build, and get a rough installation. I also prepared a userid for the service. It was at that point in the Dovecot installation instructions where they started talking about certificates, and I side-tracked into Certificate Authorities and certificate installation.

In /etc/dovecot, I copied dovecot-example.conf to dovecot.conf. In dovecot.conf, I updated the following lines to get things started:

protocols = imap imaps
disable_plaintext_auth = no
ssl = no
mail_location = maildir:~/Maildir
#mail_location = maildir:/%h/Maildir
auth_debug_passwords = yes

Dovecot Wiki does a good job of explaining the installation process. In fact, the non-ssl installation process is quite painless, and consists mostly of testing the connection.

Once the basic configuration is tested, then enable the configuration for ssl, and restart Dovecot.

disable_plaintext_auth = yes
ssl = yes
auth_debug_passwords = no
# Same keys from the sendmail installation
ssl_cert_file = /etc/ssl/private/mail.example.com.crt
ssl_key_file = /etc/ssl/private/mail.example.com.key

Startup an IMAP session with a Mail Client and try IMAP and IMAPS. Try sending email as well through the SMTP Sendmail connection with encryption. Tcpdump can be used to look at packets.

There is a Sample Dovecot init.d script which can be used to start, stop, and reload the service. The sample can be pasted verbatim into /etc/init.d/dovecot. Also do a 'chmod 755 /etc/init.d/dovecot'. Then '/etc/init.d/dovecot start'.

With a successful send and receive of email, that wraps up the rather lengthy configuration of a reasonably protected email solution encompassing Sendmail as an email transport mechanism, Dovecot as an IMAP/IMAPS service, and MailScanner with SpamAssassin/F-Prot for email scanning and protection.

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 4

It has taken a series of articles to get Sendmail installed and working with authentication, inline encryption, and some inline DNSBL capabilities. In this article, I'll see if I can get MailScanner, SpamAssassin and a virus scanner up and running with Sendmail.

Before starting into that though, I have a couple of links to other sites which have good information for tuning the sendmail.mc file:

• Acme's Sendmail Config
• Technologies OC9 Sendmail Config
• Older site of Sendmail Links

Back to the install. Starting with SpamAssassin, which looks like the last version is 3.2.5 from June of 2008, which is a Perl based utility, it can be downloaded from CPAN by starting the command line with 'perl -MCPAN -eshell':

install Bundle::CPAN
install Term::ReadLine
install MIME::QuotedPrint
install YAML
install YAML::Syck
install MIME::Base64
install Time::HiRes
install Digest::SHA1
install Net::DNS
install Mail::SPF
install IP::Country
install Net::Ident
install Mail::DomainKeys
install Mail::DKIM
install DBI
install LWP::UserAgent
install HTTP::Date
install Encode::Detect
install Mail::SpamAssassin

The pre-requisites build nicely, but the main Mail::SpamAssassin unit does not test well because it tries to start a daemon, which doesn't appear to do so. To find the reason will take some digging, but in the meantime, a force install may or may not be required. It probably is irrelevant anyway as MailScanner does not use spamd.

For a virus scanner, I've used f-prot in the past, and I'll try it again for this install. Others have used ClamAV, and I may add it as a secondary scanner. (Note, the file downloaded is a 64bit version). The last bit of the install script will ask if the daemon should be installed in crontab.... select no as MailScanner will it start it manually. Nor should Sendmail be configured to run the scanner.

cd /usr/src/
wget http://files.f-prot.com/files/unix-trial/fp-Linux-x86_64-ws.tar.gz
cd /opt
tar -zxvf /usr/src/fp-Linux-x86_64-ws.tar.gz
cd f-prot
./install-f-prot.pl
fpscan /etc/passwd

Create a test file and put the EICAR virus into it. Run 'fpscan test' to ensure it finds the virus.

For MailScanner, the following Perl modules are required:

install Sys::Syslog
install Net::CIDR
install IO::Stringy
install Mail::Util
install File::Spec
install HTML::Tagset
install HTML::Parser
install MIME::Tools
install File::Temp
install Convert::TNEF
install Compress::Zlib
install Archive::Zip
install Check::ISA

Next steps:

cd /usr/src
wget http://www.mailscanner.info/files/4/tar/MailScanner-install-4.77.10-1.tar.gz
tar -zxvf MailScanner-install-4.77.10-1.tar.gz
cd MailScanner-install-4.77.10
./install.sh

A few changes, like the domain name, may need to be changed in the /opt/MailScanner/etc/MailScanner.conf file.

Add the following with 'crontab -e' (the minute offsets may be randomized):

37      5 * * * /opt/MailScanner/bin/update_phishing_sites
07      * * * * /opt/MailScanner/bin/update_bad_phishing_sites
58     23 * * * /opt/MailScanner/bin/clean.quarantine
#42      * * * * /opt/MailScanner/bin/update_virus_scanners
#3,23,43 * * * * /opt/MailScanner/bin/check_mailscanner

In /etc/mail/sendmail.conf MailScanner install notes recommend changing 'DAEMON_PARMS="";' to:

DAEMON_PARMS="-ODeliveryMode=d -OQueueDirectory=/var/spool/mqueue.in";

Instead, use:

DAEMON_PARMS="-ODeliveryMode=background -OQueueDirectory=/var/spool/mqueue.in";

By default, Sendmail will use a Delivery Mode of Background, which operates by forking itself and processing the message. With a MailScanner Delivery Mode of Deferred, no DNS or DB lookups are performed. QueueOnly mode will actually perform DNS lookups, which is what I need for handling the SpamHaus enhdnsbl Features, but serializes all inbound connections. Queue mode sounds like the most straight forward option for working with MailScanner but may not be just right. I think that Background will work better, as it will fork and handle simultaneous connections. However, on further testing, I find that Sendmail delivers mail with Background mode, and queues it for Sendmail with QueueOnly mode, so QueueOnly mode it is.

Rerun /usr/sbin/sendmailconfig, then '/etc/init.d/sendmail restart' to get the mta agent and queue runner running as separate processes.

Add a 'crontab -e' entry to ensure MailScanner is always running:

0,20,40 * * * * [ -x /opt/MailScanner/bin/check_mailscanner ] && /opt/MailScanner/bin/check_mailscanner >/dev/null 2>&1

Edit the /opt/MailScanner/etc/MailScanner.conf file:

• Set 'Virus Scanning' to yes
• Set 'Virus Scanners' to f-port-6

Test the virus scanner with '/opt/MailScanner/lib/f-prot-6-wrapper /opt/f-prot eicar.virus'.

Restart MailScanner.

2009 Jul 18 - Sat

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 3

In part two of this series, I started into the installation of the Dovecot IMAP service. The IMAP serivce can use validation and encryption through the use of SSL/TLS services. SSL/TLS services require the use of Certificates signed through a Certificate Authority. Many installation directions provide information for using the simple expedient of self-signed certificates. As some of these services I'm building are quasi-public, I wanted to go through the exercise of getting my certificates signed through a Certificate Authority. As such, I was side-tracked into doing some research to come up with two intermediate articles:

• Certificate Authorities
• OpenSSL Server Certificates

I'm going to step back to my SendMail install, and get a certificate installed in order to utilize SendMail's TLS based verification and encryption capabilities.

In the /etc/mail/sendmail.mc file, the following needs to be available (I've enabled AUTH as well):

include(`/etc/mail/sasl/sasl.m4')dnl
include(`/etc/mail/tls/starttls.m4')dnl

Don't put these lines in the submit.mc file as they will cause permission errors.

For configuring AUTH (SASL2), edit /etc/default/saslauthd and make sure 'MECHANISMS="pam"' is included and then start the service: /etc/init.d/saslauthd start. Shell users should now be able to authenticate, otherwise use /usr/sbin/saslpasswd2 to add users.

You cancheck in /etc/mail/tls to see various self-signed certificates which have already been created and linked within the configuration file /etc/mail/tls/starttls.m4. The various settings can be changed to match the new certificate. I changed the line with confCACERT to match my StartCom CA found in /etc/ssl/certs. I had placed my new server key and cert in /etc/ssl/private, and in sendmail.mc, updated confSERVER_CERT and confSERVER_KEY to match.

Once the certificates are properly installed and SendMail restarted, it can be tested by connecting to telneting to port 25, running 'ehlo localhost' and looking for a line with '250-STARTTLS'. If it is there, all is well.

I found the page at SMTP STARTTLS in sendmail/Secure Switch to help somewhat in building the scenario.

For testing the STARTTLS capability, one can use the one of the following openssl commands (the first works better than the second):

openssl s_client -starttls smtp -connect localhost:25
openssl s_client -ssl3 -state -debug -msg -connect localhost:25

For other OpenSSL s_client command line parameters, visit: s_client man page.

At one point, I was getting errors in sendmail logs with:

STARTTLS=read: 12080:error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number:s3_pkt.c:284:
STARTTLS: read error=generic SSL error (-1), errno=104, 
  get_error=error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number, retry=1, ssl_err=1

I think these are permissions related depending upon privleges of certificate files and the username under which sendmail is running. Sendmail is now running under root and no longer has these problems. The errors magically disappeared during some restart so I can't confirm this for sure. ... further information: the errors happen when running the 'openssl s_client -ssl3 -state -debug -msg -connect localhost:25' command, but not the 'openssl s_client -starttls smtp -connect localhost:25'. I havn't spent the time to determine why yet.

I was also getting errors like:

STARTTLS=client: file /etc/ssl/private/sub.class1.server.ca.pem unsafe: Permission denied
STARTTLS=client, error: load verify locs /etc/ssl/certs, /etc/ssl/private/sub.class1.server.ca.pem failed: 0

These errors went away by taking the starttls.m4 and sasl.m4 macros out of submit.mc.

2009 Jul 12 - Sun

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 2

Now that email is inbound and being stored, now I need a mechanism of accessing it remotely. In the past I used courier-imap. Lately, the in-thing appears to be Dovecot. It appears to be fast, simple, and effective.

The Debian package repository is not really up-to-date, so I'll have to download the source and compile. The source is Dovecot v1.2.1. I usually put it into /usr/src and 'tar -zxvf ' it to expand the source. For configuring and compiling, I used:

./configure \
  --sysconfdir=/etc/dovecot \
  --with-storages=maildir \
  --localstatedir=/var/local/dovecot \
  --with-rundir=/var/local/dovecot/run \
  --with-statedir=/var/local/dovecot/state \
  --with-pam
make
make install

A user dovecot needs to be added with 'useradd -r dovecot'.

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 1

I am in the process of migrating and updating my email server to something bigger-better-faster. Last time I built an email server was a number of years ago on a Redhat system. Things have changed since then. During my re-learning process, here are some notes I've made on getting Sendmail and related processes on to a Debian Lenny system.

Once upon a time, Sendmail was the MTA (Message Transfer Agent) of choice. Most Linux operating systems used it by default. Currently it looks as though Exim and Postfix are now the primary choices for an MTA on the Debian flavour of Linux. Well, I can't let my Sendmail books go to waste, so I'm sticking with Sendmail as my MTA. In this installment, I describe some of the bits I needed for getting the Sendmail part installed and partially configured.

For the system, I did a basic install of Debian Lenny 5.0.1. When the package list came up, I unselected everything, including the Email and Standard System choices. That keeps the basic operating system foot print small.

Only a few packages are needed for Sendmail:

apt-get install libsasl2-modules
apt-get install libsasl2-modules-ldap
apt-get install sasl2-bin
apt-get install openssl
apt-get install ca-certificates
apt-get install build-essential
apt-get install libssl-dev
apt-get install libpam-dev
apt-get install sendmail

I had problems with the amd64 version of Debian Lenny 5.0.1 and sendmail. I was able to build everything, but the only thing that didn't work were the 'enhdnsbl' FEATUREs. I'll have to perform the build from scratch to see if I can recreate the problem. For now, just to get things done, I built the server with 32 bit i386 and the enhdnsbl FEATURE is functioning fine. (Note: after having rebuilt this in 32 bit mode and testing the enhdnsbl feature through the course of the build, I find that the problem occures due to the MailScanner requested DAEMON_PARMS setting in sendmail.conf. This problem is discussed further in my installment 4 of this series.)

To enable saslauthd, edit /etc/default/saslauthd and set START=yes (warning). Run '/etc/init.d/saslauthd start'

The package sensible-mda is installed along with sendmail. Sensible-mda is called by the MTA, and will in turn call whichever of the following MDAs that it finds (in this order): procmail, maildrop, deliver, mail.local.

In a previous installation, I used Courier's mail drop program to get messages into a MailDir format directory. It didn't work so well this time (it was very hard to troubleshoot as it turns off debugging information in local delivery mode). Instead, procmail can delivery to Maildir format directories, so I used that instead. To make this work, /etc/procmailrc needs the line DEFAULT=$HOME/Maildir/ .

To get things done the fast easy way, I'm simply storing email in ~/Maildir until I can get an LDAP mechanism up and running.

Maildir folders store email as one file per email. File locking requirements are reduced. Mbox files store all messages in one, possible large, single file.

Just so that the /home directory isn't completely shallow and wide, I edited the /etc/adduser.conf file and changed LETTERHOMES to yes. "The created home directories will have an extra directory - the first letter of the user name. For example: # /home/u/user."

I'll try this out on the next user I create, but I believe that by creating the directory Maildir in /etc/skel, 'touch /etc/skel/Maildir' and doing a 'chmod 740 /etc/skel/Maildir', the directory will automatically be available in the new users directory.

Instead of setting up a bunch of aliases for a bunch of email addresses that default to my standard email address, I created a virtusertable. The first lines provide explicit email address to local user mappings, something like

john@oneunified.net	john

The remainder of the file has entries like:

@oneunified.net		ray

The sendmail.mc file requires a corresponding 'FEATURE(`virtusertable')dnl' line.

I'm getting ahead of my self here, but for testing the configuration, commands can be sent to sendmail by telnet to port 25 or by creating a small test content file and sending a message with a command similar to 'sendmail ray@example.com < test.msg'. Content of test message:

to:Ray Burkholder 
from:Example 
subject:test from tester

test message

dnsbl resource seems to think that SpamHaus is pretty good as a DNS based BlackList source. I had been using a number of different sources, and I needed to make things current as some dnsbl sources have disappeared or turned unreliable. I've ended up using two sources, and spamhaus seems to prevent a very large chunk of spam getting further into my system, ie, a large percentage doesn't make it through the opening shots of the Sendmail pathways.

A DNS based Black List source (dnsbl) works by taking an email originator's ip address a generating a dns query to specialized spam black list site. Based upon the response to the query, mail can be accepted or rejected immediately, without further processing. A return code is simply a loopback address flavour, with an implicit 127.0.0.1 (an empty response) being a sign of a problem free address, and anything with 127.0.0.2 or greater signifying some issue with the address. More info can be found at Spamhaus.

The two dnsbl entries I use are:

dnl FEATURE(`enhdnsbl', `example.com', `"Spam block is hardcoded"', `t')dnl
FEATURE(`enhdnsbl', `zen.spamhaus.org', `"Spam blocked see: http://www.spamhaus.org/query/bl?ip="$&{client_addr}', `t')dnl
FEATURE(`enhdnsbl', `bl.spamcop.net', `"Spam blocked see: http://spamcop.net/bl.shtml?"$&{client_addr}', `t')dnl

Before using a dnsbl, be sure to read, understand, and conform to their terms of service.

To quickly test that the enhdnsbl FEATURE if functioning (assuming you have access to a dns server for example domain example.com):

• choose a machine from which you can telnet to sendmail on port 25
• determine it's ip address, say in this case, 10.23.43.5
• insert a line into the dns server similar to '5.43.23.10.example.com. IN A 127.0.0.2' (the address is backwards)
• uncomment the enhdnsbl FEATURE in the collection of 3 above, rebuild sendmail.cf, and reload sendmal
• telnet to the sendmail server, and you should see a 'ruleset=check_relay, arg1=[10.23.43.5], arg2=127.0.0.2, .... ' type line in mail.log

In the sendmail.mc file, I also disabled 'dnl FEATURE(`delay_checks', `friend', `n')dnl' (if it has been turned on by default) as it will accept a message, check the recipient, then perform the dnsbl lookup. This feature is for when you need to accept someone from a blacklisted address, but no one else. By disabling this, all users from the address are denied. In addition, with the option enabled, the mail.log file will have check_rcpt entries, with it disabled, the mail.log file will have check_relay entries.

To look at messages that have made it through Sendmail, have been locally delivered with procmail, a program called Mutt can be used to read the messages. By default Mutt, can read mbox mail files. A configuration change is required to read Maildir folders. The Mutt FAQ goes into more detail, but the basics are to put the folloing lines into ~/.muttrc:

set mbox_type=Maildir

set spoolfile="~/Maildir/"
set folder="~/Maildir/"
set mask="!^\\.[^.]"
set record="+.Sent"
set postponed="+.Drafts"

Richard Curnow has written a program to index, search, and create links to email messages stored in the Maildir folders.

During testing of my Sendmail configuration, from a email client, I was seeing a messages like the following:

sendmail dsn=5.0.0, stat=Service unavailable
554 5.3.0 rewrite: map access not found

It turned out to be an error in my sendmail.cm configuration file where I was missing a closing single quote. The file that processes a sendmail.mc file to create a sendmail.cf file is not very helpful in tracking down simple errors of syntax such as what caused this problem.

I don't know if it is legal or not, but I found online the Sendmail 3rd Edition. I don't know for how long the link will be valid.