One Unified Global Perspective
Communications with a Global Perspective
Contact Us
Voice over IP
PBX Solutions
Open Source


2009 Jul 24 - Fri

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 6

I've spent the last articles writing about getting an open source email server up and running. So far so good. My email logs show that a tremendous amount of spam is being blocked. One begins to wonder if there any real email remaining any more.

During the building of this server, a number of web sites provided useful information for troubleshooting and for configuration. I'm listing them here for reference before I close them out.

In some follow-up, I came across MailWatch, which is a web-based front-end to MailScanner written in PHP, MySQL and JpGraph and is available for free under the terms of the GNU Public License.

[/OpenSource/Debian/email] permanent link

2009 Jul 19 - Sun

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 5

A couple of articles ago, I started with a DoveCot Installation. I managed to download, build, and get a rough installation. I also prepared a userid for the service. It was at that point in the Dovecot installation instructions where they started talking about certificates, and I side-tracked into Certificate Authorities and certificate installation.

In /etc/dovecot, I copied dovecot-example.conf to dovecot.conf. In dovecot.conf, I updated the following lines to get things started:

protocols = imap imaps
disable_plaintext_auth = no
ssl = no
mail_location = maildir:~/Maildir
#mail_location = maildir:/%h/Maildir
auth_debug_passwords = yes

Dovecot Wiki does a good job of explaining the installation process. In fact, the non-ssl installation process is quite painless, and consists mostly of testing the connection.

Once the basic configuration is tested, then enable the configuration for ssl, and restart Dovecot.

disable_plaintext_auth = yes
ssl = yes
auth_debug_passwords = no
# Same keys from the sendmail installation
ssl_cert_file = /etc/ssl/private/
ssl_key_file = /etc/ssl/private/

Startup an IMAP session with a Mail Client and try IMAP and IMAPS. Try sending email as well through the SMTP Sendmail connection with encryption. Tcpdump can be used to look at packets.

There is a Sample Dovecot init.d script which can be used to start, stop, and reload the service. The sample can be pasted verbatim into /etc/init.d/dovecot. Also do a 'chmod 755 /etc/init.d/dovecot'. Then '/etc/init.d/dovecot start'.

With a successful send and receive of email, that wraps up the rather lengthy configuration of a reasonably protected email solution encompassing Sendmail as an email transport mechanism, Dovecot as an IMAP/IMAPS service, and MailScanner with SpamAssassin/F-Prot for email scanning and protection.

[/OpenSource/Debian/email] permanent link

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 4

It has taken a series of articles to get Sendmail installed and working with authentication, inline encryption, and some inline DNSBL capabilities. In this article, I'll see if I can get MailScanner, SpamAssassin and a virus scanner up and running with Sendmail.

Before starting into that though, I have a couple of links to other sites which have good information for tuning the file:

Back to the install. Starting with SpamAssassin, which looks like the last version is 3.2.5 from June of 2008, which is a Perl based utility, it can be downloaded from CPAN by starting the command line with 'perl -MCPAN -eshell':

install Bundle::CPAN
install Term::ReadLine
install MIME::QuotedPrint
install YAML
install YAML::Syck
install MIME::Base64
install Time::HiRes
install Digest::SHA1
install Net::DNS
install Mail::SPF
install IP::Country
install Net::Ident
install Mail::DomainKeys
install Mail::DKIM
install DBI
install LWP::UserAgent
install HTTP::Date
install Encode::Detect
install Mail::SpamAssassin

The pre-requisites build nicely, but the main Mail::SpamAssassin unit does not test well because it tries to start a daemon, which doesn't appear to do so. To find the reason will take some digging, but in the meantime, a force install may or may not be required. It probably is irrelevant anyway as MailScanner does not use spamd.

For a virus scanner, I've used f-prot in the past, and I'll try it again for this install. Others have used ClamAV, and I may add it as a secondary scanner. (Note, the file downloaded is a 64bit version). The last bit of the install script will ask if the daemon should be installed in crontab.... select no as MailScanner will it start it manually. Nor should Sendmail be configured to run the scanner.

cd /usr/src/
cd /opt
tar -zxvf /usr/src/fp-Linux-x86_64-ws.tar.gz
cd f-prot
fpscan /etc/passwd

Create a test file and put the EICAR virus into it. Run 'fpscan test' to ensure it finds the virus.

For MailScanner, the following Perl modules are required:

install Sys::Syslog
install Net::CIDR
install IO::Stringy
install Mail::Util
install File::Spec
install HTML::Tagset
install HTML::Parser
install MIME::Tools
install File::Temp
install Convert::TNEF
install Compress::Zlib
install Archive::Zip
install Check::ISA

Next steps:

cd /usr/src
tar -zxvf MailScanner-install-4.77.10-1.tar.gz
cd MailScanner-install-4.77.10

A few changes, like the domain name, may need to be changed in the /opt/MailScanner/etc/MailScanner.conf file.

Add the following with 'crontab -e' (the minute offsets may be randomized):

37      5 * * * /opt/MailScanner/bin/update_phishing_sites
07      * * * * /opt/MailScanner/bin/update_bad_phishing_sites
58     23 * * * /opt/MailScanner/bin/clean.quarantine
#42      * * * * /opt/MailScanner/bin/update_virus_scanners
#3,23,43 * * * * /opt/MailScanner/bin/check_mailscanner

In /etc/mail/sendmail.conf MailScanner install notes recommend changing 'DAEMON_PARMS="";' to:

DAEMON_PARMS="-ODeliveryMode=d -OQueueDirectory=/var/spool/";

Instead, use:

DAEMON_PARMS="-ODeliveryMode=background -OQueueDirectory=/var/spool/";

By default, Sendmail will use a Delivery Mode of Background, which operates by forking itself and processing the message. With a MailScanner Delivery Mode of Deferred, no DNS or DB lookups are performed. QueueOnly mode will actually perform DNS lookups, which is what I need for handling the SpamHaus enhdnsbl Features, but serializes all inbound connections. Queue mode sounds like the most straight forward option for working with MailScanner but may not be just right. I think that Background will work better, as it will fork and handle simultaneous connections. However, on further testing, I find that Sendmail delivers mail with Background mode, and queues it for Sendmail with QueueOnly mode, so QueueOnly mode it is.

Rerun /usr/sbin/sendmailconfig, then '/etc/init.d/sendmail restart' to get the mta agent and queue runner running as separate processes.

Add a 'crontab -e' entry to ensure MailScanner is always running:

0,20,40 * * * * [ -x /opt/MailScanner/bin/check_mailscanner ] && /opt/MailScanner/bin/check_mailscanner >/dev/null 2>&1

Edit the /opt/MailScanner/etc/MailScanner.conf file:

  • Set 'Virus Scanning' to yes
  • Set 'Virus Scanners' to f-port-6

Test the virus scanner with '/opt/MailScanner/lib/f-prot-6-wrapper /opt/f-prot eicar.virus'.

Restart MailScanner.

[/OpenSource/Debian/email] permanent link

2009 Jul 13 - Mon

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 3

In part two of this series, I started into the installation of the Dovecot IMAP service. The IMAP serivce can use validation and encryption through the use of SSL/TLS services. SSL/TLS services require the use of Certificates signed through a Certificate Authority. Many installation directions provide information for using the simple expedient of self-signed certificates. As some of these services I'm building are quasi-public, I wanted to go through the exercise of getting my certificates signed through a Certificate Authority. As such, I was side-tracked into doing some research to come up with two intermediate articles:

I'm going to step back to my SendMail install, and get a certificate installed in order to utilize SendMail's TLS based verification and encryption capabilities.

In the /etc/mail/ file, the following needs to be available (I've enabled AUTH as well):


Don't put these lines in the file as they will cause permission errors.

For configuring AUTH (SASL2), edit /etc/default/saslauthd and make sure 'MECHANISMS="pam"' is included and then start the service: /etc/init.d/saslauthd start. Shell users should now be able to authenticate, otherwise use /usr/sbin/saslpasswd2 to add users.

You cancheck in /etc/mail/tls to see various self-signed certificates which have already been created and linked within the configuration file /etc/mail/tls/starttls.m4. The various settings can be changed to match the new certificate. I changed the line with confCACERT to match my StartCom CA found in /etc/ssl/certs. I had placed my new server key and cert in /etc/ssl/private, and in, updated confSERVER_CERT and confSERVER_KEY to match.

Once the certificates are properly installed and SendMail restarted, it can be tested by connecting to telneting to port 25, running 'ehlo localhost' and looking for a line with '250-STARTTLS'. If it is there, all is well.

I found the page at SMTP STARTTLS in sendmail/Secure Switch to help somewhat in building the scenario.

For testing the STARTTLS capability, one can use the one of the following openssl commands (the first works better than the second):

openssl s_client -starttls smtp -connect localhost:25
openssl s_client -ssl3 -state -debug -msg -connect localhost:25

For other OpenSSL s_client command line parameters, visit: s_client man page.

At one point, I was getting errors in sendmail logs with:

STARTTLS=read: 12080:error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number:s3_pkt.c:284:
STARTTLS: read error=generic SSL error (-1), errno=104, 
  get_error=error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number, retry=1, ssl_err=1

I think these are permissions related depending upon privleges of certificate files and the username under which sendmail is running. Sendmail is now running under root and no longer has these problems. The errors magically disappeared during some restart so I can't confirm this for sure. ... further information: the errors happen when running the 'openssl s_client -ssl3 -state -debug -msg -connect localhost:25' command, but not the 'openssl s_client -starttls smtp -connect localhost:25'. I havn't spent the time to determine why yet.

I was also getting errors like:

STARTTLS=client: file /etc/ssl/private/ unsafe: Permission denied
STARTTLS=client, error: load verify locs /etc/ssl/certs, /etc/ssl/private/ failed: 0

These errors went away by taking the starttls.m4 and sasl.m4 macros out of

[/OpenSource/Debian/email] permanent link

2009 Jul 12 - Sun

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 2

Now that email is inbound and being stored, now I need a mechanism of accessing it remotely. In the past I used courier-imap. Lately, the in-thing appears to be Dovecot. It appears to be fast, simple, and effective.

The Debian package repository is not really up-to-date, so I'll have to download the source and compile. The source is Dovecot v1.2.1. I usually put it into /usr/src and 'tar -zxvf ' it to expand the source. For configuring and compiling, I used:

./configure \
  --sysconfdir=/etc/dovecot \
  --with-storages=maildir \
  --localstatedir=/var/local/dovecot \
  --with-rundir=/var/local/dovecot/run \
  --with-statedir=/var/local/dovecot/state \
make install

A user dovecot needs to be added with 'useradd -r dovecot'.

[/OpenSource/Debian/email] permanent link

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 1

I am in the process of migrating and updating my email server to something bigger-better-faster. Last time I built an email server was a number of years ago on a Redhat system. Things have changed since then. During my re-learning process, here are some notes I've made on getting Sendmail and related processes on to a Debian Lenny system.

Once upon a time, Sendmail was the MTA (Message Transfer Agent) of choice. Most Linux operating systems used it by default. Currently it looks as though Exim and Postfix are now the primary choices for an MTA on the Debian flavour of Linux. Well, I can't let my Sendmail books go to waste, so I'm sticking with Sendmail as my MTA. In this installment, I describe some of the bits I needed for getting the Sendmail part installed and partially configured.

For the system, I did a basic install of Debian Lenny 5.0.1. When the package list came up, I unselected everything, including the Email and Standard System choices. That keeps the basic operating system foot print small.

Only a few packages are needed for Sendmail:

apt-get install libsasl2-modules
apt-get install libsasl2-modules-ldap
apt-get install sasl2-bin
apt-get install openssl
apt-get install ca-certificates
apt-get install build-essential
apt-get install libssl-dev
apt-get install libpam-dev
apt-get install sendmail

I had problems with the amd64 version of Debian Lenny 5.0.1 and sendmail. I was able to build everything, but the only thing that didn't work were the 'enhdnsbl' FEATUREs. I'll have to perform the build from scratch to see if I can recreate the problem. For now, just to get things done, I built the server with 32 bit i386 and the enhdnsbl FEATURE is functioning fine. (Note: after having rebuilt this in 32 bit mode and testing the enhdnsbl feature through the course of the build, I find that the problem occures due to the MailScanner requested DAEMON_PARMS setting in sendmail.conf. This problem is discussed further in my installment 4 of this series.)

To enable saslauthd, edit /etc/default/saslauthd and set START=yes (warning). Run '/etc/init.d/saslauthd start'

The package sensible-mda is installed along with sendmail. Sensible-mda is called by the MTA, and will in turn call whichever of the following MDAs that it finds (in this order): procmail, maildrop, deliver, mail.local.

In a previous installation, I used Courier's mail drop program to get messages into a MailDir format directory. It didn't work so well this time (it was very hard to troubleshoot as it turns off debugging information in local delivery mode). Instead, procmail can delivery to Maildir format directories, so I used that instead. To make this work, /etc/procmailrc needs the line DEFAULT=$HOME/Maildir/ .

To get things done the fast easy way, I'm simply storing email in ~/Maildir until I can get an LDAP mechanism up and running.

Maildir folders store email as one file per email. File locking requirements are reduced. Mbox files store all messages in one, possible large, single file.

Just so that the /home directory isn't completely shallow and wide, I edited the /etc/adduser.conf file and changed LETTERHOMES to yes. "The created home directories will have an extra directory - the first letter of the user name. For example: # /home/u/user."

I'll try this out on the next user I create, but I believe that by creating the directory Maildir in /etc/skel, 'touch /etc/skel/Maildir' and doing a 'chmod 740 /etc/skel/Maildir', the directory will automatically be available in the new users directory.

Instead of setting up a bunch of aliases for a bunch of email addresses that default to my standard email address, I created a virtusertable. The first lines provide explicit email address to local user mappings, something like	john

The remainder of the file has entries like:		ray

The file requires a corresponding 'FEATURE(`virtusertable')dnl' line.

I'm getting ahead of my self here, but for testing the configuration, commands can be sent to sendmail by telnet to port 25 or by creating a small test content file and sending a message with a command similar to 'sendmail < test.msg'. Content of test message:

to:Ray Burkholder 
subject:test from tester

test message

dnsbl resource seems to think that SpamHaus is pretty good as a DNS based BlackList source. I had been using a number of different sources, and I needed to make things current as some dnsbl sources have disappeared or turned unreliable. I've ended up using two sources, and spamhaus seems to prevent a very large chunk of spam getting further into my system, ie, a large percentage doesn't make it through the opening shots of the Sendmail pathways.

A DNS based Black List source (dnsbl) works by taking an email originator's ip address a generating a dns query to specialized spam black list site. Based upon the response to the query, mail can be accepted or rejected immediately, without further processing. A return code is simply a loopback address flavour, with an implicit (an empty response) being a sign of a problem free address, and anything with or greater signifying some issue with the address. More info can be found at Spamhaus.

The two dnsbl entries I use are:

dnl FEATURE(`enhdnsbl', `', `"Spam block is hardcoded"', `t')dnl
FEATURE(`enhdnsbl', `', `"Spam blocked see:"$&{client_addr}', `t')dnl
FEATURE(`enhdnsbl', `', `"Spam blocked see:"$&{client_addr}', `t')dnl

Before using a dnsbl, be sure to read, understand, and conform to their terms of service.

To quickly test that the enhdnsbl FEATURE if functioning (assuming you have access to a dns server for example domain

  • choose a machine from which you can telnet to sendmail on port 25
  • determine it's ip address, say in this case,
  • insert a line into the dns server similar to ' IN A' (the address is backwards)
  • uncomment the enhdnsbl FEATURE in the collection of 3 above, rebuild, and reload sendmal
  • telnet to the sendmail server, and you should see a 'ruleset=check_relay, arg1=[], arg2=, .... ' type line in mail.log

In the file, I also disabled 'dnl FEATURE(`delay_checks', `friend', `n')dnl' (if it has been turned on by default) as it will accept a message, check the recipient, then perform the dnsbl lookup. This feature is for when you need to accept someone from a blacklisted address, but no one else. By disabling this, all users from the address are denied. In addition, with the option enabled, the mail.log file will have check_rcpt entries, with it disabled, the mail.log file will have check_relay entries.

To look at messages that have made it through Sendmail, have been locally delivered with procmail, a program called Mutt can be used to read the messages. By default Mutt, can read mbox mail files. A configuration change is required to read Maildir folders. The Mutt FAQ goes into more detail, but the basics are to put the folloing lines into ~/.muttrc:

set mbox_type=Maildir

set spoolfile="~/Maildir/"
set folder="~/Maildir/"
set mask="!^\\.[^.]"
set record="+.Sent"
set postponed="+.Drafts"

Richard Curnow has written a program to index, search, and create links to email messages stored in the Maildir folders.

During testing of my Sendmail configuration, from a email client, I was seeing a messages like the following:

sendmail dsn=5.0.0, stat=Service unavailable
554 5.3.0 rewrite: map access not found

It turned out to be an error in my configuration file where I was missing a closing single quote. The file that processes a file to create a file is not very helpful in tracking down simple errors of syntax such as what caused this problem.

I don't know if it is legal or not, but I found online the Sendmail 3rd Edition. I don't know for how long the link will be valid.

[/OpenSource/Debian/email] permanent link

New blog site at: Raymond Burkholder - What I Do

Blog Content ©2013
Ray Burkholder
All Rights Reserved
(519) 838-6013
(441) 705-7292
Available for Contract Work

RSS: Click to see the XML version of this web page.

View Ray 
Burkholder's profile on LinkedIn
Add to Technorati Favorites

Su Mo Tu We Th Fr Sa
20 21 22 23 24 25 26
27 28 29 30      

Main Links:
Monitoring Server
SSH Tools
QuantDeveloper Code

Special Links:

Blog Links:
Quote Database
Nanex Research
Sergey Solyanik
Marc Andreessen
Micro Persuasion
... Reasonable ...
Chris Donnan
Trader Mike
Ticker Sense
Stock Bandit
The Daily WTF
Guy Kawaski
J. Brant Arseneau
Steve Pavlina
Matt Cutts
Kevin Scaldeferri
Joel On Software
Quant Recruiter
Blosxom User Group
Wesner Moise
Julian Dunn
Steve Yegge
Max Dama

AprMay Jun
Jul Aug Sep
Oct Nov Dec

Mason HQ

Disclaimer: This site may include market analysis. All ideas, opinions, and/or forecasts, expressed or implied herein, are for informational purposes only and should not be construed as a recommendation to invest, trade, and/or speculate in the markets. Any investments, trades, and/or speculations made in light of the ideas, opinions, and/or forecasts, expressed or implied herein, are committed at your own risk, financial or otherwise.