Qmail Support

Note: This page is for historic reference only. Neither Qmail nor my patches for Qmail are under current development. A follow-up for Qmail is s/qmail which inherits all aspects of my development for Qmail (and much more).

Qmail

Qmail is a UNIX-based fast, secure, and uncompromised SMTP MTA.

System resources are moderate even under high E-Mail load and volume.

Qmail was developed by Dan J. Bernstein, and is well recommended internationally. In 2008 Dan Bernstein placed Qmail in the public domain.

D.J. Bernstein is Research Professor in the Department of Mathematics, Statistics, and Computer Science at the University of Illinois at Chicago and Professor at the Faculteit Wiskunde & Informatica at the Technical University of Eindhoven. Please have at look at his remarks on:

• Internet E-Mail
• Simple Mail Transport Protocol SMTP
• E-Mail Header

Qmail 1.03 was released in June 1998 and lacks support for some ESMTP enhancement which have been published since then, in particular SMTP Authentication and SMTPS/STARTTLS. However, some of the ESMTP enhancements are of very little use, in today's email world even counter-productive, and often weakly documented, as discussed in my

• SMTP Authentication tutorial and
• SMTP and Transport Layer Security (TLS) tutorial

which was originally planned as part of my Qmail book. In addition, I have set up a particular email discussion list to deal with the following issues:

• SPAMCONTROL: release announcements, additional usage information, forthcoming developments.
• QMVC: release announcements, usage, additional templates.
• NEWANALYSE: release information, particular customization.
• Other topics: New viruses / WARLORD, SMTP Authentication, STARTTLS usage, MAV, documentation.

You can participate in those discussions subscribing to the email list, which is run by EZMLM, of course.

Note: The email list has been moved from 'qmail' to 'sqmail'; my new development.

Support

Since 1998 I install and maintain Qmail and in addition Qmail's E-Mail list manager EZMLM on commercial and non-commercial base.

High volume (> 1 mio emails/day), high efficient, and low latency (< 10 sec/email) mail servers can be achieved with Qmail and my recommended extensions on current hardware. This allows in addition a high efficient spam and virus protection with little headache and administration efforts.

Development

Qmail's home page as maintained by Russel Nelson Qmail Homepage is unfortunately out-of-date. Over the past ten years not only the public recognition of email (and its abuse) has been changed, but in addition the requirements for a SMTP MTA. However, by means of my extensions for Qmail, I not only keep track of current developments, but in addition try to incorporate the requirements for large email sites.

Currently under investigation & development:

Some major paradigm changes need to be considered:

• Email needs to be considered as private communication to be transported encrypted. Exceptions are acceptable for not un-disclosed communication, i.e. mailing lists.
• Today's hardware is mostly 64 bit capable. While Qmail is written in ANSI-C, under some OS it won't compile it any more; or would fail invoking it (FreeBSD).
• Running large and offical/popular sites requires authentication of messages. Though neither SPF nor DKIM are in situ qualified for this, the new approach DMARC has been invented combining both.
• Though probably SMTP email will be the last protocol to require IPv6 on a larger scale, there is a growing need to support IPv6. Some current 'defense' mechanisms - like Relay Black Lists - need to support IPv6 as well.

My plans are to support fully IPv6 and TLS throughout. ucspi-tcp6 together with it's secure companion ucspi-ssl has already been finished; next is djbdnscurve6, completed by s/qmail which I consider as successor of vanilla Qmail.

Currently not supported:

• greylisting: postponed
• DKIM: Currently no plans (rblsmtpd ?)
• Bounce Address Tag Validation (BATV): No plans.

Some of the major patches/extensions are available for vanilla Qmail:

• Greetdelay (for rblsmtpd): Absorbing resources from Bot-Nets.
• RECIPIENTS extension (for qmail-smtpd): Recipients address verification.
• SMTP Authentication (qmail-smtpd/qmail-remote): LOGIN, PLAIN and CRAM-MD5.
• Warlord (qmail-smtpd): On-the-fly content scanner.
• QHPSI (qmail-queue): Ultra-fast AV scanner interface.
• Mail From Address Verification (qmail-smtpd): The 'Mail From:' address, you can trust.

Patches, mostly for qmail-smtpd:

SMTP HELO/EHLO Greeting delay

A flexible greetdelay approach is available in ucspi-tcp6. Within rblsmtpd a defined waiting delay can be defined after the SMTP client has initiated the SMTP session and before qmail-smtpd answers with "220 ESMTP". The delay is control as part of the rblsmtpd's options or via the environment variable GREETDELAY which is typically set by means of tcpserver's rule-set file. A value of GREETDELAY="30" will delay the qmail-smtpd's greeting for 30 seconds.

Time enough to discourage a typical Spambot's SMTP client from continuing the SMTP session (the session will be aborted by the client). Setting of GREETDELAY does not harm 'regular' mail clients, but will (currently) reduce Spam traffic by about 50%.

It has been argued, that the sleeping qmail-smtpd blocks local resources and it should be better to extend the algorithm to send the client a 55x reply (in case of protocol violations). However, one should keep in mind, that not only the resources on the mail server are used, but in addition those of the spam mail sender are blocked for that particular session. Thus, the GREETDELAY will not only save you for spam mails, but unlike Greylisting and/or filtering a la SpamAssassin, this is the only mean to really reduce the overall amount of spam because the timeslot required for the spam sender to deliver messages (whether successfully or unsuccessfully) is raised from typically one second to (<=) GREETDELAY seconds.

On the 2007th "Frühjahrsfachtagung" of the German Unix User Group (GUUG) in Berlin, I have given a brief presentation (in english) about pros'n'cons of GREETDELAY.

Note: In case you run redundant mail servers, you have to install the patch on every system.

A typical result looks like:

In case you are nosey about the efficiency, you can download the little script greetdelay.ksh. This script only works in conjunction with the patched rblsmtpd and SPAMCONTROL > 2.5.

After invoking the script like

you get the following output (sample from today):

You may use the options '-v' for verbose, '-t' to be verbose and to resolve the tai64 timestamp. Further, the input files can be provided 'stacked', e.g. '"@* current"' to analyse all multilog generated files. The speed of the analysis depends of course on the size of your log files and the DNS resolution.

RECIPIENTS extension (0.7)

qmail-smtpd accepts messages if the SMTP domain part of recipient address ("RCPT TO: ") matches an entry in control/rcpthosts or control/morercpthosts.cdb. The existence of a mailbox/maildir for the corresponding SMTP recipient is checked later in the delivery chain. In case no Mailbox/Maildir exists, the message is bounced back to the SMTP sender ("MAIL From: "). For normal SMTP mail traffic thats fine as long as the rate of undeliverable messages dont exceed 10% and the sender is 'legitmate'; ie. exists.
Todays situation is different: Spam and Virus attacks with forged/faked sender addresses to a bunch of random recipient addresses yield a undeliverable rate up to 90%. Worse: The generated bounces will never reach the sender and a double-bounce is eventually send to the postmaster.

There is an ongoing need to facilitate the existence of the Recipient's mailbox (and thus the "RCPT TO:" address) during the SMTP dialogue. For large sites, this is a complex task, since often multi-tier systems are involved, and the required efficiency can only be guaranteed at the borderline (= Internet) MTA.

From the usage and administrative point-of-view, for any email user one has to setup up:

• a valid email address
• a userid + password for SMTP Authentication
• a userid + password for POP3 Authentication
• a userid + password for IMAP4 Authentication
• a userid + password for Web access

The different authentication methods may require different security frameworks, which depend in particular on the client. Not only that this raises the need for a central repository of the mail user's attributes for administration purposes, in addition the email system should be able to use the repository. While my RECIPIENTS extension makes qmail-smtpd aware of acceptable recipients, the knowledge of those recipients has to be kept in a local database.

The RECIPIENTS extension (0.7.2) (MD5: a39b001c4c049893049cc15677d9dab0) drops this requirement and provides a checkpassword compliant API to allow practically any source to be queried, preferable a LDAP directory (as well as Microsoft's ADS), though a SQL database may be fine as well (feel free to write your own PAM!).

The following PAMs are supplied:

• (Easy customizable) ldap_pam.pl (here: version 0.9.2) allowing to access any LDAP server.
• qmail-smtpam to allow the verification/existance of a RFC 822 mailbox against a downstream SMTP server.

Features:

Scripts to generate recipients.cdb:

• A whitelist generator from David Du SERRE-TELMON.
• A python script to generate the "recipients" database out of several sources, contributed by Robert Sander.
• A Lotus-to-Recipients PERL script contributed by Norman Maurer from ByteAction GmbH.
• And most recently, Frank Tegtmeyer's Python script.

Note: These scripts are included unaltered without a particular 'fitness' check.
Please check yourself, if they fulfill your requirements.

Warlord (1.4.0)

These days, MTA's are flooded with virus E-Mails in high frequency, resulting in considerable delays in E-Mail communication. Out-of-band E-Mail Anti-Virus processing is cost-intensive and bounces are meaningless and even counter-productive for forged "MAIL FROM:" addresses.

Based on the idea of Russell Nelson's (and Charles Cazabon's) so-called "qmail-smtpd-viruscan-1.1.patch" to filter Windows executables attached as BASE64 encoded MIME parts in the E-Mail, I created a "Warlord" (MD5:5f77272038774d1b1a3cf164dcb682a5) enhancement for qmail-smtpd.

"Warlord" enables a wire-speed filtering of E-Mails containing BASE64 encoded attachments with about 99,5% efficiency. The Warlord algorithm employs

• a robust MIME type identification. Particular MIME types can be added on-the-fly in a badmimetypes.cdb and the badmimetype check is enabled, invoking the environment variable BADMIMETYPE,
• a high efficient and unique Loader type recognition. Specific Loader types (i.e. for the Windows OS) can be included in a badloadertypes.cdb and the badlodertype check is employed, if the environment variable BADLOADERTYPE is set with a characteristic pattern (i.e. BADLOADERTYPE='M'),
• additional capability to reject a BASE64 encoded attachment, in case it includes white spaces (spaces or tabs) which is typically the case for malware. In order to employ the check, use BADMIMETYPE='!' .

Note: The WARLORD algorithm has been made "public domain" in the Unix magazine iX (issue November 2004) and is therefore not patent-able.

QHPSI (0.2.0)

The Qmail High Performance Scanner Interface (MD5: Sfabf359926197ccc0e18498ba3a7b6e2) is a new approach to an old problem:

The QHPSI works exceptionally well with Clam AV's clamd/clamdscan. Simply use in your cdb :allow,QHPSI="clamdscan" and you are done. The entire power of QHPSI is exploited in SPAMCONTROL.

Version 0.9.x.y of ClamAV includes a bug, which prevents clamscan/clamdscan to log to STDERR. Here, you find a patch for clamav-0.9x.y (borrowing some code from 0.7x).

And I've implemented a new approach to cope with email sender (originator) verification:

'Mail From:' Address Verification (MAV)

There is an ongoing discussion, how to verify that a particular email sender (originator) is authorized to use a specific MTA (Mail Transfer Agent):

• MTAMark (the source has been purged),
• SPF,
• DKIM, and
• DMARC

are the most prominent candidates, though only DKIM is deployed more commonly.
However, in any case, the burden to verify the sender is put on the recipient MTA's shoulder.

Reversely, my proposal 'Mail From:' Address Verification tries to actively label the email to originate authoritatively from a certain MDA (identified per IP/Domain name) or an (authenticated) User (or both).

Unlike eg. SPF, there is no heavy-weight patching of Qmail necessary. By means of the simple plug-in MAV 0.20 (MD5: 9f1b33a4ab14ea03caa0a1d94d3605fa) qmail-smtpd can be made verification aware for local addresses.

Those extensions in combination are part of SPAMCONTROL:

SPAMCONTROL

SPAMCONTROL is a major extension for Qmail, in particular for qmail-smtpd to permit an advanced controlling of incoming emails with respect to the SMTP envelope and the data.

SPAMCONTROL provides a full featured SMTP AUTH and ESMTP STARTTLS support for qmail-smtpd and qmail-remote as well for qmail-pop3d based on Scott Gifford's extension and is highly integrated with ucspi-ssl (> 0.8).

A qmail-queue.scan script is available to act as front-end for qmail-queue allowing to run Spam and Virus checking on a RAM-Disk. Further, SPAMCONTROL includes (almost) all the Qmail enhancements presented above.

The current version of SPAMCONTROL is suited for the AMD64 architecture and compiles with MacOS X/BSD's clang.

Yet more Qmail add-ons:

Any service -- and in particular -- mission critical services like email shall be setup such, to work in an un-attended, error-free manner. Qmail together with supervise and multilog provides this kind of service. However, monitoring and reporting is important as well.

qmail-mrtg

The log information written by qmail-send, qmail-smtpd, and qmail-popup/qmail-pop3d (as patched with SPAMCONTROL) are written by qmail-mrtg in a format to be used and displayed by MRTG.

qmail-mrtg 3.02 (md5: 24c9fe643d0bcd09b3a3d6786bc0974c) is now available in version 3.0 and can be used to analyse multilog's log output for:

• qmail-send
• qmail-smtpd
• qmail-popup/qmail-pop3d

qmail-mrtg follows a modular concept and can easily be extended to include other information. qmail-mrtg allows to customize the feeding period such, only the most recent logfile - current - can be read even for very busy servers. qmail-mrtg can read simultaneously multiple logfiles produced by several instances of qmail-smtpd, i.e. running on different ports.
The current version 3.0.2 fixes some counting bugs.

Here you find a living sample.

Note: In the forthcoming release of s/qmail qmail-mrtg will be integrated by default.

• [FEHCom]
• [Impressum]
• [News]
• [top]