README_SPAMCONTROL (2.7)

Objectives

 


 

Objectives

SPAMCONTROL is an extension for qmail. It provides the following basic features:

Enhancements for qmail-smtpd:

Enhancements for qmail-remote:

Enhancements for qmail-pop3d:

Enhancements for qmail-queue:

Enhancements for qmail-send:

SMTP Pluggable Authentication Modules

Tow additional PAMs are now delivered with SPAMCONTROL:

Qmail Multiple Queue

This version of SPAMCONTROL provides support for the Qmail Multiple Queue (QMQ):

 

With SPAMCONTROL, qmail-smtpd can stand the two most common threats:

Additionally, qmail-smtpd allows

 


 

0. Conventions

0.1 Definitions

Sender = SMTP envelope sender (Mail From: <Return-Path>)
Recipient = SMTP envelope recipient (Rcpt To: <Forwarding-Path>)
Nullsender Mail = E-Mail with empty Sender (Mail From: <>)
Full qualified SMTP E-Mail address: "user@domain.com" (for Sender/Recipient)

0.2 daemontools run scripts

Throughout this document, I assume that qmail-smtpd is under control of supervise (out of the Daemontools package) and served by sslserver (part of the UCSPI-SSL package) or tcpserver (part of the UCSPI-TCP package).

Some skeletons for the daemontools run-scripts are provided which are useful to set up the services:

Note: These scripts are suggestions and need to be customized for your needs.

A typical - minimal - so called run script looks like follows:

#!/bin/sh
# qmail-smtpd startup
QMAILDUID=`id -u qmaild`
QMAILDGID=`id -g qmaild`
HOSTNAME=`hostname`
exec tcpserver -R \
      -u $QMAILDUID -g $QMAILDGID \
      -l $HOSTNAME 0 smtp \
      /var/qmail/bin/qmail-smtpd 2>&1

0.3 Environment Variables

Qmail - and SPAMCONTROL - relies on the concept of environment variables which are available for a task (sharing the same environment). qmail-smtpd may be fed by environment variables in three different fashions:

  1. (Gobal) Exported variables in the run script; eg. export RELAYCLIENT="", or export BADMIMETYPE="+".
  2. (Gobal) By means of the envdir facility as part of the Daemontools package.
  3. (Gobal) Using a "profile" configuration file called in the run script by means of the "dot" syntax (. pofile).
  4. (Individual) Setting the appropriate variable in the tcpserver cdb database.

While the first three cases define static and "global" environments variables, the last case makes the environment variables client-dependent and - by means of tcprules - dynamically changeable. Any mixture is possible, though only the "last" setting of an environment variable is effective!

0.4 tcpserver/sslserver cdb

As a convention, I will call the tcperver's cdb, which rules the behaviour of qmail-smtpd, tcp.smtp. A typical tcp.smtp would look like

=mydomain.com:allow,RELAYCLIENT="",LOCALMFCHECK=""
1.2.3.4:allow,RELAYCLIENT="",LOCALMFCHECK="otherdomain.com"
5.6.7.8:allow,UCSPITLS="!",BADLOADERTYPE="-"
:allow,MFDNSCHECK="",BADMIMETYPE="+",BADLOADERTYPE="M"

The cdb is constructed on the fly:

tcprules tcp.smtp.cdb tcp.smtp.tmp < tcp.smtp

Caution: For use with tcpserver, the value of the environment variable has to be included in quotes.

0.5 rcpthosts/morercpthosts.cdb

Though qmail can live happily without the knowledge of domains to be responsible for as provided by rcpthosts/morercpthosts.cdb, it is highly advisable to include all domains to receive emails for (as per DNS MX Records) into those control files. Otherwise, qmail-smtpd may act as an Open Relay. Further, some LOCALMFCHECKs will fail, as discussed below.

0.6 Pre-requisites

In order to compile and use SPAMCONTROL in the discussed way, you need to install the following packages prior setting up this software:

 


 

1. qmail-smtpd Extensions

1.1 Size Extension

qmail-smtpd's "Mail From:" parameter parser is used to detect and evaluate the SIZE parameter and to eventually reject messages which initially exceed the databytes limit.

Nevertheless, qmail-smtpd checks the size of the incoming message anyway.

For incoming E-Mails which exceed the message size values (in Bytes) defined in

or via the $DATABYTES environment variable.

1.2 SMTP Authentication

SMTP Authentication requires a Client to authenticate and a Server to honor the authentication procedure. In this version of SPAMCONTROL, Qmail acts as an Authentication Server for qmail-smtpd and as an Authentication Client for qmail-remote.

Usually, a MTA (such as Qmail) will accept transmissions of E-Mails anyway as long as the "Rcpt To: <forwarding-path>" is targeted to a local Recipient (according to control/rcpthosts). However, with SMTP Authentication you may allow an authenticated User to relay E-Mails. In this respect, SMTP Authentication copes with the deficiencies of the POP3/IMAP4 protocol and is applied as an alternative to SMTP-after-POP, which is ugly as well.

I have taken the SMTP-Auth Patch from Krzysztof Dabrowski and included this into SPAMCONTROL. However, SPAMCONTROL's implementation is compliant with the checkpassword API designed by Dan Bernstein (the Pluggable Authentication Module PAM).

SPAMCONTROL provides the following features:

1.2.1 Pluggable Authentication Module PAM

While SASL is a generic concept, the information flow for authentication between e.g. qmail-smtpd and the PAM is defined by Dan Bernstein's checkpassword API. SPAMCONTROL provides the PAM on file descriptor 3 as an informational string composed of:

  • The BASE64-decoded Userid followed by a 0,
  • The BASE64-decoded Password (Login and Plain) or Digest (CRAM-MD5) followed by a 0, and
  • The plain Challenge followed by a trailing 0, thus the PAM can reconstruct and validate the Digest in the CRAM-MD5 case.
  • You are free to choose or even write your own PAM program, but in any case, the SASL Procedure of the client and the server has to match and the procedure has to be advertised. Compliant PAMs:

    1.2.3 qmail-smtpd Setup for SMTP Authentication

    qmail-smtpd including SMTP Authentication may be called by tcpserver/sslserver in a supervise run script. Here is an example (with some more features):

    #!/bin/sh
    # qmail-smtpd startup with SMTP Authentication
    QMAILDUID=`id -u qmaild`
    QMAILDGID=`id -g qmaild`
    HOSTNAME=`hostname`
    export SMTPAUTH=""
    exec softlimit -m 2000000 \
    tcpserver -vR -l $HOSTNAME \
     -u $QMAILDUID -g $QMAILDGID 0 smtp \
       /var/qmail/bin/qmail-smtpd /bin/cmd5checkpw true 2>&1

    Beware! Unlike the original implementation, I omitted the inclusion of the Hostname as argument for qmail-smtpd.

    Unlike the standard qmail-smtpd, now you have

    1. define the environment variable SMTPAUTH to allow SMTP authentication and
    2. to provide in addition a PAM program, here cmd5checkpw, which itself calls a shell named 'true' (/bin/true or /usr/bin/true) exiting simply with "0".

    1.2.4 Tailoring Authentication

    The environment variable SMTPAUTH can be used to specify the type of SMTP Authentication. Your choices are:

    SMTPAUTH Meaning
    "" Left blank to allow Authentication types "PLAIN" and "LOGIN"
    "+cram" Add "CRAM-MD5" support
    "cram" Just (secure) "CRAM-MD5" support, no other types offered
    "!" Enforcing SMTP Auth (of type "LOGIN" or "PLAIN")
    "!cram" Enforcing SMTP Auth of type "CRAM-MD5"
    "!+cram" Enforcing SMTP Auth of type "LOGIN", "PLAIN" or "CRAM-MD5"
    "-" Disabling SMTP Auth (for a particular connection)

    Unlike the Submission feature, these settings may be realized in the tcprules cdb per connection.

    1.2.5 Submission Port

    SMTP clients requiering SMTP authentication expect the SMTP server to listen to the Submission port 587 instead of the standard SMTP port 25. To use this feature, you need to set up a second qmail-smtpd instance bound to port 587:

    #!/bin/sh
    # qmail-smtpd startup with SMTP Authentication
    QMAILDUID=`id -u qmaild`
    QMAILDGID=`id -g qmaild`
    HOSTNAME=`hostname`
    export SMTPAUTH="!"
    exec softlimit -m 2000000 \
    tcpserver -vR -l $HOSTNAME \
     -u $QMAILDUID -g $QMAILDGID 0 submission \
       /var/qmail/bin/qmail-smtpd /bin/cmd5checkpw true 2>&1

    If used in conjuntion with the previous run-script [1.2.3], qmail-smtpd will accept SMTP authentication sessions on port 25 and 587; however demanding a successful authentication on the submission port, while providing a fall-back to none-authentication on the standard SMTP port.

    Now, the Submission feaure requires the additional declaration of the SMTPAUTH with a leading ! exclamation mark instead of the SUBMISSION variable, which has been removed.

    1.2.6 User Database for cmd5checkpw

    For SMTP Authentication, a User Database has to be generated and maintained. The SMTP Authentication User may exist independently of any System Users, Qmail Users, or E-Mail Accounts. In case of the modified cmd5checkpw I decided to keep the User in the Qmail directory as

    There exist other flavors, in particular the saslpasswd scheme or the Cyrus SASL Library you may want to use. Further, for users with POP3/IMAP4 Accounts on the system it is advisable to use a common User Database. For Vpopmail you may use vchkpw.

    However, since you are free to use any other checkpassword compliant PAM, it's up to you whatever you apply. Please remember: In order to access the Unix /etc/passwd the respective program has to run as root.

    1.2.7 SMTP Authentication and Vpopmail

    SMTP Authentication works well with vpopmail, however, you have to use a checkpassword compatible PAM. Older versions of vchkpw have to be patched accordingly (see http://www.fehcom.de/qmail/smtpauth.html).

    vchpkw offers a lot of authentication capabilities; it supports login, plain, and CRAM-MD5 and may authenticate the user against a mysql database and others. In start-up script for qmail-smtpd you have to make sure to access the user database with the correct user access rights:

    #!/bin/sh
    # qmail-smtpd startup with SMTP Authentication + vpopmail
    QMAILDUID=`id -u vpopmail`
    QMAILDGID=`id -g vpopmail`
    HOSTNAME=`hostname` export SMTPAUTH="crammd5"
    exec softlimit -m 2000000 \
    tcpserver -vR -l $HOSTNAME \
     -u $QMAILDUID -g $QMAILDGID 0 smtp \
       /var/qmail/bin/qmail-smtpd /home/vpopmail/bin/vchkpw true 2>&1

    If you use Sqwebmail in addition, the user is free to set his/her own password.

    1.3 (START)TLS support

    SPAMCONROL's STARTTLS support for qmail-smtpd is aligned with Scott Gifford's approach and depends on the following:

    1. Superscript's sslserver (0.8x) instead of tcpserver -- and --
    2. the availability of a valid X.509 certificate, an appropiate key, and additionally a Diffie-Hellman parameter file,
    3. the correct feeding (via environment variables) of those settings to sslserver to allow encryption.
    Usually, TLS support is provided by two different usage schemes:

    My STARTTLS implementation conforms with RFC 3207 but lacks support of

    These functions need to be implemented in sslserver however are of little benefit for email encryption by means of TLS.

    1.3.1 (START)TLS implementation

    Most current STARTTLS/TLS solutions depend on the existence and availability of the OpenSSL libraries -- so does SPAMCONTROL. However, unlike other implementations, qmail-smtpd is insulated against OpenSSL by means of sslserver. In fact, all encryption and certificate verification is facilitated by sslserver. In this respect, Scott's and my STARTTLS implementation is very much OSI-like. The communication and presentation happens at a well defined environment, typically assigned to the user and group ssl. Any potential attacks or bugs are kept away from the application and don't harm.

    The reading and response to client cerificates and the actual encryption happens in the assigned user spaces; which should never be root.

    1.3.2 Prereqs

    Install ucspi-ssl (version >= 0.80).

    Further, it is helpful to create a low privileged user and group ssl, which will be used by sslserver for SSL/TLS communication purposes. Please follow Scott Giffords' advices.

    1.3.3 TLS Envrionment

    Apart from the 'global' environment variable UCSPITLS sslserver are fed by several environment typically included in a "profile" /var/qmail/ssl/env.

    On my system, this file includes the the following settings:

    # Set
    SSL_USER=ssl
    SSL_GROUP=ssl
    SSL_DIR="/var/qmail/ssl"
    # Rest
    SSL_CHROOT="$SSL_DIR"
    CERTCHAINFILE="/var/qmail/ssl/chainfile"
    CERTFILE="/var/qmail/ssl/cert"
    KEYFILE="/var/qmail/ssl/key"
    DHFILE="/var/qmail/ssl/dhparam"
    SSL_UID=`id -u $SSL_USER`
    if [ $? -ne 0 ] ; then echo "No such user '$SSL_USER'" >&2 ; exit; fi
    SSL_GID=`id -g $SSL_USER`
    if [ $? -ne 0 ] ; then echo "No such group '$SSL_GROUP'" >&2 ; exit; fi
    export SSL_UID SSL_GID SSL_CHROOT
    export CERTFILE KEYFILE DHFILE

    Of course it is required, to have raised the directory /var/qmail/ssl before and to generate via openssl the appropriate files before.

    Note: These settings are 'global'; however, by means of sslserver and the settings in your tcp.smtpd file it is possible to use different certificates per connection.

    Comment: Please read the documentation of UCSPI-SSL carefully w.r.t. the "mod-ssl" variables. It might in addition be necessary to define CAFILE, CADIR and other SSL options to your needs.

    After you verified your settings, restart qmail-smtpd. Whether qmail-smtpd will present "STARTTLS" in the EHLO dialogue, depends on the presence of the UCSPITLS environment variable. These can be set i.e. per IP in the tcp.smtpd control file.

    1.3.4 STARTTLS Settings

    Substitute tcpserver with sslserver in the run script for qmail-smtpd. If you use softlimits, it might be necessary to raise those settings significantly due to the increased memory requirements. Here is my run script:

    #!/bin/sh
    QMAILDUID=`id -u qmaild`
    QMAILDGID=`id -g qmaild`
    HOSTNAME=`hostname`
    export SMTPAUTH="crammd5"
    export UCSPITLS=""
    MAXCONCURRENCY=`cat /var/qmail/control/concurrencyincoming`
    . /var/qmail/ssl/env
    exec softlimit -m 180000000 \
    sslserver -sevn -l $HOSTNAME -c $MAXCONCURRENCY \
    -x /var/qmail/etc/tcp.smtpd.cdb \
    -u $QMAILDUID -g $QMAILDGID 0 smtp \
    /var/qmail/bin/qmail-smtpd cmd5checkpw true 2>&1

    It is absolutely necessary to use the "-n" flag for sslserver, since this will trigger the availability of encrypted communications channels between sslserver and qmail-smtpd.

    1.3.5 SMTPS Settings

    Unlike the SMTP+STARTTLS the SMTPS setup facilitates a TLS connection immediately the TCP handshake is finished and thus the entire SMTP session dialoge is protected against espionage and by the same token not vulnerable against sabotage (see: VU#555316).

    Setting up qmail-smtpd on the SMTPS port is easy:

    #!/bin/sh
    QMAILDUID=`id -u qmaild`
    QMAILDGID=`id -g qmaild`
    HOSTNAME=`hostname`
    export SMTPAUTH="crammd5"
    export UCSPITLS="" MAXCONCURRENCY=`cat /var/qmail/control/concurrencyincoming`
    . /var/qmail/ssl/env
    exec softlimit -m 180000000 \
    sslserver -sev -l $HOSTNAME -c $MAXCONCURRENCY \
    -x /var/qmail/etc/tcp.smtpd.cdb \
    -u $QMAILDUID -g $QMAILDGID 0 smtps \
    /var/qmail/bin/qmail-smtpd cmd5checkpw true 2>&1

    Please realize also, that sslserver is called without the option "-n" thus no delay is required. In this case, sslserver and qmail-smtpd communicate over the standard file descriptors entirely.

    1.3.6 Flexible STARTTLS settings per Client

    Once UCSPITLS ist set and exported, qmail-smtpd accepts STARTTLS connections. However, it might be necessary to customize this behavior per sending MTA:

    While typically UCSPITLS='' is set qmail-smtpd's run file, you can redefine this variable in sslserver's tcp.smtpd control file per inbound connection.

    1.3.7 TLS Client verification by means of a X.509 certificate

    Typically, the ESMTP client connecting to the server does not provide a X.509 certificate. In this case, the ESMTP connection is still TLS encrypted and the client might use the server's certificate for verification/validiation purpose.

    However the server may request a valid X.509 certificate from the client. In this case, the (ssl)server needs to have the certificate of the CA (Certificate Authority) which has signed the client's certificate in the first place (in PEM format).

    1. Make sslserver aware of the CA certificate by means of setting the environment variable $CACERT accordingly or include the 'hashed' certificate in a directory available via $CADIR.
    2. Include the IP or the FQDN of the specific SMTP client into tcp.smptd database and define and populate the environment variable $CCACERT with the path to the specific CA X.509 certifiate (in PEM) format.
    3. Call sslerver with the option '-i' and '-h' (default) to interrogate the X.509 certificate of those clients which have additionally the environment variable $CCACERT set and to lookup the FQDN in the DNS.

    sslserver will not only ask for the client certificate but will verify it by means of the CA certificate and validate it's content matching the received FQDN with the presented 'SubjectNameAltName' and perhaps the 'Distinguished Name' in the client certificate.

    In case, any of the checks fail, the connection attempt will be refused.

    1.3.8 (START)TLS and SMTPAUTH

    In case qmail-smtpd is instructed to use STARTTLS and SMTPAUTH, SMTP Authentication always takes place after the TLS session is active, but never reverse. Thus, all SMTP parameters like username and password are already encrypted. Of course, SMTP Authentication is still available for unencrypted SMTP connections, and STARTTLS does not require per se SMTP Authentication. However, STARTTLS and SMTP Authentication is a strong and powerful couple to secure the SMTP communication.

    1.3.9 (START)TLS Received Header Extensions

    SPAMCONTROL displays the use of TLS in the Received header (according to RFC 3207). The following information is added:

    Here's a sample using Thunderbird als email client:

    Received: (qmail 35450 invoked from network); 15 Mar 2006 20:22:09 -0000
    Received: from dornroeschen.fehnet.de (HELO ?192.168.192.19?) (erwin@192.168.192.19)
    encrypted via TLSv1: RC4-MD5 [128/128] DN=unknown
    by orion.fehnet.de with ESMTPSA; 15 Mar 2006 20:22:09 -0000

    Note: The received DN ist typically 'unknown' because the client doesn't provide a certificate.

    In the above case the keyword ESMTPSA tells that the connection was established via SMTPS and additionally was authenticated A: (E)SMTPS(A).

    1.3.10 Client Certificates and Relaying

    With version 2.7.31 qmail/SPAMCONTROL supports SMTP client authentication by means of X.509 client certificates. How to achieve this ?

    1. Each SMTP client needs to be provisioned with a valid X.509 certificate. This certificate typically is a user certificate and includes the email address as part of the Distinguished Name (DN) or as part of the 'Subject Alternative Name' (SAN).
    2. The client certificates need be be signed by a CA (called the Client CA, CCA), for which the root certificate needs to be present and deployed to the SMTP server (in our case: sslserver).
    3. Both, the SMTP server and the SMTP client need to be aware to request/respond to client certificate requests. In case of sslserver, the option -z> is dedicated for this operation.
    4. For sslserver this means to provide the root certificate by providing CCAFILE=path_to_certificate. For the client this requieres usually the setting 'Authentication=TLS Certifiate'.

    SMTP client authentication is a private mean; thus does not require an 'official X.509' certificate. Self-signed are good enough. Practically, one may use 'official' certificates provded by means of $CERTFILE, while the path to $CCAFILE may include serveral concatinated X.509 root certs against those the clients are checked.

    In case, serveral authentication domains are required -- which have their own CA -- those CAs can be 'stacked' into one PEM file presented by means of $CCAFILE. If the client cert matches one, the client is authenticated.

    Since the SSL environment variables are fed from sslserver to qmail-smtpd by means of MOD_SSL, qmail-smtpd has only knowledge of the Distinguished Name and it's component, i.e. the email-address. Unfortunately, their is no way in inquire the SAN component of a X.509 V3 certificate. Thus, in oder to verify the provided email address in the SMTP envelope (the Return-Path) against the address in the X.509 cert, only the DN can be evaluated.

    It is important to understand the roles of CA and CCA certs:

    Thus, for practical reasons, CA and CCA certs need to be identical, where CCA certs are a subset of CA certs (which may be taken from a CADIR).

    In order to support X.509 user certificates by sslserver it needs to be called with the option '-m'. Thus, sslserver posses three different understandings about client certificate:

    1. -Z (Default): Don't request a client certificate.
    2. -z: Request for client certificates and check the provided hostname in the cert against the FQDN (if given).
    3. -m: Request a client certificate but don't check the hostname.

    sslserver hast two means to control the client certificate request:

    1.3.11 Client Certificate Authentication

    Setting up qmail-smtpd for Client Certificate Authentication inflicts with standard SMTP Authentication. Thus, once qmail-smtpd recognizes a valid client certificate (either using SMTPS or STARTTLS), it disables SMTP Auth by default.

    As a consequence, the MUA does not need to posses a valid userid/password. In case the client cert is accepted, it reacts as RELAYCLIENT. The DN in the cert is considered as TCPREMOTEINFO.

    However, the provided Return-Path address in the 'Mail From:' may be checked against the email address in the client's DN. Use LOCALMFCHECK='?' to enforce a mapping of both.

    1.4 MAV: "Mail From: Address Verification"

    Mail From: Adress Verification (MAV) is a mean to enforce the use of the SMTP "Mail From:" address for particular Relayclients. Former versions of SPAMCONTROL used a "LOCALMF" check which allowed only a very limited granularity. However, with MAV you can control/enforce

    MAV is in particluar very useful if emails from your domains have to be undoubtly "officially" send.

    1.4.1 Requiring MAV

    Mail From: Address Verification is only be done if the flag 'relayclient' is set. This flag is set if

    For these circumstances MAV can be enforced by means of the environment variable LOCALMFCHECK:

    1.4.2 Control files mfrules and mfrules.cdb

    The file control/mfrules follows roughly the same syntax as the common file tcp.smtpd used for tcpserver/sslserver. It assigns either a complete SMTP address, a FQDN, an IP adress or a domain to a set of allowed Originator addresses. In practice control/mfrules allows

    Once you have populated control/mfrules run qmail-mfrules to derive control/mfrules.cdb from the input file. Additionally, define LOCALMFCHECK="!" either gobally or in the tcp.smtpd file.

    1.4.2. Reply to the relayclients

    Mail Clients may be setup wrongly or a user may want to use the relaying MTA to send emails for a different name. In case, MAV is in place and well configured, the particular user will not be allowed to send the mail over the gateway receiving the following SMTP reply:

    Since this might not be helpful for the (innocent) sender, you might use the environment variable REPLYMAV to add a qualification to that message.

    1.4.3 MAV SMTP protocol extension

    MAV puts the burden of SMTP Originator address verification ot the relaying MTA; that is the reverse scheme compared to SPF and others. Emails qualified through MAV are labeled with "ESMTPM" in the Received header, which is generated by qmail-smtpd.

    1.5 Customizable SMTP Reply Messages

    qmail-smtpd reads serveral environment variables which allows the customization of the SMTP Reply messages:

    Those (customized) variables and thir specific content can be defined in a single file smtpreplies (sample attached) to be loaded from the run script for qmail-smtpd.

    For instance, qmail-smtpd will send a SMTP 554 Error Reply under the following conditions:

    The SMTP Reply code for the first three conditions is always "554 sorry, invalid message content (#5.3.2)". The rejection of email because of the message content is due to some internal policy. For those users, which are subject of this policy innocently (and did not send ie. a virus mail on purpose), it might be advisable to explain the company's email policy here.

    The environment variable

    allows to include a particular SMTP 554 Reply. Typically, an URL might be referenced: REPLY_CONTENT="[ see: http://www.fehcom.de/emailpolicy.html]" which allows to detail possible circumventions.

    1.6 Interrogating SMTP envelope variables

    For special purposes, it might be necessary to have knowledge of the SMTP envelope information while qmail-smtpd is running. SPAMCONTROL (>= 2.5.26) puts the following informations into the environment -- as soon as available:

    The provided qmail-queue.scan script makes use of these variables. You may also populate the DELIVERTO environment variable -- read by qmail-smtpd -- to include additional recipients.

    In case of the SSL/TLS session further environment variables a la mod_ssl are available.

    1.7 X-RBL-Info Header

    qmail-smtpd will include the information received by a Relay-Blacklist (i.a. Spamhaus) into the header of the message, once the environment variable $RBLSMTPD is available and filled with some information.

    Here is a sample:

    Received: (qmail 9477 invoked from network); 7 Feb 2013 03:39:16 -0000
    Received: from 79-101-71-86.dynamic.isp.telekom.rs (79.101.71.86) by mail.fehcom.net with ESMTP; 7 Feb 2013 03:39:16 -0000
    X-Rbl-Info: http://www.spamhaus.org/query/bl?ip=79.101.71.86
    This information can be used by a MUA/LDA to filter the mail.

    Note: Currently only ucspi-tcp6 supports this with it's rblsmtpd program invoked in interrogation mode.

    1.8 SPF-Header

    qmail-smtpd is able to use received SPF (Sender Policy Framework) information to display those in the email header:

    Received-SPF: pass (example.com: domain of mydomain.com
        designates 192.168.1.2 as permitted sender) receiver=erwin
        client-ip=192.168.1.2; envelope-from=<me@fehnet.de>

    The implementation is able to include all cases of SPF results ('+' pass, '-' fail, '~' softfail, '?' neutral, 'o' none, 't' temperror, 'e' permerror); however depends on a feed of SPF variables. The idea is to extract the SPF information in a forefront process, like rblsmtpd since the retrieval and extraction of the SPF information is cumbersome and to put those values into the environment for later processing.

    For this purpose, SPAMCONTROL includes an API:

    Note: Except for result any of these variables may be empty/unsupplied.

    Currently, I'm not aware of any SPF retrieval and feeding process, though their exists a patch for ucspi-tcp including a SPF lookup. However, considering the demand this kind of information -- which can be understood as a prototype even for the competing DKIM algorithm -- this should be understood as starting point.

     


     

    2. qmail-smtpd SMTP Envelope Filters

    The SMTP Envelope consists of three parts:

    SPAMCONTROL allows to filter E-Mails according to the bad* criteria with a so-called wildmat search, which is a subset of the known Regualar Expressions (RegEx). The wildmat search works in order least significant to most significant and includes

    1. a case-insensitive straight search - and -
    2. a case-sensitive wildmat search

    The following sets of wildmat control characters can be used:

    2.1 SMTP Envelope Addresses

    Any E-Mail address, lets say <user@host.domain.com> consists of a

    local part: user (left from the "@") and a
    host part: host.domain.com (right hand side from the "@").
     
  • qmail-smtpd converts hostnames and domainnames taken from the tcp-env/tcpserver environment to lowercase (see man addresses).
  • qmail-smtpd reads the SMTP envelope address inside the recommended SMTP envelope brackets "<" and ">".
    • Originally, SMTP addresses without the brackets are accepted; and are terminated if a blank is seen " ",
    • however with SPAMCONROL (and RFC 2821) brackets are mandatory.
    • Double-quotes in SMTP addresses "forwarding path" are stripped.
  • E-Mail addresses for local accounts are considered case-insensitive and delivered irrespective of their case.

    Lets say - if the local account is "admin" and the RCPT to: tells <AdMin> or <adMIN> the delivery will be successful.

    2.2 Filtering HELO/EHLO Greetings (badhelo)

    RFC 2821 says: "These commands (HELO/EHLO) are used to identify the SMTP client to the SMTP server. The argument field contains the fully-qualified domain name of the SMTP client if one is available. In situations in which the SMTP client system does not have a meaningful domain name (e.g., when its address is dynamically allocated and no reverse mapping record is available), the client SHOULD send an address literal (see section 4.1.3), optionally followed by information that will help to identify the client system. y The SMTP server identifies itself to the SMTP client in the connection greeting reply and in the response to this command."

    Qmail records the HELO/EHLO greeting string for every received message in the E-Mail "Received:" header in case the provided HELO/EHLO string is different from the connecting hosts FQDN:

    Received: from foo.bar.de (HELO foo) (192.168.192.11)
    by heaven.bar.de with SMTP; 25 Apr 2003 15:01:42 -0000

    The HELO/EHLO string is included as "(HELO foo)". The HELO/EHLO string is usually generated by the sending MTA without much control (MUAs often use their generic hostname).

    SPAMCONTROL allows a flexible filtering of the clients HELO/EHLO greeting string, which depends on the setting of the environment variable HELOCHECK:

  • HELOCHECK="": evaluate control/badhelo control file
  • HELOCHECK="!": reject session, if no HELO/EHLO greeting is not provided/empty.
  • HELOCHECK=".": reject session, if no HELO/EHLO greeting is provided/empty and evaluate control/badhelo control file
  • HELOCHECK="=": require that the HELO/EHLO greeting corresponds to the FQDN (= TCPREMOTEHOST) of the host.
  • HELOCHECK="A": DNS A lookup for the HELO/EHLO greeting and evaluate control/badhelo.
  • HELOCHECK="M": DNS MX lookup for the HELO/EHLO greeting and evaluate control/badhelo.
  • The HELOCHECKs are only done, in case RELAYCLIENT is not set (split-horizon fashion). In my current setup, a useful setting is HELOCHECK="." and with the following input in control/badhelo

    myIPAddress
    myFQDN
    localhost
    localhost.localdomain

    These settings exclude the spoofing of the MTA's own address, which is typical for spam senders, since they determine the EHLO/EHLO greeting from the initial IP/SMTP session parameter.

    2.3 Filtering Mail From: SMTP Addresses (badmailfrom)

    SPAMCONTROL allows four types of checks against the provided "Mail From:" SMTP envelope address (which I often call the "Originator"):

    2.3.1 DNS MX Checks of the Envelope Sender

    Invoking the environment variable MFDNSCHECK in the qmail-smtpd startup script enables globally the DNS check for the envelope's Sender.

    Example:

    #!/bin/sh
    export MFDNSCHECK=""
    .....

    Additionally, the environment variable may be defined individually within a cdb of tcpserver/sslserver. Typically, this is done for "non-trusted" hosts within a tcpservers cdb:

    127.0.01:allow,RELAYCLIENT=""
    :allow,MFDNSCHECK=""

    If environment variable MFDNSCHECK is not set, qmail-smtpd does not perform this DNS MX check.

    Note: All DNS checks are either done by means of the libresolv library which comes with BIND, or my means of DJBDNS's routines, which can be included installing DJBDNS and using Nikola Vladov's enhancements for DJBDNS in addition with the modified Makefile.djbdns.

    2.3.2 Standard badmailfrom Checks

    control/badmailfrom was the only SMTP envelope filter Dan Bernstein originally implemented for qmail-smtpd. Here, only particular names or perhaps domains were listeted to be rejected in the SMTP dialogue. Since then, various flavours of badmailfrom have been brought out. However, the approach to reduce spam emails feeding control/badmailfrom with known spammer addresses is comparable trying to hit a moving target. Almost all Originator addresses spammer use today are fake and in this sense are meaningless.

    2.3.3 badmailfromunknown Checks

    There exist a special case, where you expect an email with a specific Originator address to be send via particular MTAs. For instance, if you see an email with Originator address "support@microsoft.com", it has to be send from a Microsoft MTA. qmail-smtpd has the knowledge of the sender's IP and FQDN (by means of the environment variables TCPREMOTEIP and TCPREMOTEHOST) in case you use tcp-env, tcpserver, or sslserver with the appropriate argument, i.e. tcpserver -h.

    MTAs for which the FQDN can't be resolved are unqualified. In particular, emails from the large webmail providers (aol, hotmail, yahoo, gmx, t-online ...) have always to be send from qualified MTAs. Reversely, you can safely reject emails with those Originator hostparts, which can not be resolved tcpserver/sslserver records them as "unkonwn".

    With SPAMCONTROL's badmailfrom implementation, you simply include the Originator addresses for which you enforce a qualified TCPREMOTEINFO into control/badmailfrom in the following way appending a dash ("-"):

    @aol.com-
    @hotmail.com-
    @yahoo.com-
    @gmx.de-

    Note 1: Since tcp-env/tcpserver/sslserver relies on a qualified DNS lookup, it is certainly helpful to use DJBDNS' dnscache as frontend.
    Note 2: Wildmat support is not provided; thus an entry "@*.yahoo.com-" won't work.

    2.3.4 badmailfromwellknown Checks

    In particular for webmailer (ie. hotmail.com, yahoo.com) the domain-part of the provide Mail From: address coincides with the provided domain name in TCPREMOTEHOST. Enforcing coincidence can be achieved for addresses appended with an equal-sign ("=") in control/badmailfrom:

    @t-online.de=

    Receiving a Mail From: address like "max.mustermann@t-online.de" will only be accepted if TCPREMOTEHOST ends with "t-online.de", for instance "mailout03.t-online.de". Otherwise, the email will be rejected by badmailfrom.

    2.3.5 badmailfrommismatcheddomains

    Reversely to badmainfromwellknown now the MTA and it's FQDN is considered in the first place.
    Thus, if you receive emails from mx1.yahoo.com you may require that the provided Mail from: address matches yahoo.com.

    In order to enforce this, within control/badmailfrom you define extended (domain) addresses with a leading similar-sign ("~"):

    ~yahoo.com

    Caution: It should be noted, that this filter is not in accordance with RFC 2821, since it couples the SMTP envelope with the domain of the sender. For example, yahoo uses DKIM records to label allowed envelope addresses for their domain. Further, this entry does not affect the 'Mail From:' address in the first place, but rather the value of TCPREMOTEHOST.

    2.3.6 badmailfrom Anti-Spoofing

    Another special case is given, rejecting none-Relayclient emails with Originator addreses spoofing your domain name or email addresses. Email can be rejected if the "responsible domains" are included with a trailing plus ("+") in the following way into control/badmailfrom:

    god@mydomain1.com+
    @mydomain2.net+
    @mydomain3.org+

    2.3.7 badmailfrom Test by-passing

    Under some conditions, it might by necessary to by-pass all reqular badmailfrom checks and in addition the MFDNSCHECK. For this purpose, individual email addresses included in badmailfrom and enhanced with a leading question mark ("?") can be defined. While recognizing this address, qmail-smtpd will skip all further tests:

    ?nobody@example.com

    Note: The enhanced addresses don't allow wildcarding.

    2.4 Controlling the Rcpt To: SMTP dialogue and Filtering the Recipient Address

    Apart for the RECIPIENTS mechanism, which is detailed later, you can reject SMTP Recipient addresses (Rcpt To: <Recipient>) by means of control/badrcptto. However, qmail-smtpd lets you effectively

    Note: The provided Rcpt To: <Recipient> information by the SMTP client is (apart from it's IP/FQDN) the only information which can not be faked, though these addresses are today often randomly generated by means of lexical/dictionary attacks by spammers or gathered by address harvesting. Standard qmail will accept any addresses which matches an entry in control/rcpthosts or control/morercpthosts.cdb and in case the Recipient does not exist tires to bounce the email to the Originator after control/qeueulifetime has exceeded (default one week).

    2.4.1 badrcptto

    By populating control/badrcptto you reject emails to Recipients listed there in already in the SMTP session. Wildcards are allowed. If you don't wont to receive emails for root (from the Internet) include in control/badrcptto:

    root@*

    2.4.2 badrcptto Whitelisting

    Alternatively to the Recipients mechanism, as a side-effect of the wildmat filtering, you can use the control/badrcptto file as an effective whitelisting mechanism. The trick is, to initially reject everything while later to allow specific Recipients:

    *
    !*@otherdomain.com
    !user1@mydomain.com
    !user2@mydomain.com

    Note: The evaluation of control/badrcptto is done independent from the setting of the RELAYCLIENT environment variable.

    2.4.3 Restricting the number of Recipients

    The environment variable

    MAXRECIPIENTS

    can be used to restrict the number of counted "Rcpt To: "s in the SMTP session. By default, no restriction is facilitated.

    2.4.4 Tarpitting

    I have included Chris Johnson's TARPITTING patch into SPAMCONTROL:

    "What is tarpitting? It's the practice of inserting a small sleep in a SMTP session for each "Rcpt To:" after some set number of "Rcpt To:"s. The idea is to that spammers who would hand your SMTP server a single message with a long list of RCPT TOs. If a spammer were to attempt to use your server to relay a message, say, 10,000 Recipients, and you inserted a five-second delay for each Recipient after the fiftieth, the spammer would be 'tarpitted', and would most likely assume that the connection had stalled and give up."

    Typically, the environment variables TARPITCOUNT and TARPITDELAY are set by menas of tcpserver's default-allow rules:

    :allow,TARPITCOUNT=5,TARPITDELAY=20.

    TARPITCOUNT denotes the number of sessions before starting the TARPITDELAY, which defaults 5 seconds.

     


     

    3. Recipients Extension

    3.1 Scope

    qmail-smtpd accepts messages if the SMTP domain part of Recipient address ("Rcpt to: <recip@domain>") matches an entry in control/rcpthosts or control/morercpthosts.cdb. The existence of a mailbox/maildir for the corresponding SMTP Recipient is checked later in the delivery chain. In case no Mailbox/Maildir exists, the message is bounced back to the SMTP Sender ("Mail From: <send@example.com>").

    For normal SMTP mail traffic that's fine as long as the rate of undeliverable messages don't exceed 10% and the Sender is 'legitmate'; ie. exists. Today's situation is different: Spam and Virus attacks with forged/faked Sender addresses to a bunch of random Recipient addresses yield an undeliverable rate up to 90%. Worse, the generated bounces will never reach the Sender and a double-bounce is eventually send to the postmaster.

    3.2 qmail-smtpd Recipients

    The RECIPIENTS extension makes qmail-smtpd aware of acceptable recipients, which are fetched from an external source.
    Which source to query depends on the domain-part of the recipient address.

    The RECIPIENTS check is done only in a none-RELAYCLIENT case and after control/rcpthosts, control/morercpthosts.cdb has been successfully consulted.

    3.2.1 VERP Support

    The RECIPIENTS mechanism supports natively Qmail's address extensions (VERP). If a recipient address like 'foo@mydomain.com' defined, all VERP addresses like 'foo-bar@mydomain.com' are accepted for SMTP reception.

    3.2.2 Domain recognition and Wilddomain Support

    The RECIPIENTS lookup is triggered by the recipient domain, thus is domain-specific. You can specify which lookup is performed
    per domain within control/recipients. Consider the following:

    Compatibility Note: Due to this new syntax, the old RECIPIENTS (version 0.4x) wilddomain support (as part of the cdb) is not supported anymore.

    3.2.3 Fail-Open Operation

    The RECIPIENTS extension can be used in a 'fail-closed' or 'fail-open' mode for the domains included in control/recipients.
    Typically the recipient check is done 'fail-closed', thus if all queries are negative, the incoming email with this recipient address will be rejected.

    A 'fail-open' behaviour can be achieved adding '!*' as last statement in control/recipients.
    Thus, emails for domains not listed in control/recipients will finally be accepted.

    3.2.4 Smart Rejections/Tarpitting Reloaded

    Defining a TARPITCOUNT can be used to terminate the SMTP session if the number of invalid Recipients ("Rcpt to:") exceeds the TARPITCOUNT. Unlike the typical tarpitting mechanism, this is a hard limit (Smart Rejection).

    However, defining a TARPITCOUNT together with a TARPITDELAY < 0 acts as 'tarpitting reloaded':

    3.3. Setting up the recipients control file

    Release 0.5 the RECIPIENTS extension provides a flexible new syntax to interprete control/recipients on a domain base, as part of the RCPT TO: envelope address.

    Lines in control/recipients starting with a '#' are not evaluated, thus are treated as comment lines.

    Read 'man qmail-smtpd' and 'man qmail-recipients.' Some additional scripts can be found in doc.

    3.4. Generating a cdb with recipient addresses

    Verify that list to be found under users/recipients.

    If you have a different Qmail home directory, modify the above scripts.

    You may need to change "localhost" in the above scripts to the real hostname.

    3.5. Requirements for a checkpassword compatible PAM

    The checkpassword API is defined in:

    http://cr.yp.to/checkpwd/interface.html

    and typically consists of the string:

    username\0password\0timestamp\0otherdata\0

    written to file descriptor 3 (FD 3) to be read by the checkpassword compatible PAM.

    For email address (recipient) verification, we replace

    username\0

    with

    email-address\0

    ie.

    recipient@domain.tld\0

    The attached PERL ldap_mail.pl serves as a sample.

    Note: The PAM has to be either in the Unix $PATH or explicitely defined in control/recipients.

    3.6. Calling a PAM from the recipients file

    The general syntax invoking a PAM from control/recipients follows the principals invoking a cdb. However, the triggering token is a '|' and the PAM may be accompanied by at most five (5) arguments separated by white spaces:

    @mydomain.com|/usr/local/bin/ldap_pam.pl -h:ldap.mydomain.com -d:admin -p:secret
    customer.com|bin/qmail-smtpam exchange.customer.com
    global.net|/usr/local/bin/ldapam -h:ldap.global.net -d:admin -p:topsecret
    !*

    3.7. The ldap_pam.pl

    The ldap_pam.pl is a PERL script using the PERL library 'Net::LDAP' from http://www.cpan.org while providing simple and TLS encrypted (strong) binds to an LDAP directory server.

    3.7.1 Syntax of the ldap_pam.pl

    ldpap_pam.pl -h:host:port -d:DN -w:password -b:base -s:scope -c:certificate -l(ogging) -ll

    The ldap_pam.pl shall be considered as prototype only. Due to the large variety of LDAP servers and possibilities to store the information into and to fetch it from the directory, there is the need for specific customization. Logging might be activated in two steps.

    3.7.2 Fetching the RCPT TO: information from the LDAP directory

    Customization for the LDAP server:

    3.8. qmail-smtpam

    The additional module qmail-smtpam is a generic part of SPAMCONTROL and can be considered as stripped-down version of qmail-remote providing the first steps of the email dialoge using the standard Helo message, a Nullsender Mail From: address, and finally the provided the Rcpt To: address available on FD 3.
    Once it receives the SMTP return code, it exits either with '0' in case of a '250' positive reply, a '1' in case of a failure, or a '111' realizing communication problems.

    qmail-smtpd thus uses a standard SMTP dialoge without advanced features, though complying to RFC 821 while using the canonical name for host to connect to.

    3.9 Customization

    The Recipients extension needs no customization except for the following circumstances:

  • Compile-time options:
  • Run-time options:
  • Optional scripts:
  • 3.10 Results

    With the Recipients extension qmail-smtpd will act for none-RELAYCLIENTs like follows

    a) The domain part of the Recipient address is checked against rcpthosts/morercpthosts.cdb and accepted if either it matches or the domain is not provided.
    b) the Recipient address is checked against all readable sources as listed in control/recipients and accepted if

    In any other case, a SMTP temporary failure protocol error is issued to the client saying:

     


     

    4. Filtering E-Mails on behalf of the Message Content

    Based on the "qmail-smtp-viruscan-1.1.patch" by Russell Nelson (and Charles Cazabon), SPAMCONTROL includes my WARLORD extension, which is a much robuster and efficient filter for BASE64 encoded MIME attachments and bundled with the Qmail High Performance Scanner Interface (QHPSI):

  • BASE64 encoded MIME attachments are detected and can be filtered accordingly to an easily extendable badmimetypes file.
  • Within the BASE64 encoded MIME attachments so-called specific loader assignments (ie. for Windows OS) can be detected and messages - containing these suspicious badloadertypes patterns - are rejected.
  • Additional and optional on-the-fly scanning of emails through the QHPSI.
  • Initial badmimetype or badloadertype flagged messages are not subject of the QHPSI.
  • Employing the environment variable BASE64, QHPSI is advised to by-pass virus scanning if no BASE64 encoded attachment is found.
  • In case a badmimetype or badloadertype filter condition is met or a virus is detected, qmail-smtpd sends a SMTP 554 reply to the sender "554 sorry, invalid message content (#5.3.2)". Populating the REPLY554 environment variable, allows to include additional information (typically an URL), which can be used to deal with potential false-positives.

    4.1 The BADMIMETYPE-Filter

    The badmimetype filter becomes active if

  • the environment variable BADMIMETYPE="" is set
  • and the control file badmimetypes is populated and readable by qmail-smtpd.
  • Reversely, if BADMIMETYPE="-" is supplied, the check is disabled. This is useful for particular hosts as provided within smtpd.tcp.cdb.
  • The badmimetypes check can be conditionally disabled for RELAYCLIENTS (eg. once authenticated), in case BADMIMETYPE="+" is set. Thus here, the badmimetypes check is only performed in a none-RELAYCLIENT case (standard incoming SMTP traffic). Unlike the "-" flag, this can be used as a global setting.
    Note: Setting BADMIMETYPE="+" without a cdb is equivalent to BADMIMETYPE="!" for none-RELAYCLIENTS.
  • 4.2 Control file badmimetypes and badmimetypes.cdb

    The control file control/badmimetypes.cdb is populated by the additional program qmail-badmimetypes which takes the input of control/badmimetypes. New MIME signatures can be added/removed on-the-fly. Bad MIME Type signatures have to have the length of at least 9 significant characters.

    The currently included MIME signatures are:

    TVqQAAMAA
    TVpQAAIAA
    TVpAALQAc
    TVpyAXkAX
    TVrmAU4AA
    TVrhARwAk
    TVoFAQUAA
    TVoAAAQAA
    TVoIARMAA
    TVouARsAA
    TVrQAT8AA
    # *.zip
    # UEsDBAkAA
    # *.z (gnu-zip)
    # H4sIADWWb
    # double Base 64 Windows Executable
    VFZxUUFBT
    # triple Base 64 Windows Executable
    VkZaeFVVR
    # Pif File
    TVoAAAEAA
    # Bagle Virus
    ZGltIGZpb

    Adding new badmimetypes is simple:

    1. Send an E-Mail to a Unix account with corresponding attachment (i.e. *.zip).
    2. Use an editor to view the E-Mail and spot the corresponding BASE64 encoded content-type.
    3. Take the first nine significant characters (for type "zip" its "UEsDBAkAA") and include them into control/badmimetypes.
    4. Run qmail-badmimetypes.

    Comments (starting with "#") are allowed in badmimetypes; the length of the signature will be truncated to nine characters.

    4.3 The BADLOADERTYPE-Filter

    The badloadertype filter becomes active if

  • the environment variable BADLOADERTYPE="M" is set (see below)
  • and the control file badloadertypes is populated and readable by qmail-smtpd.
  • As exception, you can define BADLOADERTYPE="-" to disable the check. This is useful for particular hosts as provided within smtpd.tcp.cdb.
  • The badloadertypes check can be conditionally disabled for RELAYCLIENTS (eg. once authenticated), in case BADLOADERTYPE="+" is set.
  • The BADLOADERTYPE mechanism deals in particular with "transport stealth" worms, ie. UPX encoded Windows executables.

    4.4 Control file badloadertypes and badloadertypes.cdb

    badloadertypes.cdb is populated by the additional program qmail-badloadertypes which takes the input of control/badloadertypes The badloadertype mechanism looks for five significant strings in the BASE64 encoded data-stream which is matched against an entry in control/badloadertypes.cdb. badloadertype signatures can be added/removed on-the-fly.

    The currently included Windows OS badloadertype signatures are:

    Mi5kb
    MzIuZ
    MyLmR
    MyLkR

    Comments (starting with "#") are allowed in badloadertypes; the length of the signature will be truncated to five characters.

    Caution: Unlike the badmimetype, the badloadertype signatures are placed anywhere in the BASE64 encoded datastream and are difficult to find out. In order to make the search efficient, a common character has to be providen in the environment variable BADLOADERTYPE. The provided pattern look basically for a string like "32.dll" as a subpart of "Kernel32.dll" which is an indication for an executable for the Windows OS. However, there is a small chance for false positives. Some - lets say - Word document attached as BASE64 MIME part in the message containing the buzz words "kernel32.dll" might become flagged and finally rejected as well.

    4.5 Employing an AV Scanner with QHPSI

    Unlike all other AV Scanners currently in use for Qmail, with Qmail High Performance Scanner Interface (QHPSI) there is no need for any other umbrella program, neither qmail-scanner, AMAViS, qscanq or whatsoever. Further, no additional MIME analyzing program like reformime, metamail, or ripmime is required. Even better, no "staging" area for temporary files are needed, except the one, the AV Scanners requires for itself.

    Today's AV Scanners - and in particular Clam AV - are able to read the BASE64 encoded message and eventually dig out the files in archives, ie. in zip format. In order to use an AV Scanner with QHPSI, the AV Scanner has to have the following qualifications:

    The QHPSI allows to use the following environment variables:

    The AV Scanner is directly called in the start scripts of Qmail (i.e. the run script for qmail-smtpd) or by means of tcpserver's capabilities. Here is a typical example, how to customize QHPSI together with Clam AV (clamd/clamdscan) for a tcpserver tcp.smtpd file:

    :allow,QHPSI='clamdscan',QHPSIARG1='--disable-summary'

    Comments:

    Note:

    #!/bin/sh
    export QHPSI='clamdscan'
    export QHPSIARG1='--disable-summary'
    exec /var/qmail/bin/qmail-queue

    4.5.1 QHPSI SMTP Reply messages

    Results:

    4.5.2 QHPSI logging

    Here is a sample of Clam AV without and with the argument "--disable-summary":

    @... tcpserver: pid 49943 from 192.168.192.11
    @... tcpserver: ok 49943 qmailer.fehnet.de:192.168.192.2:25 arkon.fehnet.de:192.168.192.11::1074
    @... /var/qmail/queue/mess/4/89439: Worm.Klez.H FOUND
    @...
    @... ----------- SCAN SUMMARY -----------
    @... Infected files: 1
    @... Time: 0.099 sec (0 m 0 s)
    @ Reject::DATA::Virus_Infected: S:192.168.192.11:arkon.fehnet.de H:mail.fehnet.de F:me T:erwin 'clamdscan'

    Note: Even in case no virus is detected, the "SCAN SUMMARY" is provided.

    @... tcpserver: pid 49989 from 192.168.192.11
    @... tcpserver: ok 49989 qmailer.fehnet.de:192.168.192.2:25 arkon.fehnet.de:192.168.192.11::1077
    @... /var/qmail/queue/mess/4/89543: Worm.Klez.H FOUND
    @ Reject::DATA::Virus_Infected: S:192.168.192.11:arkon.fehnet.de H:mail.fehnet.de F:me T:erwin 'clamdscan'
    @... tcpserver: end 49989 status 256

    Attention:

    Note: As with this writing, clamav 0.8x is broken, since it writes all logs to STDOUT instead of STDERR; thus no scanning messages will apear in the qmail-smtpd log.

    4.5.3 QHPSI performance improvements for Virus Scanning

    The badmimtypes and badloadertypes mechanism provides a wire-speed filtering of incoming emails. However, typically all not-filtered emails are subject of the AV Scannner as defined via the QHPSI. Almost all worms and virii are transported as BASE64 encoded attachments (except some trojans, encapsulated as HTML files). By means of the environment variable

    one can advice QHPSI to scan only those emails which contain a BASE64 encoded attachment.

    4.6 Qmail QUEUE_EXTRA

    Bruce Guenter's Qmail QUEUE_EXTRA patch has almost the rank of a recommended patch, because it's used by many Qmail extensions like the Qmail-Scanner and qmail-qfilter.

    The actual use is controlled via the content of environment variable "QMAILQUEUE", which usually set in a tcpserver's cdb ie. tcp.smtp or globally defined in the qmail-smtpd's run script. A typical use is:

    12.34.56:allow,RELAYCLIENT=""
    :allow,QMAILQUEUE="bin/qmail-qfilter"

    which advices qmail-smtpd to use the executable qmail-qfilter as first stage queueing program instead of qmail-queue itself.

    In order to reject during the SMTP DATA phase, vanilla qmail requires that the qmail-queue replacement returns RC=31.
    However, SPAMCONTROL enhances this mechanism with the following additional return codes:

    Those return codes can be extensibly used in a qmail-queue wrapper-script (see below).

    Note: The QUEUE_EXTRA patch is not applied against qmail-smtpd but rather against the module qmail.c itself, since it is just an extension to the general queue call-mechanism.

    4.6.1 qmail-queue.scan Replacement

    The qmail-queue.scan script can be used by the QUEUE_EXTRA mechanism to allow a per (recipient) domain

    For high volume/high performance scanning, the incoming message is copied to a tmp directory which typically should be raised on a ramdisk. All scanning actions can be realized now in memory which significantly reduces disk I/0.

    You can define individual SpamAssassin detection thresholds per domain using the additional control file:

    Here, you include the recipients's domains, followed by a colon, and the spam threshold for this domain:

    example.com:10
    yourdomain.com:8

    SPAMCONTROL's qmail-smtpd is able to understand the (tagged) messages identified as infected or as spam and will issue a useful SMTP return code (see below).

     


     

    5. qmail-remote Extensions

    SPAMCONTROL modifies and extends the behavior of qmail-remote in the following ways:

    5.1 Chose outgoing IP Address per Domain

    While qmail-smtpd can be bound (via tcpserver/sslserver) to any available IP address on the system, qmail-remote uses only the 'canonical' IP address of the system. However, running qmail for differnt domains, it is possible to customize the behavior of qmail-remote depending on the 'Mail From:' addresse. We consider the domain-part of the 'Mail From:' address as senddomain. The file

    advises qmail-remote to bind to senddomain-specific IP addresses and perhaps use the provided Helohost name as greeting:

    example.com:192.168.0.1|smtp.mydomain.com
    *.localhost.com:10.1.0.199

    Check the existing IP addresses qmail-remote can bind to by means of the qmail command ipmeprint.

    The first line shows an example, where qmail-remote will be adviced use the string smtp.mydomain.com als EHLO/HELO greeting instead of the default.

    This is in particular useful hosting serveral domains and if you set up your email system in the DNS with different SPF records matching the FQDN of the IP. Further, if you use client certificates for authenticated TLS connections you are now able to match the (outgoing) IP with the Subject(Alt)Name in the X.509 client certificate.

    5.2 STARTTLS and SMTPS support for qmail-remote

    qmail-remote uses the SSL routines provided by UCSPI-SSL and can be advised to set up a TLS session with the ESMTP peer in the following ways:

    1. A usual STARTTLS session only makes use of the encryption available by means of the OpenSSL libraries.
    2. A legimated STARTTLS / SMTPS session allows you to verify the server's X.509 certificate against a known CA.
    3. A particular STARTTLS / SMTPS session allows you to verify the server's X.509 certificate and to validate the provided 'Subject' in the SAN/DN against the hostname retrieved from the DNS. This can be used for server authentication.
    4. A client-authenticated STARTTLS / SMTPS session requires to have a valid client X.509 certificate to be requested by the ESMTP server application and presented on a peer-2-peer base. In consequence, the CA certificate is (mutually) used (1) for signing the client certificate and (2) to verify the client certificate at the server side.

    You can control the TLS connectivity behavior of qmail-remote by means of

  • control/tlsdestinations
  • STARTTLS support for qmail-remote becomes effective, simply populating this file with

    *:
    meaning 'use TLS encryption for any peer offering STARTTLS support, or in case of connecting to port 465 use SMTPS'. Alternatively, 'Anonymous Diffie-Hellman' is requested and no X.509 certificate is evaluated by means of:

    -*:|aNULL:!RSA

    This control file is separated into a 'destination domain' the left side of the colon and a (structured) informational part right from the colon employing '|' the bar character for separation and perhaps a colon for the port in the last position:

    destination:cafile|cipher|verifydepth:port|senddomain
    Destination

    cafile is the full-qualified path name to the CA certificate file in PEM format. This certificate file can be build of several X.509 certs. If however, cafile ends with a slash "/", a CADIR is assumed hosting the certificates in a "hashed" format.

    The expression for the accepted cipher can be constructed individually. Check the OpenSSL information for the syntax and the accepted values.

    The verifydepth parameter defaults 1 and should be adjusted to the nesting of the certificates in the chain.

    In case port equals 465 the SMTPS port, STARTTLS is not tried but rather a standard TLS connection is required.

    Finally, an entry in control/tlsdestinations can be bound against the the senddomain in case several outgoing domains are used. Thus, an entry is only valid, if it matches the senddomain.

    The file control/tlsdestinations is evaluated in the following order (for the same domain, the more specific information is taken) with additional customization samples included:

    1. !nosslhost.example.com:
    2. =myfriend.com:/etc/ssl/cacert||2
    3. mx1.example.com:/etc/ssl/cafile|:465
    4. .example.com:/etc/ssl/cadir/|!SSLv2:!aNNULL
    5. -.adh.org:|aNULL:!RSA||myexample.org
    6. *:||TLSv1+HIGH:!SSLv2:RC4+MEDIUM:!aNULL:!eNULL:!3DES:@STRENGTH

    Thus, you can specify particular hosts/domain to set up a STARTTLS connection to, - or - exclude particular hosts by means of an prepended exclamation mark (!) following the hostname.

    5.2.1 Legitimated STARTTLS / SMTPS sessions

    Since the ESMTP server always presents a X.509 certificate, this can used for additional interrogation:

  • A verification of the ESMTP server's X.509 certificate can be accomplished against a (set of) CA' certificates available in a PEM file or in a particular directory (with 'hashed' PEM file names).
  • The request for particular TLS Ciphers.
  • The acceptable VERIFYDEPTH for the server's certificate (Default: 1).
  • A particular port to connect to (ie. 465 for SMTPS) following a (last) colon.
  • 5.2.2 Particular STARTTLS / SMTPS sessions

    Here, the server's X.509 certificate can be checked:

  • A hostname prepended with a '=' (equal) sign will validate the presented 'SubjectName' / 'SublectAlternativeName (SAN)' against the received FQDN of the ESMTP server.
  • A verification of the ESMTP server's X.509 certificate can be accomplished against a (set of) CA' certificates available in a PEM file or in a particular directory (with 'hashed' PEM file names).
  • The request for particular TLS Ciphers.
  • The acceptable VERIFYDEPTH for the server's certificate (Default: 1).
  • A particular port to connect to (ie. 465 for SMTPS) following a (last) colon.
  • 5.2.3 Authenticated STARTTLS / SMTPS qmail-remote sessions

    On the ESMTP's server request, qmail-remote will provide (if available) a X.509 client certificate. The server may evaluate this certificate and set credentials (ie. relaying), accordingly.

    This information can be provided per senddomain (see domainips) by means of the control file

  • control/domaincerts
  • which is structured comparable to control/tlsdestinations. Thus the part left from the colon tells the sending domain; in case this equals "*" all outgoing connections use the same certificate/keyfile. The right parts provides the location and the certfile (in PEM format), the keyfile (if any) and perhaps the password to decrypt the keyfile; all separated by a '|':

    senddomain:certfile|keyfile|password

    5.3 qmail-remote Authentication

    The qmail-remote authentication from Bjoern Kalkbrenner has been included in a modular and RFC-complient version. qmail-remote sessions can be SMTP authenticated with the types PLAIN, LOGIN and CRAM-MD5 on a now be realized

    1. by sender (Reverse-Path) or
    2. alternatively, by destination.

    Mechanism 1:

    Authentication for outgoing SMTP sessions is faciliated, if the control file

    is populated accordingly. Sample:

    mail@example.com|test|testpass
    info@example.com:smtp.example.com:26|other|otherpw
    :mailrelay.example.com|e=mc2|testpass

    In the first case, an email with SMTP envelope sender Mail From: 'mail@example.com' tries to authenticate always with username 'test' and password 'testpass'. In the second case emails from the sender 'info@example.com' are tries to relay through 'smtp.example.com' on port 26 with authentication username 'other' and password 'otherpw'. The third sample shows, that any outgoing email irrespectively of the sender is relayed thru 'mailrelay.example.com' with authentication username 'e=mc2' and password 'testpass'.

    Mechanism 2:

    Instead of binding the authentication procedure to a specific user rather you can define it per destination, defining the control file

    accordingly:

    .example.com:smtp.example.com:587|other|otherpw
    :mailrelay.example.com|e=mc2|testpass

    This syntax follows the smtprules interpretation. The first lines shows how emails for 'example.com' is relayed through 'smtp.example.com' on port '587' (Submission) using the authentication usernaem 'other' and password 'otherpw'. In the second case any email is relayed by means of 'mailrelay.example.com' with authenticated username 'e=mc2' and password 'testpass'.

    5.4 qmail-remote MX connectivity

    Typically sites/domains on the Internet are reachable over serveral MTA listed and deployed in the DNS MX records (o.k. qmail.org is an exception). By theory, the MX with the smallest weight is the primary MX for that domain; though often sites have redundant MTA with equal weights:

    10 mailc.microsoft.com
    10 maila.microsoft.com
    10 mailb.microsoft.com

    In order to deliver emails qmail-remote follows two strategies:

    In case this MTA is exhausted and rejects the connection during the EHELO/HELO greeting, qmail-remote exists and retries the very same MTA again with it's quadratic queue schedule mechanism.

    Running EZMLM with many messages to the vary same domain but different Recipients, email delivery may become throttled, which particularly happens for t-online.de sites which don't allow too many connections from the same client MTA (a policy which is actually not covered by any SMTP RFC).

    Back 10 years ago, when Dan was designing qmail he already was aware of that problem:

    "If I successfully connect to an MX host but it temporarily refuses to accept the message, I give up and put the message back into the queue.
    But several documents seem to suggest that I should try further MX records. What are they thinking? My approach deals properly with downed hosts, hosts that are unreachable through a firewall, and load balancing; what else do people use multiple MX records for? " (THOUGHTS)

    and included already the code base into qmail-remote which I simply activated. Thus, in case qmail-remote receives a rejection during the EHLO/HELO greeting it will simply try the next MTA for the DNS MX list.

    After the MX lookup qmail-remote needs to fetch the CNAME from the DNS. Here, Dan decided (probably due to a bug in BIND 4.x) to employ a ANY query. This has some consquences:

    In essence, the ANY query is a heavy burdon on the DNS traffic and MAY result in the famous qmail-remote error message: "CNAME temporary lookup failure". It has been suggested, to replace the ANY query with a standard CNAME query, but I hesitate:

    Therefore, I added a verbose CNAME message into qmail-remote logs, thus you are able to figure out the problem using tools like dig. If you use dnscache, i suggest to raise the udpbuf (to [4096]) in dns_transmit.c.

    5.5 Fast delivery for qmail-remote

    qmail-remote incorporates two performance critical steps for the delivery:

    Bruce Guenter recognized the last fact and patched qmail-remote accordingly ("fastremote"), thus qmail-remotes processes the input data in chunks of 4 Kbyte. This patch has been included into SPAMCONTROL.

    5.6 qmail-remote as QMTP client

    qmail-remote includes now a QMTP client. This extends the 'mini-qmail' scheme and allows to setup the internal email system on QMTP rather then SMTP.

    QMTP delivery is triggered by means of the file control/qmtproutes which follows the same symtay as control/smtproutes and obeys the syntax

    domain:relay:port

    5.7 qmail-remote routing order

    qmail-remote applies the definded routing advices in the following order:

    1. authsender routes have precedence.
    2. qmtproutes and smtproutes are evaluated side-by-side.
      1. A more specific (domain) entry has precedence over a less-specific one.
      2. Equal-level qmtproutes are used in favour of smptroutes.

    5.8 Greylisting recognition by qmail-remote

    According to RFC 6647 the peer MTA may use Greylisting for spam prevention. In this case, the SMTP dialoge is terminated by the server issuing either the return code '421' or '450'.

    In case qmail-remote recognizes any of those return codes and within the qmail-send log the information

    Connected to host but greylisted
    is displayed, where host is the FQDN of the peer MTA.

     


     

    6. qmail-pop3d/qmail-popup

    6.1 Flexible APOP support

    Though qmail-pop3d and qmail-popup support APOP out-of-the box while providing a digest in the POP3 greeting, there are cases in which this behaviour is unwanted, in particular if no APOP supporting PAM is used, ie. Bruce Guenther's checkvpw.

    Aligned with qmail-smtpd the (new) environment variable POP3AUTH has been introduced, allowing the following settings:

    resulting in the same behavior.

    Without these settings, qmail-popup only provides username/password annoncements in addition with the CAPA=USER statement according to RFC 2449 .

    Note: This helps to avoid authenticaiton problems in conjunction with MacOS 'Mail.app' which requires the CAPA=USER annoncement for none-APOP connections.

    6.2 STLS support

    The STLS (Start TLS support) for qmail-popup follows the same scheme as qmail-smtpd.

    Requirements:

    Actually, you can use the same "env" file as for qmail-smtpd. In this case qmail-popup announces in the capability list "STLS" and the following POP3 dialogue is encrypted as is the transmission of the received emails as well.

    6.3 qmail-pop3d run Script

    In order to use SSL encryption for a POP3 connection, the following run script for qmail-pop3d is appropriate:

    Note: In this run script I use Bruce Guenter's checkvpw as PAM (for vmailmgr), which requires the additional presence of the "Maildir" argument after the call of qmail-pop3d. Unfortunately, checkvpw does not support APOP but rather USER authentication only.

    The profile /var/qmail/ssl/env is the same as for qmail-smtpd. Defining the environment variable UCSPITLS directly in the run script instead of the profiles, allows a flexible use of the STARTTLS/STLS option for qmail-pop3d and qmail-smtpd without modifying the common profile.

    6.4 qmail-popup logging

    The access to the POP3 accounts can be monitored now in the logfiles. The provided information is available on FD 5. In order to display those in the logfile, additionally '5>&1' has to be included in the call of qmail-popup.

    The information provided is equivalent to those available for qmail-smtpd. You will realize the following line per connection:
    @400000004fbf28d603b6f744 qmail-popup: pid 24852 Accept::AUTH::User P:POP3S S:87.168.17.32:p57a81120.dip0.t-ipconnect.de ?= 'username'

    Authentication attempts are labled 'Accept' or 'Reject'. The type of authentication is indicated as 'User' or 'Apop' and additionally the provided authentication-id is displayed together with the protocol type.

    Rejected POP3 session where STLS is required but not honored are displayed as:

    @400000004fbf28dab3b6f744 qmail-popup: pid 7466 Reject::STLS::Any P:POP3 S:84.136.217.6:p5488d906.dip0.t-ipconnect.de ?= 'unknown'
    Since the session is terminated at a very early stage, neither the authentcation method nor the username is available.

     


     

    7. qmail-send, qmail-queue, and the qmail's sendmail

    7.1 Bounces

    Bounces have generally a Null-Sender address (Mail From: <>) and are out-of-band error-messages to indicate a failure in the delivery process. In fact, RFC 2821/821 requires that all notification E-Mails have to have a Null-Sender address!

    For every undeliverable message, generates a bounce to the Sender. While this is legitimate and necessary for normal operation, in case of SPAM attacks the bounces are meaningless:

    Unless you use a 'whitelisting' of Recipient E-Mail addresses, there is not much to do about. However, SPAMCONTORL helps you in theses cases:

    7.1.1 Limiting the Size of Bounces

    By definition, a bounce is a SMTP notification for a failure situation. It is common practice, to include the original message in the bounce. Qmail uses a specific format, introduced by Dan Bernstein and called "QSBMF" (qmail-send Bounce Message Format); other MTA encapsulate the original message as MIME attachment in rfc822/message format.

    Anyway, for a legitimate bounce reaching the Sender the original message is usually of no interest, except for identification purposes. In order to save bandwidth, you can limit the size of bounces using the control file

    Unlike the original patch (from Frank DENIS aka Jedi/Sector One <j@4u.net>), the default value is '0' byte, meaning no limits. A useful limit would by 2000 (byte), which covers the header and some body part information. The average size of a SPAM E-Mail is 5 Kbyte.

    The original message included in the bounce will be limited to the defined bouncemaxbytes and truncated, which is displayed in the bounce with "--- Rest of message truncated." at the end of the bounce.

    7.1.2 Defining bounceroutes

    qmail-remote is now able to recognize bounces by means of the missing SMTP Mail From: address "<>".

    To support efficient bounce handling, all bounces can be redirected to a particular 'bounce host'. Simply include

    !@:bouncehost.af.mil:27

    into control/smtproutes or control/qmtproutes and you are done.

    7.1.3 Dumping Double Bounces

    Double bounces are generated, if the bounce can not be delivered to the Sender.

    Double bounces are usually delivered to the 'Postmaster' account. It is convenient that this account is local and eventual double bounces are stored in a mbox/Maildir for later inspection. However, Qmail allows you to forward double bounces to some other account defined in

    However, due to the forged Sender address in SPAM E-Mails, practically all bounces become double bounces eventually. In this case any storage and inspection is fruitless. Taken from Russell Nelson and Charles Cazabon, you can optionally dump all double bounces immediately. This is facilitated if doublebounceto contains a '@' in the first line.

    Those dumped double bounces show up in the qmail-send log as: "double bounce: discarding".

    7.2 qmail-send Gadgets

    As a further gadget, the qmail-send control files

    are re-read by means of a HUP signal (eg. svc -h /service/qmail-send).

    Note: The 'silent' spawn limit has been increased to 500. Thus, in maximum qmail-send can run 500 qmail-local (and more reasonable) 500 qmail-remote processes.

    7.3 qmail-queue: BigToDo enhancement

    The queue directories ./intd and ./todo are splitted (as per conf-split) into subdirectories to allow a more efficient treatment of many incoming messages.

    Caution: Make sure, that the directories ./queue/todo and ./queue/intd are empty before applying the patch; otherwise qmail-send will not be able to process those messages anymore!

    Note: The shell script qmail-qstat and in addition some qmail-mrtg analyses are affected by this change.

    7.4 sendmail Fixes

    The following fixes for Qmail's sendmail wrapper have been included for compatibility reasons:

     


     

    8. SMTP Protocol Return Codes

    SMTP allows to reject Sessions based on some technical and/or political criteria, which are not well expressed in the RFCs (2821, 2554, 2505, 1122).

    The SMTP protocol mechanism between the client and the server are defined as Commands and Replies. SMTP uses a three-letter Reply Code. The first digit tells whether a command was accepted and completed (2), transaction begin (3), or whether there was as transient (4) or permanent failure (5). In addition, an explanatory description may be given.

    RFC 1893 introduces a concept of "Enhanced Mail System Status Codes" (EMSSC) which should provide easily parseable SMTP server conditions and transaction statuses, usually at the end of the SMTP reply and included in parenthesis, eg. (#5.5.1).

    The SMTP Reply Codes and the EMSSC are detailed in the corresponding RFCs, but don't fit well to each other, thus either providing redundant information or almost no additional information at all. In short, the EMSSC is nowadays almost meaningless.

    Here's a breakdown of SPAMCONTROL's SMTP Reply Codes, informational texts, and the used EMSSC.

     Reply

     Informational text

     EMSSC

     421  unable to check recipients  (#4.3.0)
     450  sorry, mailbox currently unavailable  (#4.2.1)
     451   DNS temporary failure  (#4.3.0) 
     452 Too many recipients  (#4.5.3)
     454  TLS not available due to temporary reason   (#5.7.3)
         
     501  auth exchange canceled  (#5.0.0)
     501  malformed auth input  (#5.5.4)
     503  you're already authenticated  (#5.5.0)
     503  no auth during mail transaction  (#5.5.0)
     503  sorry, SMTP Authentication not available  (#5.7.3)
     504  auth type unimplemented  (#5.5.1)
     535  authorization failed  (#5.7.1)
     535   authentication required  (#5.7.1)
     535   STARTTLS required   (#5.7.1)
         
     550  sorry, invalid HELO/EHLO greeting  (#5.7.1)
     550  sorry, your envelope recipient is in my badrcptto list  (#5.7.1)
     550  sorry, invalid sender address specified  (#5.7.1)
     550  sorry, bounce messages should have a single envelope recipient  (#5.7.1)
         
     552  sorry, that message size exceeds my databytes limit  (#5.3.4)
     553  sorry, your envelope sender is in my badmailfrom list  (#5.7.1)
     550  sorry, that domain isn't in my list of allowed rcpthosts  (#5.7.1)
     553  sorry, your envelope sender domain must exist  (#5.7.1)
         
     554  too many hops, this message is looping  (#5.4.6)
     554  sorry, invalid message content (optional text)  (#5.3.2)

     


     

    9. qmail-smtpd Logging

    Normally, qmail-smtpd doesn't log anything. With SPAMCONTROL qmail-smtpd logs accepted and some (important) rejected SMTP session attempts. The logging is done

    Note: Including the PID in every line makes it possible to follow the current. transaction on port SMTP/ESMTP: From initialisation over tcpserver/sslserver, thru rblsmtpd and eventually to the (missing, rejected, accepted) (E)SMTP session in qmail-smtpd.

    9.1 Extensible logging scheme

     Action

     Type

     Condition

     Explanation

     Reject  SMTP  Toomany_Hops  Message Hop count exceeded
     Reject  SMTP  Syntax_Error  Malformed SMTP address (e.g. missing brackets)
           
     Reject  DATA  Invalid_Size  DATA exceeds sizelimit
     Reject  DATA  Bad_MIME  DATA includes BASE 64 MIME type listed in badmimetypes
     Reject  DATA  Bad_Loader  DATA includes BASE64 loader type listed in badmimetypes
     Reject  DATA  Virus_Infected  DATA includes virus infected message ('<scanner>' | 'AV scanner')
     Reject  DATA  Spam_Message DATA includes an identfied Spam message 
           
     Reject  SNDR  Bad_Helo  SNDR's HELO is in the badhelo
     Reject  SNDR  DNS_HELO  SNDR's HELO has no DNS A/MX RR
     Reject  SNDR  Invalid_Relay  SNDR's tries relaying; but not allowd
     Reject   SNDR  Missing_TLS  STARTTLS was required but not granted by client
     Accept  SNDR  Relay_Client  SNDR was identfied as relay client
     Info  SNDR  TLS  SNDR accepted TLS connection
           
     Reject  ORIG  Bad_Mailfrom  ORIG is in badmailfrom
     Reject  ORIG  DNS_MF  Domain part of ORIG has no DNS MX RR
     Reject  ORIG  Failed_Auth  ORIG tried SMTP Authentication; but failed
     Reject  ORIG  Invalid_Sender  ORIG not allowed to send
     Reject  ORIG  Missing_Auth  SMTP Authentication required, but not granted
     Info  ORIG  Valid_Auth  ORIG was successful authenticated
     Accept  ORIG  Local_Sender  ORIG was identified as local sender address
     Accept  ORIG  Relay_Mailfrom  ORIG was accepted als Relaymailfrom
           
     Reject  RCPT  Bad_Rcptto  RCPT is in badrcptto
     Reject  RCPT  Toomany_Rcptto  Too many RCPTs
     Reject  RCPT  Failed_Rcptto  RCPT could not acceptd as per recipients/cdb.
     Accept  RCPT  Recipients_Cdb  RCPT was accepted as per recipients/cdb.
     Accept  RCPT  Recpients_Pam  RCPT was accepted per PAM lookup.
     Accept  RCPT  Recpients_Wild  RCPT was accepted per Domain wildlisting in recipients.
     Accept  RCPT  Rcpthosts_Rcptto  RCPT was accepted as per rcpthosts/morercpthosts
           

  • SNDR corresponds to the sending MTA.
  • ORIG is the "MAIL From: <Return-Path>". (Sender)
  • RCPT is the "RCPT To: <Forwarding-Path>" (Recipient).
  • DATA is the Message.
  • The Information includes typically the following

    This scheme is easy extensible to other successful/deferred SMTP sessions. Sample:

    Accept::SNDR::Relay_Client: P:orion.fehnet.de S:81.173.229.48:xdsl-81-173-229-48.netcologne.de H:mail.fehcom.de F:feh@fehcom.de T:erwin@example.com

    9.2 qmail-smtpd Start-Up Script Using tcpserver/sslserver And splogger

    A typical tcpserver/sslserver start script applying standard splogger:

    #!/bin/sh
    QMAILDUID=`id -u qmaild`
    QMAILDGID=`id -g qmaild`
    /usr/local/bin/tcpserver -v -R -H -u $QMAILDUID -g $QMAILDGID QMAIL_IP_ADDRESS smtp \
      /var/qmail/bin/qmail-smtpd /bin/cmd5checkpw true 2>&1 | \
      /var/qmail/bin/splogger smtpd &

    Since splogger is now facilitated, ACCUSTAMP time information is included.

    9.3 qmail-smtpd Start-Up Script Using Daemontools, tcpserver/sslserver, and multilog

    A better choice would be multilog. multilog allows you to write separate filtered logs; to individual directories, and/or files, STDERR respectively. A typical Daemontools qmail-smtpd run script would look like:

    #!/bin/sh
    QMAILDUID=`id -u qmaild`
    QMAILDGID=`id -g qmaild`
    /usr/local/bin/tcpserver -R -u $QMAILDUID -g $QMAILDGID QMAIL_IP_ADDRESS smtp \
       /var/qmail/bin/qmail-smtpd /bin/cmd5checkpw true 2>&1

    Note: tcpserver/sslserver's logging via the '-v' flag can be omitted to get mostly a full comprehensive and terse one-line logging of the SMTP session.

    The corresponding multilog run script allows not only to filter the log information and write them to the file "current" in a specific directory but in addition to feed a file with specific information; here's a sample:

    #!/bin/sh
    exec setuidgid qmaill multilog t \
       '-*tcpserver: status:*' /var/log/qmail-smtpd\
       '-*' '+*Reject::*' =/var/log/qmail-smtpd/rejected \
       '-*' '+*DNS*' =/var/log/qmail-smtpd/nodnsmx

    In this case, multilog adds at first a TAI64 time stamp.

    1. In a second step, the log information - except those line including the string "tcpserver: status" - are written into the log file /var/log/qmail-smtpd/current.
    2. On the third line the filter "-*" removes any unwanted lines and "+*Reject::*" picks up from qmail-smtpd's log all rejected sessions and placing the last rejected condition in the file 'rejected'!
    3. On the fourth line the last failing DNS MX lookups for Mail From: <address> is logged in a file named /var/log/qmail-smtpd/nodnsmx.

     


     

    10. Howto

    10.1 SPAMCONTROL compile Options

    In this version of SPAMCONTROL I have substantially reduced the number of compile-time options:

    In order to consistently change all relevant binaries, use the file conf-spamcontrol which is evaluated by the installation routine install_spamcontrol.sh and passes the changes to the Qmail c-files:

    # Configuration for SPAMCONTROL (no tabs allowed)
    #
    # Additional DELIVERING
    #
    verp=yes # allow VERP addresses for RECIPIENTS
    pam111421=yes # otherwise qmail-smtpd will allow the email in case of PAM failures.

    10.2 Setting up qmail+SPAMCONTROL

    In case your E-Mail environment complies to the assumption in PURPOSE do the following:

    1. Stop your Qmail system (receive and send).
    2. Remove SMTP/POP3 connection to be serviced by INETD/XINETD.
      Employ sslserver (> ucspi-ssl-0.8x) instead.
    3. Follow the INSTALL.spamcontrol instructions.
    4. To activate qmail-remote's TLS capabilities create the following file with content:

    In case of TLS support for qmail-smtpd and qmail-pop3d:

    See above samples and check the included samples for ./badmailfrom and ./badrcptto.

    TLS support for qmail-remote depends on:

    Restart Qmail:

    1. Check your configuration by means of qmail-showctl. It is a good idea to pipe the output of that command with timestamp information to a file.
    2. Test your filters locally (check TEST.* in ./doc directory).
    3. If you are already blacklisted, inform those sites that you don't act as an OPEN RELAY anymore.
    4. Watch the Qmail behavior by means of the log file information.

    Good luck!

    10.3 Incompatibilities

    The SPAMCONTROL patch is incompatible with the Qmail LDAP patch. It should be applied against qmail-1.03 and not against netqmail-1.0x.

    10.4 AMD64 and clang support

    SPAMCONTROL will happily run under AMD64 and compile successfully with clang under the following conditions:

    #!/usr/local/bin/ksh

    FILES=$(grep -l "void main" *.c)
    for FILE in $FILES; do
      echo $FILE
      sed s/void\ main/int\ main/ $FILE > $FILE.new
      mv $FILE.new $FILE
    done

     


     

    11. Authors

    Major enhancements for SPAMCONTROL are taken from these authors:

     


     

    12. Thanks

    Thanks to the discussion in the Qmail Mailing List (qmail@list.cr.yp.to) in particular:

    Erwin Hoffmann (feh@fehcom.de)
    Hoehn, 2014-04-26