qmail-todo - schedule state change of message for delivery
qmail-todo
s/qmail with a high local and remote concurrency number is able to deliver a tremendous amount of messages (throughput). Depending on the provided resources however, often this can not be achieved since qmail-send becomes a bottleneck on delivery.
qmail-send preprocesses all new messages before deploying them for local or for remote delivering. In a particulur run, qmail-send does one ’todo’ processing, but has the ability to close multiple jobs. Due to this layout, potentially qmail-send can not feed all the new available (local/remote) delivery slots and therefore, it is not possible to achieve the maximum throughput.
This is a minor problem, given qmail-send is able to complete this in short time; but due to many file system calls (fsync and (un)link) a ’todo’ run is expensive and throttles the throughput.
qmail-todo solves this ’silly qmail (queue) problem’ which is apparent only on system with high injection rates, delegating the scheduling of ’todo’ runs to a dedicated process.
qmail-todo interfaces with qmail-send on file descriptors [1,8] on sending and [7,0] for receiving. qmail-todo communicates with qmail-clean on file descriptors [2,0] for sending and [3,1] for receiving.
qmail-todo and qmail-send share an extended and peristent message exchange format:
D[LRB]<mesgid>
Start delivery for new message with id <messid>.
The character L, R or B defines the type
of delivery: Local, Remote, or Both, respectively.
L<string>
Dump string to the logger without adding additional
’\n’ or similar.
qmail-todo sends "\0" terminated messages, whereas qmail-send just sends one character to qmail-todo.
+-------+
+-------+
| clean | | clean |
+--0-1--+ +--0-1--+ +-----------+
trigger ^ | ^ | +->0,1 lspawn |
| | v | v / +-----------+
+-------+ v +--2-3--+ +--5-6--+ /
| | | | 0<--7 1,2<-+
| queue |--+--| todo | | send |
| | | | 1-->8 3,4<-+
+-------+ +-------+ +---0---+ \
| \ +-----------+
v +->0,1 rspwan |
+---0---+ +-----------+
| logger|
+-------+
qmail-todo exits 0 if the messages have been processed successfully. It exits 1 in case there is a communication problem with qmail-send. The exit code 111 together with a diagnostic message is facilitated by qmail-todo in case it failes reading the required control files.
qmail-todo provides additional diagnostic messages to qmail-send to be displayed in the logs. In particular, in case of problems creating and (un)linking files.
qmail-todo included in s/qmail has been created by Andre Oppermann (http://www.nrg4u.com) as part of this LDAP patch for qmail. This man-page uses parts of his EXTERNAL discription.
qmail-send(8), qmail-queue(8).