At the OSDC conference in 2006 I presented a paper on mail relay performance based on the new BHM program that is now part of Postal.

I have a Postal category on my main blog that I use for a variety of news related to Postal. This post (which will be updated periodically) will be the main reference page for the software. Please use the comments section for bug reports and feature requests.

It works by taking a list of email addresses to use as FROM and TO addresses. I originally used a template to generate the list of users because if each email address takes 30 bytes of storage then 3,000,000 accounts would take 90M of RAM which would be more than the memory in the test machine I was using at the time. Since that time the RAM size in commodity machines has increased far faster than the size of ISP mail servers so I removed the template feature (which seemed to confuse many people).

When sending the mail the subject and body will be random data. A header field X-Postal will be used so that procmail can easily filter out such email just in case you accidentally put your own email address as one of the test addresses. ;)

I have now added two new programs to the suite, postal-list, and rabid. Postal-list will list all the possible expansions for an
account name (used for creating a list of accounts to create on your test server). Rabid is the mad Biff, it is a POP benchmark.

Postal now adds a MD5 checksum in the header X-PostalHash to all messages it sends (checksum is over the Subject, Date, Message-ID, From, and To headers and the message body including the “\r\n” that ends each line of text in the SMTP protocol). Rabid now checks the MD5 checksum and displays error messages when it doesn’t match.

I have added rate limiting support in Rabid and Postal. This means that you can specify that these programs send a specific number of messages and perform a specific number of POP connections per minute respectively. This should make it easy to determine the amount of system resources that are used by a particular volume of traffic. Also if you want to run performance analysis software to determine what the bottlenecks are on your mail server then you could set Postal and Rabid to only use half the maximum speed (so the CPU and disk usage of the analysis software won’t impact on the mail server).

I will not release a 1.0 version until the following features are implemented:

• Matching email sent by Postal and mail received by BHM and Rabid to ensure that each message is delivered correctly (no repeats and no corruption)
• IMAP support in Rabid that works
• Support for simulating large numbers of source addresses in Postal. This needs to support at least 2^24 addresses so it is entirely impractical to have so many IP addresses permanently assigned to the test machine.
• Support for simulating slow servers in Postal and BHM (probably reducing TCP window size and delaying read() calls)
• Making BHM simulate the more common anti-spam measures that are in use to determine the impact that they have on list servers
• Determining a solution to the problem of benchmarking DNS servers. This may mean just including documentation on how to simulate the use patterns of a mail server using someone else’s DNS benchmark, but may mean writing my own DNS benchmark.

Here are links to download the source:

• postal-0.72.tgz – made LMTP work and accept TAB as a field delimiter.
• postal-0.71.tgz – rewrote the md5 checking code and fixed lots of little bugs.
• postal-0.70.tgz – tidied up the man pages and made it build without SSL support.
• postal-0.69.tgz – fixed some compile warnings, and really made it compile with GCC 4.3
• postal-0.68.tgz – fixed some compile warnings, made it compile with GCC 4.3, and I think I made it compile correctly with OpenSolaris.
• postal-0.67.tgz – changed the license to GPL 3
• postal-0.66.tgz – made GNUTLS work in BHM and added MessageId to Postal.
• postal-0.65.tgz – significant improvement, many new features and many bugs fixed!
• postal-0.62.tgz – Slightly improved the installation documents and made it build with GCC 3.2.
• postal-0.61.tgz – version 0.61. Fixed the bug with optind that stopped it working on BSD systems, and a few other minor bugs.
• postal-0.60.tgz – version 0.60. Fixed the POP deletion bug, made it compile with GCC 3.0, and added logging of all network IO to disk.
• postal-0.59.tgz – version 0.59.
• postal-0.58.tgz – version 0.58. Added some new autoconf stuff, RPM build support, and the first steps to OS/2 and Win32 portability.
• postal-0.57.tgz – version 0.57. Fixed lots of trivial bugs and some BSD portability issues.
• postal-0.56.tgz – version 0.56. Added Solaris package manager support. Made it compile without SSL. Added heaps of autoconf stuff.
• postal-0.55.tgz – version 0.55. Made Rabid work with POP servers that support the CAPA command. Fixed some compile problems on Solaris.
• postal-0.54.tgz – version 0.54. Added a ./configure option to turn off name expansion (for systems with buggy regex). Fixed a locking bug that allowed Rabid to access the same account through two threads.
• postal-0.53.tgz – version 0.53. Don’t use NIS domain name etc for SMTP protocol.
• postal-0.52.tgz – version 0.52. Better portability with autoconf.
• postal-0.51.tgz – version 0.51. Supports compiling without SSL and some hacky Solaris support.
• postal-0.50.tgz – version 0.50. Adds SSL support to Postal (Rabid comes next).

• memlockd_1.0.tar.gz
• memlockd_0.05.tar.gz
• memlockd_0.04.tar.gz
• memlockd_0.03.tar.gz
• memlockd_0.02.tar.gz
• memlockd_0.01.tar.gz

• Portslave version 2010.04.19 source
• Portslave version 2010.03.30 source

• clfmerge will merge a number of Common Logfile Format web log files into a single file while also re-ordering them in a sliding window to cope with web servers that generate log entries with the start-time of the request and write them in order of completion.
• logprn operates like tail -f but will (after a specified period of inactivity) spawn a process and write the new data in the log file
  to it’s standard input.
• clfsplit will split up a single CLF format web log into a number of files based on the client’s IP address.
• funnel will write it’s standard-input to a number of files or processes.
• clfdomainsplit split a CLF format web log containing fully qualified URLs (including the host name) into separate files, one for each host.

Download:

• logtools-0.13d.tgz – made it compile with GCC 4.3.
• logtools-0.13c.tgz – minor documentation improvements and made it compile with recent GCC.
• logtools-0.13a.tgz – made clfdomainsplit work correctly with URLs that specify the port.
• logtools-0.13.tgz – compiles with GCC 3.x.
• logtools-0.12b.tgz – minor improvement to man pages.
• logtools-0.12a.tgz
• logtools-0.11.tgz
• logtools-0.10.tgz
• logtools-0.08.tgz
• logtools-0.07.tgz
• logtools-0.06.tgz
• logtools-0.05.tgz

This program solves that problem by creating a single file with the message data and creating links to it from the ~/Maildir/new directory of every user who is in the group (it delivers mail based on Unix groups). This is fast (can deliver a bulletin to 30000 users in minutes on a slow machine), saves disk space, works with all Maildir client software, and makes it very easy to undeliver or modify a bulletin.

Download links:

• maildir-bulletin-0.69.tgz – version 0.69 – fixed some Debian packaging bugs, included this web page URL in the man pages, and merged patches to make it build with GCC 4.3.
• maildir-bulletin-0.67.tgz – version 0.67 – fixed more minor warnings and compile errors with recent GCC versions.
• maildir-bulletin-0.65.tgz – version 0.65 – fixed some minor warnings and a compile error.
• maildir-bulletin-0.63.tgz – version 0.63 – fixed a minor bug in displaying errors and documented ALLOWEDSENDERS properly.
• maildir-bulletin-0.62.tgz – version 0.62 – now with autoconf.
• maildir-bulletin-0.61.tgz – version 0.61.
• maildir-bulletin-0.60.tgz – version 0.60.