Postal
This is a program I wrote to benchmark SMTP servers. I started work on this because I need to know which mail server will give the best performance with more than 1,000,000 users. I have decided to release it under the GPL because there is no benefit in keeping the source secret, and the world needs to know which mail servers perform well and which don’t!
At the OSDC conference in 2006 I presented a paper on mail relay performance based on the new BHM program that is now part of Postal.
I have a Postal category on my main blog that I use for a variety of news related to Postal. This post (which will be updated periodically) will be the main reference page for the software. Please use the comments section for bug reports and feature requests.
It works by taking a list of email addresses to use as FROM and TO addresses. I originally used a template to generate the list of users because if each email address takes 30 bytes of storage then 3,000,000 accounts would take 90M of RAM which would be more than the memory in the test machine I was using at the time. Since that time the RAM size in commodity machines has increased far faster than the size of ISP mail servers so I removed the template feature (which seemed to confuse many people).
When sending the mail the subject and body will be random data. A header field X-Postal will be used so that procmail can easily filter out such email just in case you accidentally put your own email address as one of the test addresses. ;)
I have now added two new programs to the suite, postal-list, and rabid. Postal-list will list all the possible expansions for an
account name (used for creating a list of accounts to create on your test server). Rabid is the mad Biff, it is a POP benchmark.
Postal now adds a MD5 checksum to all messages it sends (checksum is over the subject and message body including the “\r\n” that ends each line of text in the SMTP protocol). Rabid now checks the MD5 checksum and displays error messages when it doesn’t match.
I have added rate limiting support in Rabid and Postal. This means that you can specify that these programs send a specific number of messages and perform a specific number of POP connections per minute respectively. This should make it easy to determine the amount of system resources that are used by a particular volume of traffic. Also if you want to run performance analysis software to determine what the bottlenecks are on your mail server then you could set Postal and Rabid to only use half the maximum speed (so the CPU and disk usage of the analysis software won’t impact on the mail server).
I will not release a 1.0 version until the following features are implemented:
- Matching email sent by Postal and mail received by BHM and Rabid to ensure that each message is delivered correctly (no repeats and no corruption)
- IMAP support in Rabid that works
- Support for simulating large numbers of source addresses in Postal. This needs to support at least 2^24 addresses so it is entirely impractical to have so many IP addresses permanently assigned to the test machine.
- Support for simulating slow servers in Postal and BHM (probably reducing TCP window size and delaying read() calls)
- Making BHM simulate the more common anti-spam measures that are in use to determine the impact that they have on list servers
- Determining a solution to the problem of benchmarking DNS servers. This may mean just including documentation on how to simulate the use patterns of a mail server using someone else’s DNS benchmark, but may mean writing my own DNS benchmark.
Here are links to download the source:
- postal-0.70.tgz – tidied up the man pages and made it build without SSL support.
- postal-0.69.tgz – fixed some compile warnings, and really made it compile with GCC 4.3
- postal-0.68.tgz – fixed some compile warnings, made it compile with GCC 4.3, and I think I made it compile correctly with OpenSolaris.
- postal-0.67.tgz – changed the license to GPL 3
- postal-0.66.tgz – made GNUTLS work in BHM and added MessageId to Postal.
- postal-0.65.tgz – significant improvement, many new features and many bugs fixed!
- postal-0.62.tgz – Slightly improved the installation documents and made it build with GCC 3.2.
- postal-0.61.tgz – version 0.61. Fixed the bug with optind that stopped it working on BSD systems, and a few other minor bugs.
- postal-0.60.tgz – version 0.60. Fixed the POP deletion bug, made it compile with GCC 3.0, and added logging of all network IO to disk.
- postal-0.59.tgz – version 0.59.
- postal-0.58.tgz – version 0.58. Added some new autoconf stuff, RPM build support, and the first steps to OS/2 and Win32 portability.
- postal-0.57.tgz – version 0.57. Fixed lots of trivial bugs and some BSD portability issues.
- postal-0.56.tgz – version 0.56. Added Solaris package manager support. Made it compile without SSL. Added heaps of autoconf stuff.
- postal-0.55.tgz – version 0.55. Made Rabid work with POP servers that support the CAPA command. Fixed some compile problems on Solaris.
- postal-0.54.tgz – version 0.54. Added a ./configure option to turn off name expansion (for systems with buggy regex). Fixed a locking bug that allowed Rabid to access the same account through two threads.
- postal-0.53.tgz – version 0.53. Don’t use NIS domain name etc for SMTP protocol.
- postal-0.52.tgz – version 0.52. Better portability with autoconf.
- postal-0.51.tgz – version 0.51. Supports compiling without SSL and some hacky Solaris support.
- postal-0.50.tgz – version 0.50. Adds SSL support to Postal (Rabid comes next).
Hi Russell,
Do you have any clues yet on what the best performing SMTP servers are?
Thx,
February 23rd, 2009 at 7:23 pmJeroen
that would be grat to know without installing XX MTAs
March 27th, 2009 at 12:55 amhi, Russell,
i can’t compile postal0.7.0&0.6.5 on RHEL5U3, which have same error.
postal.cpp: In function ‘int gcry_pthread_mutex_init(void**)’:
postal.cpp:16: error: invalid conversion from ‘void*’ to ‘pthread_mutex_t*’
postal.cpp: In function ‘int gcry_pthread_mutex_destroy(void**)’:
postal.cpp:16: error: invalid conversion from ‘void*’ to ‘pthread_mutex_t*’
postal.cpp: In function ‘int gcry_pthread_mutex_lock(void**)’:
postal.cpp:16: error: invalid conversion from ‘void*’ to ‘pthread_mutex_t*’
postal.cpp: In function ‘int gcry_pthread_mutex_unlock(void**)’:
postal.cpp:16: error: invalid conversion from ‘void*’ to ‘pthread_mutex_t*’
postal.cpp: At global scope:
postal.cpp:16: warning: missing initializer for member ‘gcry_thread_cbs::read’
postal.cpp:16: warning: missing initializer for member ‘gcry_thread_cbs::write’
postal.cpp:16: warning: missing initializer for member ‘gcry_thread_cbs::select’
postal.cpp:16: warning: missing initializer for member ‘gcry_thread_cbs::waitpid’
postal.cpp:16: warning: missing initializer for member ‘gcry_thread_cbs::accept’
postal.cpp:16: warning: missing initializer for member ‘gcry_thread_cbs::connect’
postal.cpp:16: warning: missing initializer for member ‘gcry_thread_cbs::sendmsg’
postal.cpp:16: warning: missing initializer for member ‘gcry_thread_cbs::recvmsg’
make: *** [postal] Error 1
How to resolve it? Thanks!
June 16th, 2009 at 8:29 pmI’m having the same issue on Fedora 8. Does anybody know how to resolve it?
August 7th, 2009 at 3:45 amHi Russell,
I’m playing with Postal and Rabid, using the Debian Package on Lenny a instalation and i have a question. Can be disabled de MD5 verification without recompilation?
And i’m experiencing and issue, Rabid doesn’t print the workload values
Thx!
October 3rd, 2009 at 2:34 amCongratulations for this great tool!
Here is a patch to rabid that I sent Russell a while back, not fully vetted but works for me.
====================
I ran into a problem with rabid trying to log into pop with local name logins. Instead of the second field being sent as the password, the domain of the address was being sent instead.
I changed userlist.ccp and I got it working but I having really gone thru the logic to make sure it all correct.
lars@qadev:postal-0.70 diff -c userlist.cpp.orig userlist.cpp
*** userlist.cpp.orig Tue Jun 10 11:56:28 2008
— userlist.cpp Tue Jun 10 12:10:00 2008
***************
*** 34,44 ****
printf(“Need a password for \”%s\”.”, buf);
continue;
}
if(stripDom)
strtok(buf, “@”);
m_users->push_back(buf);
- if(usePass)
- m_passwords->push_back(buf + strlen(buf) + 1);
if(strlen(buf) > m_maxNameLen)
m_maxNameLen = strlen(buf);
}
— 34,44 —-
printf(“Need a password for \”%s\”.”, buf);
continue;
}
+ if(usePass)
+ m_passwords->push_back(buf + strlen(buf) + 1);
if(stripDom)
strtok(buf, “@”);
m_users->push_back(buf);
if(strlen(buf) > m_maxNameLen)
m_maxNameLen = strlen(buf);
}
——————————————————-
November 5th, 2009 at 6:53 amHi Russell,
I just tested out Postal 0.70 which I installed using the debian repository. It is working very well. I still have to start using all the features of Postal but thought that I should thank you before that. :). Thanks a lot for writing such a wonderful tool. I will soon try to contribute to it myself.
Thanks again.
Regards,
Vignesh
November 9th, 2009 at 7:07 pm