Twisted Mail Server: The Conclusion

Holy old code, Batman -- it's been about 2.5 years since I first blogged about the Twisted mail server I cobbled together. In the intervening time, I've received tons of emails and instant messages requesting that the code be put up somewhere. Well, that time has come...

I recently decommissioned a box I had at tummy.com which was my primary mail server for several years. After many years of qmail and a few months with Postfix, I wanted a solution that I had more immediate control over. At that time, Gmail for Domains was just announced and I became one of the beta testers for it. However, it was missing a few critical features, so I sat down and put together a mail server for myself. Abe Fetting had some nice examples in his O'Reilly book, so I started there. Then I jumped into the Twisted source code and discovered the rest that I needed.

A year or so later, Gmail for Domains (now Google Apps for business) was rockin' out, so I started moving my mail there. I eventually aliased the remainder of my domains and then stopped using the Twisted mail server. I kept the tummy.com box around for a while for DNS, but due to a tightening of the budget at home, I later had to pull the plug. When I reviewed the backup files from the server, I saw the old mail code and put it at the top of the list for my down-time projects.

I've sort of done that, but the process is incomplete. In the spirit of r0ml's OSCON 2008 talk, I'm placing this code in a public space for people to play with, find bugs in, report them, add patches, and branch from. The project page on Launchpad.net is here:

https://launchpad.net/tx/txmailserver

There are a couple non-standard parts of the code (in particular, the account, alias, and mail list management), but it can be improved very easily. The Dspam stuff was nascent at best (I didn't end up using it in production). Also, the relaying really needs to be looked into to make sure that it's done safely. A quick task someone could jump on right away is to convert the copious print statments to twisted.python.log calls.

Having the code in the state that it's in could be a lot of fun, really. The code base is tiny and there's nothing too tricky going on. For those that haven't had a chance to play with Bazaar, you can easily branch it to your own lp home dir, and if you make some cool improvements, I can easily merge those back in to the main branch. Looking at the list of branches, potential users and developers will be able to easily see the efforts that others have made. txMailServer could end up being a nice little piece of utilitarian code for folks...

Enjoy! And sorry it took so long :-(

Twisted Mail Server

Update: The saga now has a conclusion! See this blog post for details.

Well, these last two months have been hell. Working on sucky projects just plain sucks. It ate up so much of my time that I've got an unhealthy backlog of blog posts waiting for release into the wild.

The first post I am compelled to write is on my new mail server. In the past, I've used sendmail, qmail and postfix. I have not been happy with any of those (though I did really like qmail, until it got too cumbersome to keep it updated). I didn't have the level of control I wanted -- and this was solely because I couldn't fit my brain into those applications (though, again, qmail came the closest).

I decided to chuck it all, and write my own mail server in twisted, using the pre-built lego code that twisted offers for this sort of thing. I've been running the server for about a month now, and all I can say is "wow". Just WOW.

The level of control I have over the operation of my mail server is insane. I can get this thing to do exactly what I want, when I want. I've got a bazillion domain names for which I (or others) receive email. I was able to write the code that lets me handle that the way that makes sense for ME (and *not* the author(s) of Postfix, etc.).

Today, I needed to add support for aliases that were actually lists of recipients. One "if" statement and an additional implementation of smtp.IMessage later, it was operational. I don't know how I ever ran a mail server any other way.

I've been testing my mail server all month, and it's running beautifully. It has continued to be free of relay issues and spammer attacks. I couldn't be happier with the results.

Now that I am feeling more secure in the custom code, I'm ready to start adding additional features I need:

• white listing: automatically updated with the address of people to whom I send email
• black listing: I am starting to maintain a list of the most heinous offenders in my junk mail box; these will be regularly pushed to the mail server
• greylisting: I have begun planning an implementation of greylisting, but this will take some time to get right
• spammer databases: I am considering using one or more of these. My only problem with them is that I don't trust them. I don't want to block someone inadvertantly just because they were unlucky enough to have one of their boxes raped into becoming an open relay 3 years ago.

Having a mail server that runs on twisted seemed a little daunting at first. I feared maintenance and security burdens, however, I have already begun reaping the benefits and my fears have been shown to be baseless. I spend 1/10th the maintenance time on the twisted version. I have *fun* while updating the server or configurations. I can plug my own code into it instead of using third party applications I don't like or patches I don't understand.

My first exposure to twisted was late 2002 as I was writing my first "real" python script (a networking script, naturally). Since that time, twisted has integrated itself into my life such that I can't imagine living without it. I literally use it for all of my coding activities: my professional life depends upon it nearly 100%, and 50% of my entertainment is derived from programming activities, all of which incorporate some aspect of twisted.

I, for one, welcome our twisted overlords.

Now playing:

Bagpipes - Flow Gently Sweet Afton