Saturday, January 23, 2010

e-mail packrat

I've long tried to keep all the e-mail I've ever sent or received; I've got an archive going back to 1985 or so, when I first realized that keeping e-mail might be a good idea. Trouble is, keeping such an archive in one place requires a fair amount of maintenance, because formats and protocols change. Is it worth the effort?

  • Until 1987, I primarily used a VMS system.
  • In 1987, I switched to a Unix machine at MIT. I was able to import my old VMS mail into whatever the mailbox format of the day was (mbox, probably) by a script I found somewhere.
  • In 1992, I switched to using POP through the client program Eudora. Eudora stored the mail locally, in a folder format that was something like 'mbox', but not exactly. (For example, attachments were not stored inlined, but instead in external files.) I managed to import my old Unix mbox files into folders.
  • In 2004, I switched from POP to IMAP. I went through an extensive process to convert my existing mail base into real mbox files that my IMAP server could read. I spent several days writing scripts to convert the Eudora pseudo-mbox files to something imapd could handle.
  • In 2006, I left Quiotix, and switched my primary mail over to Tuffmail. I took my mail archive (in mbox file format) and put it up on my server machine, and serve that up with imapd. So I now have my mail split between two servers, but Thunderbird can deal with multiple servers just fine. I tried to move the archived mail to Tuffmail as well with several different tools (imapsync, offlineimap, Thunderbird bulk-copy) but I could never get a clean copy -- I suspect that the combination of crappy old multiply-converted mbox files and the old UW-IMAPD server is to blame.

Right now its still fragmented across a number of formats and servers. Yuck.


  1. I just went through migrating about 15 years worth of Mbox files to Gmail via IMAP. I wrote these scripts for managing it:

    The Mbox parser doesn't rely on anything crazy other than a blank line before each message and a properly formatted SMTP header. You might get some luck with it.

  2. Hey Brian,

    Tuffmail would seem to be the email provider for true DIY geeks. I've used them for a couple years. Thing is, their spam filters are getting kind of stale, and feedback doesn't get much response. The entire operation appears to be somebody's hobby. Considering switching to