Saturday, January 23, 2010

e-mail packrat

I've long tried to keep all the e-mail I've ever sent or received; I've got an archive going back to 1985 or so, when I first realized that keeping e-mail might be a good idea. Trouble is, keeping such an archive in one place requires a fair amount of maintenance, because formats and protocols change. Is it worth the effort?

  • Until 1987, I primarily used a VMS system.
  • In 1987, I switched to a Unix machine at MIT. I was able to import my old VMS mail into whatever the mailbox format of the day was (mbox, probably) by a script I found somewhere.
  • In 1992, I switched to using POP through the client program Eudora. Eudora stored the mail locally, in a folder format that was something like 'mbox', but not exactly. (For example, attachments were not stored inlined, but instead in external files.) I managed to import my old Unix mbox files into folders.
  • In 2004, I switched from POP to IMAP. I went through an extensive process to convert my existing mail base into real mbox files that my IMAP server could read. I spent several days writing scripts to convert the Eudora pseudo-mbox files to something imapd could handle.
  • In 2006, I left Quiotix, and switched my primary mail over to Tuffmail. I took my mail archive (in mbox file format) and put it up on my server machine, and serve that up with imapd. So I now have my mail split between two servers, but Thunderbird can deal with multiple servers just fine. I tried to move the archived mail to Tuffmail as well with several different tools (imapsync, offlineimap, Thunderbird bulk-copy) but I could never get a clean copy -- I suspect that the combination of crappy old multiply-converted mbox files and the old UW-IMAPD server is to blame.

Right now its still fragmented across a number of formats and servers. Yuck.

2 comments:

  1. I just went through migrating about 15 years worth of Mbox files to Gmail via IMAP. I wrote these scripts for managing it:

    http://github.com/relistan/mailtools

    The Mbox parser doesn't rely on anything crazy other than a blank line before each message and a properly formatted SMTP header. You might get some luck with it.

    ReplyDelete
  2. Hey Brian,

    Tuffmail would seem to be the email provider for true DIY geeks. I've used them for a couple years. Thing is, their spam filters are getting kind of stale, and feedback doesn't get much response. The entire operation appears to be somebody's hobby. Considering switching to 25mail.st.

    ReplyDelete