Book review: Fermat's Enigma

This is a nice little book about the history of mathematics and the 350-year quest for the proof to Fermat's Last Theorem.  It was written by the fellow who wrote the BBC / Nova TV special on Andrew Wiles, but includes a lot more information than a one-hour show could.  It does a nice job at hitting many of the high points of mathematical development from Pythagoras to modern day, including the "discovery" of zero, then negative numbers, then imaginary numbers, techniques for grappling with infinity, Turing-computability, and Godel's incompleteness theorem.  It doesn't attack any of these in great depth, but it does provide a nice historical perspective while remaining about as accurate as a lay book can do.  It also does a nice job of illustrating the near-hubris required for Wiles to lock himself in a closet for eight years in order to solve a problem that had eluded mathematicians for centuries.  Mathematicians will enjoy the panorama; non-mathematicians will likely find the introduction to some of these obscure concepts accessible and enjoyable.   Also by this author: The Code Book: The Science of Secrecy from Ancient Egypt to Quantum Cryptography .

(Recommended to me by: Stuart Marks.)

Our government, protecting us

We've recently gone on an "energy efficiency" rampage at the house, replacing bulbs with CFLs, identifying devices that are unnecessarily left on all the time, wrestling with Windows to stay asleep during periods of inactivity, etc.  We also recently just installed a "continuous" or "on demand" hot water heater, replacing the 50G direct-vent tank heater we had (it was getting to the end of its lifetime and it was easier to replace it preemptively.) 

Unfortunately, the state requires all newly install water heaters to have a thermostatic mixing valve that limits the water temperature to 120 degrees.  (For tank systems, it is recommended to keep the tank water at 140, to prevent the bacteria that causes Legionnaire's disease, but 140 is hot enough to scald.  But continuous systems have a control system for the output temperature, so can be safely kept at whatever temperature you program in.)  And its probably not even working right, since the output temperature is even less than 120.  The valve adds cost to the system and to the installation (probably a dozen additional welds in addition to the valve), and while we now have an infinite supply of hot water, generated more efficiently, its not as hot as we like it. 

Reputable plumbers are not able to remove or bypass the valve, which means we need to either find a disreputable plumber or I need to do it myself (read: find an incompetent plumber.) 

Note to lawmakers: in my many years of successful shower use, I've learned a secret trick to avoid getting scalded: put your hand under the water first -- if its too hot, turn down the water temperature before getting in!

Thanks, elected officials, for making my house systems both more expensive and less useful. 

e-mail packrat

I've long tried to keep all the e-mail I've ever sent or received; I've got an archive going back to 1985 or so, when I first realized that keeping e-mail might be a good idea. Trouble is, keeping such an archive in one place requires a fair amount of maintenance, because formats and protocols change. Is it worth the effort?

• Until 1987, I primarily used a VMS system.
• In 1987, I switched to a Unix machine at MIT. I was able to import my old VMS mail into whatever the mailbox format of the day was (mbox, probably) by a script I found somewhere.
• In 1992, I switched to using POP through the client program Eudora. Eudora stored the mail locally, in a folder format that was something like 'mbox', but not exactly. (For example, attachments were not stored inlined, but instead in external files.) I managed to import my old Unix mbox files into folders.
• In 2004, I switched from POP to IMAP. I went through an extensive process to convert my existing mail base into real mbox files that my IMAP server could read. I spent several days writing scripts to convert the Eudora pseudo-mbox files to something imapd could handle.
• In 2006, I left Quiotix, and switched my primary mail over to Tuffmail. I took my mail archive (in mbox file format) and put it up on my server machine, and serve that up with imapd. So I now have my mail split between two servers, but Thunderbird can deal with multiple servers just fine. I tried to move the archived mail to Tuffmail as well with several different tools (imapsync, offlineimap, Thunderbird bulk-copy) but I could never get a clean copy -- I suspect that the combination of crappy old multiply-converted mbox files and the old UW-IMAPD server is to blame.

Right now its still fragmented across a number of formats and servers. Yuck.

zTunes released

As promised in yesterday's entry, I'm releasing my digital media management software (ztunes) to the world.  It's hosted at github: http://github.com/briangoetz/ztunes.  It is written in Ruby and based on "rake", the Ruby equivalent of "make".  It currently has a long way to go but already does a lot. 

You can download the Ruby gem here: http://github.com/briangoetz/ztunes/downloads.  (It is not currently in any sort of gem repository.)  It defines its gem dependencies, but you'll also need the Unix tools ffmpeg, flac, and lame.  It will run on Linux and Mac but currently has some trouble on Windows since it is dependent on symbolic links for some of its functionality, which Windows doesn't support.

My motivation for writing this was that iTunes is really inadequate for managing a media library unless (a) you only want to play on iPod (or other Apple) devices and (b) you are willing to let iTunes be in control of ripping and encoding.  This didn't work for us for two reasons: we have Squeezeboxes on all the stereos, and I want to rip my CDs to a non-proprietary, lossless format (that means flac, which iTunew doesn't support.)  We also have music that has been aquired in various other forms (MP3s from Amazon, AAC from iTunes, WMA from Rhapsody) and want to be able to play all the music on all the devices, without transcoding it all down to a least-common-denominator.  (In other words, if Squeezebox supports WMA but iPod doesn't, let Squeezebox play off the original WMA but let iPod play the transcoded version.)  And this should be transparent to the rest of the family.

There are several basic tasks in managing the media library that zTunes automates:

• Content ingestion.  I've got a "drop" folder, into which I want to drop the originals of my media, in whatever form, and have them be analyzed, metadata extracted, and filed into a unified library based on its metadata.  My metaphor here is the gas tank of an M1 tank: you can pour anything combustible (gasoline, jet fuel, diesel, used cooking oil) into the tank and it figures out how to burn it.  Currently it maps a media file to a filename by using the author/album/title tags for audio or the title tag for video; audio files are named like "The Who/Who Are You/Squeezebox.flac". 
• Transcoding.  Not all devices play all device types.  So ingested content also needs to be transcoded into alternate formats, which are maintained as parallel directory trees.  The transcoded trees are transient; they are merely shadows of the "authoritative" tree.  Some files may need be transcoded to multiple formats; for example, video files ripped from DVD or transferred from TiVo might be transcoded to 480 x 320 video for iPhone but 320 x 240 for the older video iPods. 
• Syncing.  I use the Windows program "Tag&Rename" to edit the metadata tags on my media files, to normalize genres, naming details like "The Cars" vs "Cars, The", "Vol 1" vs "Disk Two", etc.  When I edit the metadata on an "original" file, I'd like the file to be renamed accordingly, and metadata changes to be reflected in the transcoded copies.  When I delete an original, I want the transcoded copy to go away.  Etc.
• Device management.  I would like to have a single directory for each device type, that I can point device-specific library management software (iTunes, Squeezecenter, Creative Explorer) at, and it will see the right view of the media library for that device (will only see files it can play; will see them in the "best" format available for that device.) 

One thing it does not do yet is manage the integration of your external media library into iTunes (iTunes is particularly bad at dealing with files you didn't acquire through iTunes.) 

See more in the README file here: http://github.com/briangoetz/ztunes/blob/master/README

I'm currently using this to manage a library of ~8,000 media files in half a dozen formats.  I'd love to get some more users -- drop me a note if you're interested! 

A busman's holiday

Since I've been working way too hard, of course I decided to spend my XMas break...programming.  (http://www.answers.com/topic/busman-s-holiday).  I had two goals: rewrite my digital-media handling software, and learn Ruby.  I'm pretty happy with what I accomplished on both counts.

The motivation to rewrite my digital-media scripts came from having too many conversations like the one below with Stuart Marks:

SM: Hey, you wrote a bunch of scripts to manage audio and video files, are you willing to share them?
BG: Well, in theory, yes.  But I'm kind of embarassed to show them to anyone...
SM: Let me guess.  Perl?
BG: Yep.
SM: I have a Perl story...
BG: Don't bother -- all Perl stories end the same way.

I'll post the full details soon -- including links to the software on github -- but for now I'll just outline the problem I was trying to solve:

• Ingest digital media files in any format (MP3, AAC, WMA, WAV, FLAC, M4A, M4V, WMV, MP4, etc)
• File them into a library based on their metadata
• Additionally transcode them down to one or more "compressed" formats (MP3 for audio, iPhone-sized Mp4 for video) for memory-constrained devices, without letting go of the original
• Organize them so that each device (iPod, Squeezebox, non-iPod MP3 player) can play all the media, in the best format that the device can recognize natively (Squeezebox supports MP3, WMA, and FLAC; iPod supports MP3 and AAC; Zen supports MP3 and WMA) or a transcoded form if it can't.  For example, for a given track whose source form is WMA, Squeezebox and Zen should see the WMA but iPod should see the MP3; for a track in FLAC, Squeezebox should see the FLAC but iPod/Zen should see the transcoded MP3.