Monday, June 16, 2014
Solar -- one year in
The system was initially sized with the intention of completely canceling out our electric bill. Of course, there are a number of factors that can cause actual generation in any given year to vary from projected generation, of which variations in weather is probably the largest.
Overall, we came pretty close to hitting our target; our net electric bill for the year was $100, only about 5% of our actual electrical usage. I'll call that a success.
I knew that we'd get a lot more power generated in the summer than the winter -- longer days, better sun angle (power generation is proportional to the cosine of angle of incidence), clearer skies, and no snow on the roof. But I was surprised by just how big the swing was -- almost a 10x difference between generation in June and generation in December.
Below is a graph of cumulative power generation by month over the past twelve months; the Y axis is kWH.
Saturday, May 11, 2013
Solar -- one month in
We chose to lease rather than buy. A number of companies now offer leases on solar power systems, usually for a 20 year period, where they own and maintain the system, act as general contractor for procurement and installation, and handle the tax/permit/utility paperwork. Leasing was cost-competitive with buying -- and far less hassle and paperwork. The leasing company offered a choice of a monthly fee or a one-time payment. The one-time payment is a far better deal; the monthly payment basically represents financing at 7%. The install was completely hassle-free. The system uploads data to a monitoring site, so you can get graphs and reports of your production.
In Vermont, there are three sources of subsidy for solar: a 30% federal tax credit, a state tax credit based on generating capacity (which came out to about another 10%), and utilities buy your generated power at a premium. (Some utilities will only give you a credit; ours (Green Mountain Power) will cut a check for any credit balance at year end.) With a lease, the leasing company gets the tax rebates (which reduces the cost of the system) and the homeowner keeps any payments for the power generated, and the leasing company handled all the tax paperwork and associated risk.
I was apprehensive about whether to believe the projected generation capacity; with a month of data, I am gaining some confidence that they were reasonable. Based on these projections (and assuming that power rates stay the same), the system should offer a ten-year payback and a 8% return on investment. In hindsight, I would have gone with a slightly bigger system (there's still plenty of room on the roof); the standard approach seems to be to size the system to net out your power bill to zero, but this seems more of a psychological than financial target.
The key risk items are:
- Generation. The system may generate less than projected, though the first month looks promising. (To meet their targets, I need to average generating 26 kWh/day through the year. In May, we averaged 38 kWh/day; I would expect to generate even more in July/Aug and much less in Jan/Feb, but it is believable that we will hit this average.) Even if the projections are accurate, we are of course still dependent on weather.
- Changes in utility policy. Green Mountain Power offers an effective 7c/kWh subsidy on top of the regular tariff for any power we generate. However, the company could change this policy, and probably will sometime in the next twenty years.
- Change in power rates. If power rates go up, the return is better; if power rates go down, the return is worse. I have to assume over 20 years electric rates will go up.
Monday, June 7, 2010
Exception transparency in Java
Thursday, May 20, 2010
Memtest86+ rules!
About a week ago, the system started behaving badly -- IE crashing, Thunderbird crashing, and starting yesterday, the whole thing blue-screening. After wasting a lot of time trying to figure out "what software was updated recently", I started to suspect memory errors. So I ran the Windows memory test program that shows up on the boot screen -- nothing.
After more dorking around, I downloaded and ran MemTest86+ (www.memtest.org), burned it to a USB drive, and ran it. It immediately found thousands of memory errors; by trying various combinations and moving modules from slot to slot, I was able to identify the bad modules. I had bought Crucial's top of the line (Ballistix Tracer LED) from Newegg; the Crucial folks immediately shipped out a replacement.
Given how many errors MemTest found, its amazing that the Windows test found nothing.
Thumbs up for MemTest86+ and Crucial customer service. Thumbs down for Windows Memory Test.
Tuesday, May 18, 2010
Registration is open for the 2010 JVM Language Summit
Sunday, January 31, 2010
Book review: Fermat's Enigma
This is a nice little book about the history of mathematics and the 350-year quest for the proof to Fermat's Last Theorem. It was written by the fellow who wrote the BBC / Nova TV special on Andrew Wiles, but includes a lot more information than a one-hour show could. It does a nice job at hitting many of the high points of mathematical development from Pythagoras to modern day, including the "discovery" of zero, then negative numbers, then imaginary numbers, techniques for grappling with infinity, Turing-computability, and Godel's incompleteness theorem. It doesn't attack any of these in great depth, but it does provide a nice historical perspective while remaining about as accurate as a lay book can do. It also does a nice job of illustrating the near-hubris required for Wiles to lock himself in a closet for eight years in order to solve a problem that had eluded mathematicians for centuries. Mathematicians will enjoy the panorama; non-mathematicians will likely find the introduction to some of these obscure concepts accessible and enjoyable. Also by this author: The Code Book: The Science of Secrecy from Ancient Egypt to Quantum Cryptography .
(Recommended to me by: Stuart Marks.)
Our government, protecting us
We've recently gone on an "energy efficiency" rampage at the house, replacing bulbs with CFLs, identifying devices that are unnecessarily left on all the time, wrestling with Windows to stay asleep during periods of inactivity, etc. We also recently just installed a "continuous" or "on demand" hot water heater, replacing the 50G direct-vent tank heater we had (it was getting to the end of its lifetime and it was easier to replace it preemptively.)
Unfortunately, the state requires all newly install water heaters to have a thermostatic mixing valve that limits the water temperature to 120 degrees. (For tank systems, it is recommended to keep the tank water at 140, to prevent the bacteria that causes Legionnaire's disease, but 140 is hot enough to scald. But continuous systems have a control system for the output temperature, so can be safely kept at whatever temperature you program in.) And its probably not even working right, since the output temperature is even less than 120. The valve adds cost to the system and to the installation (probably a dozen additional welds in addition to the valve), and while we now have an infinite supply of hot water, generated more efficiently, its not as hot as we like it.
Reputable plumbers are not able to remove or bypass the valve, which means we need to either find a disreputable plumber or I need to do it myself (read: find an incompetent plumber.)
Note to lawmakers: in my many years of successful shower use, I've learned a secret trick to avoid getting scalded: put your hand under the water first -- if its too hot, turn down the water temperature before getting in!
Thanks, elected officials, for making my house systems both more expensive and less useful.
Saturday, January 23, 2010
e-mail packrat
I've long tried to keep all the e-mail I've ever sent or received; I've got an archive going back to 1985 or so, when I first realized that keeping e-mail might be a good idea. Trouble is, keeping such an archive in one place requires a fair amount of maintenance, because formats and protocols change. Is it worth the effort?
- Until 1987, I primarily used a VMS system.
- In 1987, I switched to a Unix machine at MIT. I was able to import my old VMS mail into whatever the mailbox format of the day was (mbox, probably) by a script I found somewhere.
- In 1992, I switched to using POP through the client program Eudora. Eudora stored the mail locally, in a folder format that was something like 'mbox', but not exactly. (For example, attachments were not stored inlined, but instead in external files.) I managed to import my old Unix mbox files into folders.
- In 2004, I switched from POP to IMAP. I went through an extensive process to convert my existing mail base into real mbox files that my IMAP server could read. I spent several days writing scripts to convert the Eudora pseudo-mbox files to something imapd could handle.
- In 2006, I left Quiotix, and switched my primary mail over to Tuffmail. I took my mail archive (in mbox file format) and put it up on my server machine, and serve that up with imapd. So I now have my mail split between two servers, but Thunderbird can deal with multiple servers just fine. I tried to move the archived mail to Tuffmail as well with several different tools (imapsync, offlineimap, Thunderbird bulk-copy) but I could never get a clean copy -- I suspect that the combination of crappy old multiply-converted mbox files and the old UW-IMAPD server is to blame.
Right now its still fragmented across a number of formats and servers. Yuck.
Saturday, January 9, 2010
zTunes released
As promised in yesterday's entry, I'm releasing my digital media management software (ztunes) to the world. It's hosted at github: http://github.com/briangoetz/ztunes. It is written in Ruby and based on "rake", the Ruby equivalent of "make". It currently has a long way to go but already does a lot.
You can download the Ruby gem here: http://github.com/briangoetz/ztunes/downloads. (It is not currently in any sort of gem repository.) It defines its gem dependencies, but you'll also need the Unix tools ffmpeg, flac, and lame. It will run on Linux and Mac but currently has some trouble on Windows since it is dependent on symbolic links for some of its functionality, which Windows doesn't support.
My motivation for writing this was that iTunes is really inadequate for managing a media library unless (a) you only want to play on iPod (or other Apple) devices and (b) you are willing to let iTunes be in control of ripping and encoding. This didn't work for us for two reasons: we have Squeezeboxes on all the stereos, and I want to rip my CDs to a non-proprietary, lossless format (that means flac, which iTunew doesn't support.) We also have music that has been aquired in various other forms (MP3s from Amazon, AAC from iTunes, WMA from Rhapsody) and want to be able to play all the music on all the devices, without transcoding it all down to a least-common-denominator. (In other words, if Squeezebox supports WMA but iPod doesn't, let Squeezebox play off the original WMA but let iPod play the transcoded version.) And this should be transparent to the rest of the family.
There are several basic tasks in managing the media library that zTunes automates:
- Content ingestion. I've got a "drop" folder, into which I want to drop the originals of my media, in whatever form, and have them be analyzed, metadata extracted, and filed into a unified library based on its metadata. My metaphor here is the gas tank of an M1 tank: you can pour anything combustible (gasoline, jet fuel, diesel, used cooking oil) into the tank and it figures out how to burn it. Currently it maps a media file to a filename by using the author/album/title tags for audio or the title tag for video; audio files are named like "The Who/Who Are You/Squeezebox.flac".
- Transcoding. Not all devices play all device types. So ingested content also needs to be transcoded into alternate formats, which are maintained as parallel directory trees. The transcoded trees are transient; they are merely shadows of the "authoritative" tree. Some files may need be transcoded to multiple formats; for example, video files ripped from DVD or transferred from TiVo might be transcoded to 480 x 320 video for iPhone but 320 x 240 for the older video iPods.
- Syncing. I use the Windows program "Tag&Rename" to edit the metadata tags on my media files, to normalize genres, naming details like "The Cars" vs "Cars, The", "Vol 1" vs "Disk Two", etc. When I edit the metadata on an "original" file, I'd like the file to be renamed accordingly, and metadata changes to be reflected in the transcoded copies. When I delete an original, I want the transcoded copy to go away. Etc.
- Device management. I would like to have a single directory for each device type, that I can point device-specific library management software (iTunes, Squeezecenter, Creative Explorer) at, and it will see the right view of the media library for that device (will only see files it can play; will see them in the "best" format available for that device.)
One thing it does not do yet is manage the integration of your external media library into iTunes (iTunes is particularly bad at dealing with files you didn't acquire through iTunes.)
See more in the README file here: http://github.com/briangoetz/ztunes/blob/master/README
I'm currently using this to manage a library of ~8,000 media files in half a dozen formats. I'd love to get some more users -- drop me a note if you're interested!
Friday, January 8, 2010
A busman's holiday
Since I've been working way too hard, of course I decided to spend my XMas break...programming. (http://www.answers.com/topic/busman-s-holiday). I had two goals: rewrite my digital-media handling software, and learn Ruby. I'm pretty happy with what I accomplished on both counts.
The motivation to rewrite my digital-media scripts came from having too many conversations like the one below with Stuart Marks:
SM: Hey, you wrote a bunch of scripts to manage audio and video files, are you willing to share them?
BG: Well, in theory, yes. But I'm kind of embarassed to show them to anyone...
SM: Let me guess. Perl?
BG: Yep.
SM: I have a Perl story...
BG: Don't bother -- all Perl stories end the same way.
I'll post the full details soon -- including links to the software on github -- but for now I'll just outline the problem I was trying to solve:
- Ingest digital media files in any format (MP3, AAC, WMA, WAV, FLAC, M4A, M4V, WMV, MP4, etc)
- File them into a library based on their metadata
- Additionally transcode them down to one or more "compressed" formats (MP3 for audio, iPhone-sized Mp4 for video) for memory-constrained devices, without letting go of the original
- Organize them so that each device (iPod, Squeezebox, non-iPod MP3 player) can play all the media, in the best format that the device can recognize natively (Squeezebox supports MP3, WMA, and FLAC; iPod supports MP3 and AAC; Zen supports MP3 and WMA) or a transcoded form if it can't. For example, for a given track whose source form is WMA, Squeezebox and Zen should see the WMA but iPod should see the MP3; for a track in FLAC, Squeezebox should see the FLAC but iPod/Zen should see the transcoded MP3.
Sunday, October 18, 2009
Is this a joke?
Attempt 1: 1.5TB drive, using trusty Unix tools -- put the drives into a Unix box, copy data with dd, then do the expansion with 'mfstools'. This is the approach I've used several times in the past, with good results. Put the new drive in, turn it on, and it gets stuck forever in the "Wecome, powering up" screen. Back to Google.
Turns out that the S3 can't see a partition bigger than 1TB, and mfstools expands the partition to the whole rest of the drive, yielding a too-big partition. Turns out mfstools doesn't support limiting the size of the partition, but the Windows version (winmfs) does, so I'll use that instead. (Its good to have lots of spare computers around when attempting any sort of upgrade.)
Attempt 2: 1.5TB drive, using winmfs. Put the drives in the windows box, run winmfs to copy the data, and let winmfs expand the partition. It asks me "should I limit the partition to 1TB", I say yes, good. Put the drives back -- same problem. More Googling.
So I discover that "some versions of the drive I was using (WD15EADS) are 'not compatible' with Tivo Series3." Its been years since I've heard about incompatible (system, disk) pairs, and this is a standard SATA drive, but OK, I guess I bought the wrong drive. RMA time. Sorry, NewEgg. The Tivo Upgrade FAQ (http://www.tivocommunity.com/tivo-vb/showthread.php?t=370784) is telling me I should favor the WD EVVS drives instead, so I buy a 1TB drive (WD10EVVS) from Amazon.
Attempt 3: 1TB drive, winmfs. I repeat the process, copying the 250G drive to the new 1TB drive, and put the drive back in the Tivo. (At this point I've learned to try it before I fasten all the screws.) Same deal -- stuck on the "Welcome, Powering Up" screen. More Googling.
I found this update, which was added after I'd bought my drive:
The WD10EVVS was removed from the list on October 10, because there is a new
batch of that drive, manufactured on September 20, that is not compatible
with the TiVo. These incompatible drives are labeled as follows:
MDL: WD10EVVS - 63M5B0
Product of Thailand
DATE: 20 SEP 2009
DCM: [b]HAxxxxxxxx
R/N: 701640
LBA: 1953525168
I looked at my drive, and sure enough, I had one.
Is this an elaborate joke?
Next up: RMA redux, ordered a WD 10EVDS drive. Stay tuned.
Update: installed the WD10EVDS, worked fine. Fourth time's the charm!
Thursday, October 15, 2009
You've been scammed!
I have spent about $10 with skype per year, recharging my skype account from PayPal when it ran low. Somehow (don't remember) I had authorized skype to charge my PayPal account when my balance got low. And this was the vector through which I was scammed. Someone must have gotten a hold of my skype password (don't know how), logged on, and billed EU250 to my PP account (which didn't require a PayPal webflow), and then allocated it to some bogus accounts.
First stop: dispute the charge with PayPal. They were completely unhelpful, pointing me to the authorization and told me to work it out with skype. Fortunately skype was more helpful, and they reversed the charge immediately.
I then logged on to my Skype Business Control Panel (now that I know such a thing exists), and found several bogus accounts linked to mine, which I deleted. After all was said and done, including the refund, I still somehow had a EU100 balance on my BCP, meaning somehow the scammers gave me EU100.
To see if you have any such preapprovals on file: login into your paypal account, click "Profile", and click "Preapproved Payments." You can delete them from there.
Saturday, August 29, 2009
Laptop upgrade annoyances
What I'd like is a simple way to move all the data from the existing drive to the new drive, and then just toss the old drive. But this isn't as simple as it might appear. Laptop IDE cables generally only support one drive, so I can't use (say) PartitionMagic to do a partition copy the way I would on a desktop system.
A lot of people have suggested various tricks, like:
- Get an IDE-USB adapter, put the old disk on that, put the new disk in the machine, boot from a Linux CD, and use dd to copy the data;
- Get a pair of 40 pin to 44 pin IDE adapters, put them in a desktop system, and copy using PartitionMagic (Windows) or dd (Linux);
- Find a dual-drive 44 pin IDE cable, plug both drives in, and hope that the OS / BIOS recognizes both disks;
- Just reinstall Windows and whatever apps I have on the new drive (including chasing down all the device drivers, such as the touch pad, speakers, etc)
Why is this so difficult? A hard-drive-swap should be a simple, common upgrade operation, that shouldn't require using tools from another operating system, transplanting the drives into another system, or rebuilding the world from scratch.
On a similar note, I just bought a Samsung NC10 netbook, and was going to wipe the disk and reinstall OSes. I have all the software I want ripped to ISO images, many of them bootable. Why is it so hard to take a bootable ISO and turn it into a bootable USB key? (I tried "unetbootin" but it didn't work on the PartitionMagic ISO, which is usually my first step in installing onto a new PC.)
Friday, June 12, 2009
WiFi prices finally come down
Finally there seem to be some better alternatives. Boingo now seems to have an unlimited $10/month plan, so I switched to that. Boingo claims I also get free roaming on many TMobile, AT&T, and other hotspots -- I'll report on that once I get my first bill. I downgraded my TMobile account to the "Pay as you go" plan, which has no monthly fee, and is $3 for the first hour, which seems like a good option to have.
Starbucks also has a reasonably priced plan (Starbucks Gold Card) if you spend a lot of time in or near Starbucks (they are in the process of switching their hotspots from TMobile to AT&T.) For $25/yr, you get two hours per visit of WiFi time (not sure if this is enforced or not), plus 10% discount on most Starbucks purchases.
Wednesday, November 19, 2008
Initial iPhone experience -- disappointing
I got out of the store and into my car, and noticed that the edge where the front metal rim meets the plastic case was extremely rough -- almost sharp enough to cut. This was not the seamless tactile experience I was expecting from Apple. So I went back in the store, and asked for an exchange. I was told that "Apple prevents AT&T from making exchanges" and was sent to the Apple Store. When I arrived at the Apple Store, the rep informed me that they could make an exchange, but it would be a refurb unit, not a new one, even though mine was clearly new, because I'd bought it at an AT&T store and not an apple store.
So I went back to the AT&T store and argued with the manager. He tried to send me back to Apple. He ended up calling the Apple store, who must have told him to take the exchange, so in the end I got a new, non-defective phone. All was made right, but the experience was none too pleasant, involving three store visits.
While in the Apple store, which had many iPhones on display, I took the opportunity to do some sampling. I discovered that many iPhones had rough or sharp spots, and not all in the same places. Seems that in reducing the cost of the 3G, perhaps some quality-control corners were cut as well, since many were not very pleasing to the touch and there were significant variations in perceivable quality.
Wednesday, October 15, 2008
My favorite computer science book

Pierce's Types and Programming Languages is a masterful introduction to the theory and practice of type systems. One of the things that makes this book so great is that it is equally accessible to both the theory-oriented and the practice-oriented. This was driven home to me in a conversation with Ola Bini, when I saw he was carrying this book, and he commented "I love this book because I can skip all the math and get what I need from the ML implementation." I answered that I liked it for the opposite reason; I was able to get everything I needed from the math and didn't have to look at the code. Its pretty impressive that a book can be that useful and successful from two such radically different reader approaches.
I found that Pierce's treatment was extremely accessible. He starts with almost no assumptions, introduces first the untyped lambda calculus, then the simply typed lambda calculus, some obvious extensions (records, references, subtyping, union types, functional objects, etc), operational semantics, and builds gradually to more useful type systems. Each section includes motivation, analysis, a formal description of the system, soundness proofs, and ML code; the impatient can skip some of these and still get what he's talking about. There is working code for each of the languages developed. (The type systems were developed in a system that the author wrote called TinkerType, which makes it possible to build type systems by "mixing and matching" features, and it generates both the ML code and TeX source for generating the figures used in the book -- most impressive!)
Not only is this book useful to anyone who is interested in the design and science of programming languages, but it is also a pleasure to read.
What's your favorite computer science book? (Unoriginality points for anyone who says TAOCP.)
Wednesday, September 17, 2008
David Foster Wallace, RIP
For me, the experience of reading Wallace's writing is not unlike that of watching an olympic gymnast. While the right side of the brain is being entertained by the grace and artistry, the left side is frantically marvelling at how the human body can do that at all. The tension between the two -- where your brain can't decide where to focus, not wanting to miss either part -- adds all the more to the experience.
Wallace's mastery of the language is undeniable; one could read his work simply to marvel at the construction of each sentence or his ability to move effortlessly from one writing style to another. But, unlike other authors known for their "style", the writing is merely the surface layer; Wallace actually has something to say, his arguments are compelling and challenging and beautifully constructed, and supported with relevant data drawn from disciplines ranging from literary theory to mathematics. And somewhere along the line he also manages to make you laugh out loud -- right before you have to pick up the dictionary for the seventh time.
One is, at the same time, amazed, informed, challenged, entertained, and, honestly, filled with that feeling of "I'm not worthy" on multiple levels.
I would like to be able to say "I knew him when"; he and I overlapped for a year or two at Amherst. But I never actually met him, I only heard the stories, such as his senior English thesis being published as a novel ("The Broom of the System"), or being the only student in then-recent memory to have achieved the distinction of summa cum laude for his thesis work in two separate majors (English and Philosophy.)
Harper's Magazine has graciously made the pieces he published in that magazine available for free on the web: http://www.harpers.org/archive/2008/09/hbc-90003557. If you've not had the pleasure, I suggest you read "Tense Present" -- which probes "the seamy underbelly of US lexicography" -- and then marvel at the notion of how entertaining and actually useful a book review of a dictionary could be.
Rest in peace.
Saturday, June 7, 2008
Wallboard + paint + pressure = superglue
So, my dad and I went to remove it from the wall. I removed the six screws, and we prepared to catch the mount. It didn't fall off the wall. We tugged on it, and it still didn't come off the wall. Seemed stuck so tight we thought we'd missed a screw! But we convinced ourselves there were no more screws, and the two of us pulled hard, and eventually it came off the wall -- taking some of the wallboard with it. Apparently the pressure of being screwed up against the wallboard (and maybe the heat from the TV too over a few years) turned the painted surface into a glue not only strong enough to hold a 40lb mount to a vertical wall but resist being pulled off!
Monday, June 2, 2008
Questions from the Peanut Gallery, part I
Q: About how much time did you put into your book (effort, not duration)?
A: In my case, the answers for effort and duration are the same, as I had the luxury of writing the book mostly-full-time -- I made the book my foreground activity, though I still did some consulting and training while I was working on it. I spent approximately 16 months on the book -- longer than planned (but in hindsight is no surprise.)
Q: Was the financial compensation worthwhile?
A: Unless your name is Stephen King or JK Rowling, writing books is not something you do for the financial compensation. This is more true for technical books, because (a) the audience for books like Java Concurrency in Practice is not quite as large as the audience for Harry Potter, and (b) if you have the skills to write a good technical book you probably have the skills to get a well-paying technical job. Without going into the details, I'll say that the compensation is about what I expected -- but I went into it with very realistic expectations. The compensation comes in other forms.
Q: How much support and assistance was provided by the publisher?
A: I think this is a matter of how much support and assistance you ask of the publisher -- and how much the publisher thinks you need. In our case, we did everything ourselves, including typesetting and managing the review, copy editing, and index creation. These are things the publisher often does for authors (and might even have preferred to do), but we chose to do it ourselves, and the publisher agreed. Of course, this was more work, but it was work we gladly did. The A-W team was always responsive when we did ask for things. So I think the answer is "as much as you appear to need."
Q: How does the short half-life of technical topics affect the effort?
A: I deliberately chose a topic with a longer half-life. This gave me the latitude to let the book tell me when it was done, rather than the schedule. For material with a shorter shelf life, I might be inclined to choose a shorter format, so that the book is less out-of-date by the time it is published.
Q: What would you say about books that authors release chapters to the public as they write?
A: I think this presupposes a style of writing where the author sits down and writes the book linearly. I am sure some authors do this, and some topics are more amenable to this approach than others. But one of the most important freedoms in writing is the freedom to refactor continuously; very often you don't figure out the right way to present the material until you've presented it the wrong way (just as with code.) There's nothing wrong with putting the work out there early -- this is a great source of free review -- but you have to be careful that doing so doesn't cause you to settle into the belief that the structure of the book has been decided. (The same risk is true of trying to adhere to a schedule that assigns due dates to specific chapters.)
Q: How do you avoid example source code exploding without using unrealistic examples?
A: This is really hard! But its really important. In JCiP, we set a rule for ourselves of "no code example more than a page", with the target of making most of them a half page or less. This is not easy, especially in Java! (There was only one we had to break into two separate one-page listings.) We wanted the examples to each illustrate a single point, so that the reader could look at the example and easily see what it was trying to show. There are some obvious tricks; eliminating boilerplate code like constructors, getters, and setters helps a little bit. What worked for us was to pick realistic examples that the audience would immediately understand the utility of (such as a file crawler), but abstract away the irrelevant concrete details by not showing the bodies of methods that are not needed to make the point that the example is supposed to illustrate. For example, we have a set of examples in Chapter 8 where we illustrate searching for solutions to a class of puzzles such as the "sliding block puzzles." But rather than focus on a specific puzzle -- which would take lots of space and not offer all that much insight, we abstract the nature of the puzzle by defining an interface that specifies the initial position (in terms of an abstract Position class), valid moves (in terms of an abstract Move class), and the goal position. Then we can illustrates various search techniques in terms of the abstract puzzle without getting bogged down in the details.
Q: What would you say is the role of technical books in the age where the Internet is the fastest way to publish texts and technology changes so fast that one year after publishing texts become irrelevant?
A: Some technical books are simply a form of documentation; any book that has a version number in the title is likely to fall into this category. These books have a very short shelf life. Other books, those that tend to focus on concepts rather that specific technical details, tend to have a longer shelf life. In any case, the publishing industry needs to become more agile in its approach to managing the authoring and production process, and explore more seriously alternate publication vectors such as electronic publishing.
More later.
Saturday, May 31, 2008
The making of JCiP; avoiding errors in code listings
One thing that we (mostly Tim, actually) did for JCiP was set up an infrastructure for the book, similar to what you'd do for a software project. Version control, issue tracking, one-step build script, continuous build -- all of these things offer the same benefits to book projects as they do for software.
One critical aspect of the build is the handling of program listings. It is incredibly tempting to cut and paste examples from the IDE into whatever source format you're writing in (Word, Frame, LaTeX, DocBook), but this is a recipe for disaster -- errors will invariably creep in as you try and make small tweaks (such as changing variable names) outside the IDE. And code examples with errors really undermine the reader's confidence (or worse, they copy the incorrect example into their code.) So, we wanted to make sure that every code example compiled (and ideally, was tested.)
Our approach was to check the code into Subversion with the rest of the book artifacts, ensure that the build process compiled the code and ran the unit tests, and then automatically extract the examples from the code in a format into which they could be directly included by the build. Some systems (LaTeX, DocBook) make this sort of inclusion easier than others.
We marked the examples up with comments for formatting (bold, italic) and also with "snip here" comments that excluded the irrelevant portions of the code from the listings that actually went into the book. The attached perl script (phragmite.pl), written by Tim Peierls (based on an approach designed by Ken Arnold), takes as input a set of input files and produces a set of LaTeX files representing the extracted listings.
As an example, here is the Counter listing from Listing 4.1 of JCiP:
// !! Counter Simple thread-safe counter using the Java monitor pattern
// vv Counter
@ThreadSafe
public final class Counter {
/*[*/@GuardedBy("this")/*]*/ private long value = 0;
public /*[*/synchronized/*]*/ long getValue() {
return value;
}
public /*[*/synchronized/*]*/ long increment() {
if (value == Long.MAX_VALUE)
throw new IllegalStateException("counter overflow");
return ++value;
}
}
// ^^ Counter
The first line identifies the type of the code fragment (!! for a "good example", ?? for a "bad example" which would get decorated with a Mr. Yuk), the name of the fragment (Counter), and the listing caption. The lines with the ^^ and vv mean "snip from here to here", and a listing can be made of multiple such fragments. The /*[*/ and /*]*/ comments mean "bold". The following ANT target ran the script:
<target name="listings">
<exec dir="${bin.dir}" executable="perl">
<arg value="${phragmite.pl}"/>
<arg value="${listings.dir}"/>
<arg value="${fragments.dir}/*.java"/>
<arg value="${fragments.dir}/jcip/*.java"/>
</exec>
</target>
In the book's LaTeX source, we use the following LaTeX macro to pull the listing in:
\newcommand{\JavaListing}[1]{\input{listings/#1}%

