That gave me the excuse and opportunity to go off and invest some time in learning something new. I could have whipped up a script in Perl or Python or PHP, or written an XSL transform, but I opted to try out Go for this task. I've been introduced to Go twice in the past two years and was impressed by the language, but there's only so much you absorb in a one-hour workshop, and unless you need to get real work done, you never really learn a programming language.
Thus, I present to you my first crappy Go program: eds_holdings.go As I wrote this, I noted:
I appreciate the clear reference documentation at http://golang.org but it would really benefit from more inline examples. I ended up writing the XML parsing portion of this using xml.Unmarshal primarily because there was an example for it. Of course, Unmarshal parses the entire document into structures in memory at once... I knew that wasn't what I wanted, but for whatever reason I didn't find any mention of "SAX" or "event" or "pull" that would lead me to a stream-oriented, low-memory parsing option on the page.
However, the #go-nuts IRC channel on Freenode gave me a reply within minutes, pointing me at xml.Decoder for that purpose. Which is really great - a supportive community is critical. My only problem is that without a simple example to follow, I didn't want to dive into rewriting my quasi-working code, so I ploughed onward.
My current approach to I/O is far from optimal. Not only am I parsing the entire XML structure in memory, I'm also reading the entire XML file into memory to begin with (via ioutil.FileRead as a natural outgrowth of my "begin by parsing a hard-coded string"). Don't follow my example! Please point me at a better example
The standards and consistency wonk in me likes that Go delegates whitespace wars to go fmt <filename>. Problem solved and time-wasting arguments averted for everyone.
Go is (subjectively) fast and for all of my worrying about in-memory work, it was pretty lean -- at its maximum, consuming less than 200 MB of RAM while iterating over a 32 MB XML file. For comparison, Firefox was consuming around 750 MB the entire time.
In summary: I enjoyed writing in Go, and hope to find an excuse to do it again. Also, I have much to learn!