Programming languages

For the past couple of days, I've been playing with Casey Durfee's code that uses Solr and Django to offer a faceted catalogue. My challenge? Turn a dense set of code focused on Dewey and Horizon ILS into a catalogue that speaks LC and Unicorn. Additionally, I want it to serve as both a proof of several technologies (Solr for faceted searching and Django as a Web application framework) to my colleagues and as a reasonable backup catalogue for when our main catalogue fails (as it all too often does).

I emailed Casey today to tell him that I had a number of patches to contribute as a result of my experiments. It turns out that he's not really interested in pursuing this particular project much further, so he gave me his blessing to take his throwaway code and do whatever I want with it. Thus, the emergence of the FacBackOPAC project on code.google.com. If there's a grant out there for worst project name ever, this project's in the running... Anyways, I have contorted Casey's code so that it supports both Dewey and LC, and with a bit more torture it should be flexible enough to support both Horizon and Unicorn.

Right now I've twisted it all the way to meet my Unicorn needs and consequently have broken Horizon support, but it won't take much to make it support Horizon again - or any other ILS, for that matter. The main requirement is that you have to be able to get your MARC records and holdings out of your ILS. A secondary requirement is to know how to create links to detailed item views in your current catalogue, because this thing does not yet have any current awareness about item status.

There. My itch has been scratched for the time being. Go play with the FacBackOPAC project -- I even have (very) rough documentation on how to get the pieces installed andthe MARC records indexed, although you'll have to dig through the source in the Django catalog tree to overcome some hardcoded strings and URLs for the time being. Don't worry, pulling that hardcoded stuff out of the templates is high on the list of priorities.

So, a huge thank you to Casey for freeing this code and making this possible. For something he considers throwaway code, I've learned a lot from walking through it and making it start to meet my needs. I hope it helps you, too!

Update 2007-03-18:

Edited links to point to the FacBackOPAC project page, rather than the wiki (which is subject to change, and which did -- breaking the dang links in the original version of this story. Argh!)

I gave a lightning talk at the code4lib conference today on File_MARC for PHP introducing the File_MARC library to anybody who hasn't already heard about it. I crammed nine slides of information into five minutes, which was hopefully enough to convince people to start using it and provide feedback on what could be improved or smoothed out... I'm looking forward to chatting with more people about it over the remaining days of the conference. I've already introduced a few people to PHP's simple tests for test-driven development, so even if nobody ends up using File_MARC, at least people are learning about some of the hidden gems in PHP.

The funny thing is that I had originally pitched a full session talk on this subject for the conference, and it didn't make the cut of the ruthless democracy that is code4lib. In retrospect, even a twenty-minute thunder talk probably would have been too much information for anybody but the most die-hard PHP and MARC coder out there; the lightning talk was a perfect format for the talk. I hope to let the documentation do most of the talking in the future.

Here are the slides from the talk in OpenOffice.org and PDF format. Enjoy!

/me notes that UnAPI would be useful here... Gotta try and submit a patch to the s9y repository...

2006 was a year full of change - wonderful, exhausting change. Here's a month-by-month summary of the highlights of 2006:

January

I did a whole lot of work on the PECL ibm_db2 extension, reviewed a good book on XML and PHP, and finally fixed up my blog a little bit. I've got a few more book reviews in the works for 2007, and hope I can spruce up good old Coffee|Code a little more.

February

I live blogged a few WWdN invitational poker tourneys, wrote about Larry Menard's work on building an ADOdb driver for DB2 based on the ibm_db2 extension (which is now officially supported), documented my installation of a bird feeder (update: Spook was happy because a few chickadees visited, but Amber hasn't really noticed yet), announced that I was leaving IBM, and apologized for having to back out of some engagements due to the change in jobs. I also promised to write up some fixes to PDO_ODBC and spread out my documentation efforts for PHP, neither of which really happened in 2006 (8 commits to phpdoc ain't all that) -- but I recently made use of the xmlwriter extension in implementing MARCXML support for File_MARC and noticed that xmlwriter badly needs a tech writer. So maybe I can pull it together in 2007.

March

More live blogging of WWdN tournaments. How did I manage to have this much time in the first month of my new job? Amazing. I also helped horny teenagers find relief with GTA: San Andreas. But, in the middle of all that completely trivial stuff, our friends Mike and Kelly threw an incredible baby shower for us. It was amazing to be surrounded by so many friends who were actually happy that we were procreating! I do wish that we could visit more often, but based on last weekend's shenanigans this travel thing is starting to take more of a toll on Amber and her parents.

April

Lynn and I attended an vaudeville-style horror show. Who says Sudbury doesn't have culture? I took my readers on my walk to work, and I should note that within a couple of months of taking on the new job I had lost a bunch of weight. Activity is good! I live-blogged a couple more WWdN invitationals, and Lynn and I waited and waited for the baby to arrive, including such fine events as attending Earth Day and the inaugural Nickel City Triathlon Team barbecue.

May

We continued to wait for Amber to arrive, and managed to fit in a viewing of V for Vendetta during pre-labor. Eventually Amber did decide to join us in the outer world (saving me from having to run a 10K--yay!), and I posted the pictorial evidence. Oh, and I live-blogged some more WWdN tourneys.

June

I rode around Sudbury, posted some more Amber pics while putting my antisocial tendencies on display, and rather shockingly announced my departure from online poker. Shortly thereafter, the withdrawal symptoms kicked in and I unleashed in my first library-oriented post about our library system vendor. Looking back, I can't believe it took me almost four months to publicly snap at our vendor.

July

I started my parental leave, and oddly enough wrote two blog entries in all of July. I guess I was actually spending significant quality time with Amber -- cool! On Canada Day, I ran my first 5K race in about five years. Unfortunately, the other entry dealt with the loss of my co-worker, Alain Letourneau. I found out later that some of his friends from library school only found out about his death through my brief memorial to Alain, which is a bittersweet result I suppose. Tomorrow, a new librarian who was hired to fill Alain's position is joining the library; we won't forget Alain, but in a way it will be nice to have someone new moving around that office so that I don't have to walk past that empty room and locked door anymore.

August

During the second month of my parental leave, I focused a bit more on communicating to the outside world and posted some more pictures of Amber (although I still have to install that baby gate correctly) and pictures of the French River bridge and Bell Park dragon boat races. I kicked off my efforts to build a MARC package for PHP, which ultimately spawned another package to implement a basic linked list structure. Lynn, Amber, and I made it to a couple of bad movies, and I finished my first sprint duathlon. Lynn gets the top kudos, though, for running in a Try-a-Triathlon just three months after bringing Amber into the world. This year I'm hoping to do an Olympic distance triathlon, although I understand that some swimming skill is required for that...

September

I started out the month a little bit bitter after responding to a call for a wiki on the MARC listserv, only to be told later "thanks but no thanks, we're working on something ourselves". Eventually it ended up getting hosted on pbwiki where everybody shares a single user account (shrug). Lynn, Amber, and I ran another 5K in our neighbourhood. I wrote about some recent examples of how open source works based on the PEAR proposal process and PHP ext/filter API discussions, and noted how pseudo-open-source doesn't work after I found that IBM developerWorks had completely pulled the Mapuccino project. Come to think of it, I never did find a good open-source Web site visualization tool. Late in the month, I reflected on the role I'm fulfilling as a laundry list system librarian. That kind of role satisfies my generalist nature, even though it's a bit overwhelming at times.

October

I attended Access 2006 and felt like I had finally met other members of my own species in the flesh (although I had previously met many of the same people on #code4lib, it was nice to put names to faces and share thoughts over food). After taking a ton of notes, I called for all future Access conferences to require presenters to make their presentations available. I took a trip immediately after Access 2006 to Huntsville, Alabama for a week of training by our library systems vendor, and noted how horrible their customer experience was for sales and training pitches. Maybe their new owner will take a gander at some of their customer surveys... My PEAR proposal became an official PEAR package, and I revised File_MARC based on the results of the Access 2006 Hackfest (where I learned that loading a 500 Mb file into memory and then parsing it doesn't work too well with most systems).

November

Amber dressed up for Hallowe'en, and File_MARC became an official PEAR package. I discovered that Archimedes uses Apache Derby as its database of choice, but was disappointed with the "guessing game" user interface that broke all known usability rules. I took a bus/train journey down to Windsor for the Future of the ILS Symposium, got a sneak preview of what BiblioCommons is up to, and pressed Mike Rylander for details on how Evergreen / Open-ILS supports internationalization. I began trying in earnest to build a VMWare image running Evergreen, first with Ubuntu and then with Gentoo, before stalling out somewhere in early December. This is one ball I intend to pick up again in early 2007 -- we badly need a backup OPAC. Somehow I failed to mention the week that Lynn, Amber, and I spent in Cuba for my sister-in-law's wedding.

December

In December I was fairly quiet, as I was highly focused at work on finishing a number of projects that I had artificially set year-end deadlines for myself. To be honest, I just didn't want to spend my Christmas break thinking about them... I noticed that SirsiDynix made an odd press release on December 22nd, and indulged in wild speculation over what that meant. Library Journal picked up my blog post and quoted my creative conjecture, spawning several other posts on the topic, and I resolved to take the power of the blog a bit more seriously in the future. On December 21st I started responding to an LWN article that I felt misrepresented the state of PHP security; although I wondered at times if the holiday egg nog had me tilting at windmills, the author of the article ultimately agreed with me. On Christmas eve I gave the present of more Amber photos.

So, all in all, it was a pretty full year of geekdom, some regular exercise, a bit too much poker, a ton of travel, and a whole lot of change. There wasn't nearly enough Amber (of course there can never be enough), even though I have her all to myself a couple of mornings each week. But I'm living with the people that I love, doing fulfilling work, and that's all I can really ask for.

One of my favourite online publications, the Linux Weekly News, recently published an article called The state of PHP security. Given Stefan's departure, the great taint debate, the addition of ext/filter in 5.2.0 and all of the associated security changes in both the 5.2.x and the 6 branches, I settled down to enjoy a nice pre-Christmas read. I was hoping for some provocative thoughts about the direction that PHP has been taking for the last six months or so in the arena of security.

Unfortunately, I was greatly disappointed. Beyond using Stefan's departure as a kicking-off point for the article, the author didn't even mention any of these issues. Instead, he simply rehashed the history of PHP design missteps (magic_quotes, register_globals, allowing URLs in include) and noted that many PHP tutorials rely on dangerous practices.

What bothered me the most, however, was the author's decision to paraphrase a quote Rasmus gave in an interview from 2002 without explicitly noting that the quote was from 2002. The sentence in the article, talking about register_globals, is:

It is an extremely dubious feature, but one that PHP creator, Rasmus Lerdorf, seems to think should have been left on by default.

Would it have been too much for the author to have actually asked Rasmus if he might have changed his mind in the past five years? Or perhaps the author could have done a little more research and dug up the PHP 6 planning meeting minutes that state that register_globals and magic_quotes were going to be removed entirely from the language. Instead, the author concludes with the following statement:

Security seems to fall somewhere below simplicity in the minds of the PHP language developers; that makes it more difficult to have secure PHP applications. Security is a hard problem and any attempt to 'dumb down' a language is likely to run into security issues. Encouraging amateur programmers to write web applications is unlikely to produce secure code in any language, but by providing tutorials and examples that have glaring security issues and by not concentrating on teaching secure coding, PHP makes it that much worse. A great deal of useful code has been written on the PHP platform; it would be nice to find a way to keep that code coming while simultaneously making it more secure.

The first sentence in that statement is the most damning of PHP developers, but it entirely ignores the evidence exhibited in the changes we've seen in PHP 5.2.0 and that are in the works for PHP 6. The third sentence, oddly enough, attributes the existence of "tutorials and examples that have glaring security issues" to PHP itself, as though the language itself or the core developers of the language have the ability to prevent insecure tutorials from being published.

So I launched into the fray and attempted to right those injustices, perhaps a bit too passionately -- but so be it. I've been pretty quiet in the PHP world for the past while, outside of my little PEAR projects, but I still care about the language.

If I can glean anything from this article, it suggests that it might be a good idea to revamp the php.net landing page and documentation a bit to try to highlight tutorials that teach developers how to write secure PHP applications. Right now the landing page is largely a bulletin board for events. It might benefit, say, from a prominent and permanent link to the PHP Security Consortium (if that project is actually still alive--the last posted article dates back to March 2005). We may also want to improve the visibility of the security chapter of the manual (although briefly revisiting the section on SQL injection suggests that we need to revise it to encourage the use of PDO and placeholders).