The recorded version of the Evergreen episode of the FLOSS Weekly show was released over the weekend. I'm happy to say that Lynn watched it without looking too pained at any given point, and the Evergreen project has already had several responses to our plea for assistance so far, particularly on the packaging front, which is fantastic! Just having one more skilled helping hand makes all the preparation for and stress about the show worth it.

Several points that amused me about the show as I glanced over Lynn's shoulder:

• In Randal's introduction, he said that I "worked for Coffee|Code", full-stop. Aside to Leila Wallenius, the University Librarian of Laurentian University: no, there's nothing I need to tell you, I'm still a full-time employee at the University and I'm not planning on going anywhere! (That said, Coffee|Code Consulting is a registered sole proprietorship that provides small blocks of consulting services for Evergreen software in my spare time).
• For the first half of the show, my affiliation was shown as the (misspelled) http://coffecode.net. So of course I immediately ran out and bought that domain.
• Co-host Dan Lynch expressed a wish that his own show, Linux Outlaws, had a guest list like FLOSS Weekly. Oddly enough, some time ago when the subject of librarians and their fanatical devotion to open access to information came up on Linux Outlaws, I had submitted a feedback form on their site saying (essentially) "hey, if you want to talk to a Linux-loving free software-developing librarian some time, I'm around..." but I think that comment went into the ether.

If I ever do a video interview like this again, I'm going to try to:

• Prop my laptop up on a couple of LCSH books or attach a separate web cam at a proper height so it doesn't appear that I'm looking down on the viewer.
• Cut down on the "uhm"s and "ahh"s and stare a bit more robotically at the camera instead of rolling my eyes as I rack my brains to come up with an answer
• Stop rambling and trust the interviewers to take the show in the direction that their audience will be interested in instead of trying to jam in points that I think are important or interesting
• Ensure that my erstwhile partner in crime has a better Internet connection

Last week, I started my summer vacation with a weekend at a friend's cottage. By Tuesday I was deeply engrossed in some Evergreen enhancement work for the International Institute of Social History. I'm building an authorities management user interface that properly exposes Evergreen's powerful authority support in the 2.0 release: browsing authority lists, editing authorities and having the updates ripple through to the bibliographic records with controlled fields, merging and deleting authorities... here's a screenshot of the interface in progress: . The numbers represent the number of bibliographic records linked to each authority record. These are still early days, but I think there are some cataloguers that are going to be pretty excited about this functionality when they get their hands on it.

This week, I'm on location in the Robertson Library at the University of Prince Edward Island doing some Evergreen consulting work for them. The good people at UPEI have put my family and I up in a nice cottage on the island, so I'm toiling away at improving Evergreen during the day while my family explores the island. Melissa Belvadi and Grant Johnson have put together a list of pain points that they would like me to address that happen to mesh nicely with general pain points that have come up over the years on the Evergreen mailing lists. My first priority has been to make working with spine labels a little less aggravating. I'm happy to say that after a day and a half, I've been able to teach the spine label editor how to (*gasp*) move up and down with the arrow keys and (*ooh-ahh*) insert and delete new lines and (*w00t*) have the spine label defaults come from library settings that only have to be set once instead of being individually set by each cataloguer. Oh, and I've added font size, font weight, and font family to those settings so that you can have 20 pt. bold Helvetica spine labels if you want them.

All of this code is being committed to Evergreen trunk as I hit functionality milestones; much of the authority work has made its way into the Evergreen 2.0 alpha release that was cut on Monday (although not yet announced officially). On Monday I also cut the OpenSRF 1.6.0-alpha release and uploaded a virtual image built on Debian Squeeze reflecting the OpenSRF/Evergreen alpha releases to http://evergreen-ils.org/~denials/Evergreen_trunk_2010_08_23.zip (note that it's 500 MB, and does not come with X installed, so it's primarily aimed at users that are already familiar with Evergreen and just want to see the new stuff without having to go through the entire install process).

I did take some time off of Evergreen development this afternoon, as I was honoured to be one of the two guests on the FLOSS Weekly podcast. Mike Rylander and I were there to discuss Evergreen with the hosts, Randal Schwartz and Dan Lynch. Unfortunately for Mike, me, and the audience, Mike's Skype connection kept dropping and I had to do the bulk of the talking. Despite missing the contributions from Mike's massive brain, I'm told that the show went well. So if you're interested in hearing a bit about Evergreen and why I do what I do, keep an eye open for the interview at http://twit.tv/floss132 - it should be edited and online by Friday, August 27th at the latest. I tried not to swear too often so they wouldn't have to do much editing work - heh.

Finally, somewhere in there I celebrated another birthday. Oh yeah! Older? Yes! Wiser? Probably not.

As a librarian who works at a library that primarily uses the Library of Congress classification scheme, I have been interested for a long time in teaching Evergreen to be aware of call number schemes other than Dewey. The problem, in a nutshell, is that Evergreen simply applies an alphabetical sort against the the uppercased version of the call number when generating call number browser displays - resulting in LC call numbers that sort incorrectly, like:

• K 215 .E53 W37 1997
• K 22 .U748 v.18

When the subject recently came up on the open-ils-general mailing list, I decided to follow up with some code. So, as of this weekend, Evergreen trunk now has a generalized infrastructure for generating sort keys for call numbers. The broad strokes of the current implementation are:

• The classification scheme is set the level of the call number.
• Classification schemes are defined in the asset.call_number_classification table with a pointer to a database function to call to generate a normalized sort key for the given call number.
• Three classification schemes are available out of the box:
  • Generic (the default) - a simple normalization approach that produces reasonable results in the absence of special rules for Cutters, etc
  • Dewey (DDC) - a normalization routine taken from the Koha C4::ClassSortRoutine::Dewey Perl module
  • Library of Congress (LC) - a normalization routine that simply wraps Bill Dueber's excellent Library::CallNumber::LC Perl module
  - and adding more classification schemes is just a matter of adding another row to the asset.call_number_classification table and the appropriate sortkey-generating database function.

Note that this is the first time, to my knowledge, that Koha code has been adopted directly by Evergreen. I included attribution for the copyright holders in both the Generic and Dewey normalization functions. I wrote the Generic implementation in Evergreen from scratch shortly after taking a look at Koha's approach, so in some corners my work would be considered a "derived work". Koha's Dewey normalization function was (somewhat surprisingly) the only open-source implementation that I could find for Dewey, so it made perfect sense to me to adopt that for use in Evergreen. Many thanks to Koha for their use of the GPL v2 or later licence!

There are still some limitations and low-hanging fruit that I hope to address in the near future:

• Right now you can only manipulate classification schemes via SQL. The Holdings Maintenance dialogue needs to give cataloguers the ability to set the classification scheme for each call number, because I'm sure they don't want to drop down to the command line. This setting should probably be sticky during a given session, so that if they're processing a cart of government docs, they won't have to change the scheme from the default to CODOC for each item.
• Speaking of defaults, each library needs to be able to define a default classification scheme - so your consortium can have a Dewey library and an LC library and a SUDOC library, and their preferences won't trample each other. This can just be a simple org-unit setting.
• Following on Mike Rylander's advice, the asset.call_number_classification table should gain a new column that lists the field/subfield combinations used to find the appropriate call number (if any) for each scheme in a given bibliographic record. Then the Holdings Maintenance dialogue can offer the appropriate call number based on the classification scheme.

As part of the informal partnership between the International Institute of Social History (IISH) and Project Conifer, I was pleased to be able to spend the last two weeks in Amsterdam, working side-by-side with one of the Institute's developers, Ole Kerpel, on augmenting the support for MARC21 authorities in Evergreen. To prepare for the work session, I had posted a blueprint for the authorities work on the Evergreen Launchpad instance and circulated the list of requirements we had been asked to address to the broader Evergreen development community. We were fortunate to have the attention of Mike Rylander on the proposal, who not only supplied suggestions for how to implement some of the items, but also committed significant code contributions to the effort that greatly assisted our efforts. Here is a summary of the goals we accomplished in the current development branch of Evergreen (targeted for the 2.0 release), followed by a list of the outstanding items and my finger-in-the-air estimate of how much more time it would take to accomplish each of the tasks:

Accomplishments

• Controllable control numbers

  While not, strictly speaking, a requirement for authority control in and of itself, the ability to ensure that the behaviour of the 001/003/035 fields all conformed to the MARC21 specifications was an important requirement for IISH. They plan to provide external access to their authority and bibliographic records, so making the official identifier fields linkable based on the underlying record ID was an important aspect of the work. We implemented this feature as an optional database-level trigger to ensure that the control numbers and control number identifiers are always perfectly in sync with the internal identifier of the particular system on which the records are stored.

• Links

  Where having Mike Rylander participate in your review process pays off, part one... Before I even arrived in Amsterdam, Mike implemented a tricky database trigger that tracks the links between a given bibliographic record and the authority records to which it links. The links are tracked at the database level, as well as directly in one or more 0 subfields in each field that is controlled by an authority record. Yes, a given field in a bibliographic record can be controlled by two authority records and it all works. Nice, Mike!

• Syncs

  Where having Mike Rylander participate in your review process pays off, part two... Mike also implemented the bulk of the logic for automatically updating bibliographic records that are linked to a given authority record when that authority record is modified. Yes, folks, when you add a death date to an authority record, it will automatically appear in the corresponding bib records.

• Control an uncontrolled set of bibliographic records

  You may have dealt with library systems in the past that use some sort of string matching to implement authority support. As noted above, Evergreen is not like that. However, this means that many of us, when migrating to Evergreen, have bibliographic records lacking the 0 subfields that are required for full authority support. Towards that end, I wrote a script that will walk through a set of bibliographic records, search for matching authority records for each controllable field in each bibliographic record, and add the required 0 subfields to the bibliographic records. It certainly won't be a fast solution, but you should only need to do it once, and it worked on the limited test cases that we had ready at hand.

• Teach the MARC editor about authority records

  The MARC editor knew all about fixed fields for bibliographic records, and provided a handy grid for editing those fields. However, it didn't even know how to recognize authority records, and presented a fixed field grid that was absolutely meaningless. I spent a chunk of time laboriously transcribing the fixed field rules from MARC documentation into the MARC editor and now the MARC editor presents a reasonable fixed field grid for your editing convenience.

• Merge authority records

  Something that often happens in a library is that two authority records are created that identify the same thing. Eventually somebody notices the problem and wants to merge the authority records together. Towards this end, I added a database-level stored procedure that supports the merging of authority records, such that the linked bibliographic records will automatically point to the winning authority record.

• Authority browse interfaces

  Where having Mike Rylander participate in your review process pays off, part the third... Mike also implemented basic browse interfaces that presents a series of authority records in MARCXML format matching your requested authority type (author, title, subject, topic) and the matching substring at the /opac/extras/browse and /opac/extras/startwith URL entry points. While still raw at this point, these can provide the basis for classic authority browse interfaces for those who desperately desire them.

Remaining to-do items

Note that any estimates are based on how long I think it would take me to implement, based on my own familiarity with MARC and Evergreen and all things Perl and JavaScript and PostgreSQL, and provided with the granularity of no less than one day. Actual implementation times may vary, of course; if related work items are worked on consecutively, then it is likely to take less time to achieve than if the items are tackled sporadically.

• Add an authority in the flow

  When you're working in the MARC Editor and you find that there is no match for an entry that you really think should be controlled, IISH wants to make it easy for a cataloguer to add an authority record for that entry. We thought that there might be two options that we would want to expose - a direct "create an authority record from this field" option that takes no further input, and a "create an authority record from this field and open it in another MARC editor to let me tweak it" option. Estimate: 2 person days

• Highlight controlled fields

  This is really a two-part problem. First, for uncontrolled fields, we want to teach the Validate button to offer the kind of automatic matching that the script does and add the required 0 subfield. Second, we want to highlight fields that are explicitly controlled by authority records with a subfield differently from fields that simply match an authority record, but which are not controlled by it. Estimate: 1 person day

• Simplify authority record selection

  This two-part requirement would mask many of the fields that are currently offered as options when you right-click on an uncontrolled subfield to display matching authority records. For example, it is a little weird to offer a "See from" heading to a cataloguer; we're trying to avoid adding new records with those headings, right? Heh. Second, we want to introduce the ability to invoke the authority browse list in this interface so that the cataloguer can see a given set of headings in context and select the heading to apply from there. Estimate: 2 person days

• Delete authority record

  There is currently no cataloguer-friendly way to delete authority records. We need to expose a list of authority records (probably reusing that browse list again) and make it possible for cataloguers to delete an authority record. When that record is deleted, all bibliographic records that link to it need to have their links removed - and ideally, the cataloguer would be able to tell how many bibliographic records link to that authority before the delete takes place. Estimate: 1 person day

• Edit and merge authority records

  Although the database-level support now exists for merging authority records, we need to expose a means for cataloguers to select the authority records that they want to edit or merge. This could just be a slightly evolved version of the "Delete" interface. Estimate: 1 person day

• Expose authority records via SRU/Z39.50/crawlable interface

  One of the goals of the IISH is to be able to share their authority records with other institutions. One of the standard methods is SRU + Z39.50 server support; we should be able to build on the SRU/Z39.50 server support for bibliographic records in Evergreen to provide a basic solution for authority records. Interest has also been expressed in having a crawlable implementation that would give the linked data crowd something to play with. Estimate: 2 person days for an SRU/Z39.50 server, 1 person day for a very basic crawlable linked-data implementation

In summary - hurray for Mike Rylander for helping us out to such an extent, and many thanks, again, to IISH for giving me an opportunity to focus on Evergreen development for an extended period of time, and to Laurentian University for supporting my efforts. I hope that between Ole and myself that it will be possible to finish the rest of these work items prior to the Evergreen 2.0 release. It has been exhilarating to see far Evergreen's authority support has come in less than a month, and given a little more time I suspect that Evergreen's authority support will be the envy of other library systems.