Sunday, April 22. 2007Evergreen and the business case for choosing an open source ILSDue to a sad event, Art Rhyno asked me to be his co-presenter at the OLITA Digital Odyssey 2007. Our broad subject was Evergreen, more specifically introducing the Evergreen ILS to an audience that was aware of Evergreen's existence but wanted to know more about it from both a technical and a business perspective. I had two days' notice to prepare for the presentation, so I split my time between polishing the VMWare image of Evergreen and creating the slides for my presentation (PDF). Art gave a general introduction to open source development, told the story of how Evergreen came about, and described its architecture and the capabilities currently demonstrated on the in-production system at PINES. Perhaps of most interest to the audience, Art talked a bit about the direction that he's taking Woodchip, the serials and acquisitions module based on Apache OFBiz that the University of Windsor has agreed to develop for Evergreen. No pressure, Art Then the presentation was handed off to me. I started by asking for demographic information from the audience; to no surprise, about half of the audience of approximately 60 ran Horizon systems. Many of the attendees in the audience paid more than $20,000 annually for support and licensing costs. Most of the sites had the equivalent of one full-time position devoted to the care and feeding of their current library system. The goals of my presentation were to:
My self-assessment? I did not want to come across as an open source zealot; rather, I wanted to point out where our current relationships with our vendors are failing us and how open source can fill in some of those gaps. Unfortunately, I feel that I probably veered a little too much towards the rant side of the continuum a couple of times -- my passion for this subject came through, no doubt, but it was perhaps a little too strong. I knew my presentation was text-heavy, but I didn't beat myself up too much because a good visual presentation needs more than just a couple of days to come together and I didn't have a variation of this already in the can somewhere... this was brand new content. I was pleased that I came up with and shared the visual image of migration ninjas. As the closed vendors' licensing terms might prevent us from openly sharing migration kits or migration how-tos, the I wasn't at all happy with my live demo. First, I failed to arrange with the conference hosts to obtain an Internet connection, so the cover art in the catalog and the Z39.50 copy cataloging in the staff client facets of the demo were a bust. Second, while I knew it would be an exploratory live demo, given that I had just achieved a full working install a few days prior to the session, it's not very impressive for an audience to watch a presenter fumbling around the command line in response to a question about the API. Third, I failed to show off some really cool features of Evergreen such as the shelf-browser (although without cover art it wouldn't have been nearly as impressive). I tried firing up the reports Web interface and failed. So, now that I have a working install, I'll be able to prepare a much better live demo in the future - I just hope that our audience didn't take away a bad impression from our session on Friday. Questions from the audienceWe had some good questions from the audience; here's what I can remember. Please add more to the comments on this post, if you have them! Why is there so much interest in Evergreen and why aren't we hearing much about Koha?Dan said something about how his first investigation of Koha revealed evidence of classic MySQL dependencies and assumptions in the codebase that, as a former product planner for IBM DB2 relational database, made him cringe. Evergreen, in comparison, is built on PostgreSQL which was reassuring. I failed to note at the time that Evergreen has been developed so that it can support other databases, although some work would be required to convert to the SQL dialect and full-text search required by the target database. Art mentioned that while Koha had been quite popular internationally for the past number of years, it had not been as popular in North America. Part of that reason may have been a severe scalability problem that kicked in somewhere around 450,000 records. Dan suggested that problem could be traced directly to MySQL 3 / 4, but that it might have been alleviated in MySQL 5 (which Koha does not yet support). Art noted that Koha ZOOM, using indexdata.dk's Zebra indexing engine, overcame that performance problem but some extra care was required to commit updates to the index. What about the dangers of someone forking the code?In my opinion, we didn't really answer this question well. Art didn't think that a fork was likely as Evergreen had been built with the best-of-breed components and plenty of input from the PINES library staff and community. What I should have added was that the ability to create a fork of a project is actually a wonderful feature of open source - it enables communities to route around projects that become overly bureaucratic, or closed to new developers, or not interested in input or exploring new directions. You (Dan) talked a lot about the benefits of a system built on standards. Can you show us what the Web templating language looks like?I fumbled this one badly. I quickly brought up footer.xml, but that doesn't contain any dynamic content so it was a bad example. I then suffered from presentation brain and couldn't remember the word SummaryI believe that a solid business case needs to be developed on a library-by-library basis or on a consortial basis for migrations to Evergreen. I think that my presentation provided some useful input to those business cases, but in and of itself is not enough. Certainly, as our own library considers its options in the coming years, we're going to have to have a much more solid set of criteria before we can make any decision. I encourage you to take what you can from the presentation and improve, polish, and contribute your own analysis back to the Evergreen community so that you can help other libraries make an informed decision. |
QuicksearchAbout MeI'm Dan Scott: barista, library geek, and free-as-in-freedom software developer.
I hack on projects such as the Evergreen
open-source ILS project and PEAR's File_MARC package .
By day I'm the Systems Librarian for Laurentian University. You can reach me by email at dan@coffeecode.net. Identi.ca microblogging
LicenseCategories |

All in all, another good question for an Evergreen FAQ, methinks.
I thought the demo was bang-on, it showed that Evergreen was real and could run in a minimal environment. And your talk was great, an amazing amount of ground to cover with half a session and two days' notice to work with. I almost think the Horizon folks really need their own forum to discuss their options, I understand they have a users group meeting coming up so maybe there will be an Evergreen session there. I also thought the points you made about how open your previous employer, IBM, was in regard to things like listservs were provocative and need to be brought forward more often.
Thanks again for stepping in at the last minute, OLITA events are always fun but it's a lot to ask someone to pull out all the stops to get to an event that requires driving from Sudbury to Toronto on such short notice.
Perhaps, as a former product planner for IBM DB2, Dan would be willing to explain how Koha is more dependent on MySQL than Evergreen is on Postgres. Both projects rely on modules such as DBI that are database independent. I personally know of Koha installations on both Postgres and Oracle, though they definitely did require some minor customizations.
It would also be good to understand what 'assumptions' Dan has discovered in Koha that make him cringe, I'm sure the Koha community would appreciate those insights.
As far as Art's comments, that's certainly an accurate historical perspective on the development of Koha over the past 8 years or so. Just to clarify, the scalability limitations have been eliminated with Zebra, MySQL 5.0 is now supported (as of Koha 2.2.8), and the 'extra care' for index updates has been fully resolved, thanks to the excellent and speedy updates that Index Data released for Zebra.
Great... stepping into a classic religious war here (and I should note that my summary was, well, a summary). When I was at IBM, I was always interested in the progress of MySQL and PostgreSQL, and found it interesting that PostgreSQL measured up to the likes of DB2 and Oracle in terms of standards compliance and technology long before MySQL could be considered a valid threat. We're talking about the days when transactions were something the MySQL AB people said should be handled by applications... old days, obviously, and they've covered a lot of ground since then.
At any rate, I spent a good deal of my spare time playing with applications and porting them to support DB2. I put a lot of work in on Serendipity, for example (s9y.org) and helped a friend with Gallery2. I found a purely unscientific (but still significant to me) correlation between the difficulty of porting apps written first and foremost for MySQL and apps written to support PostgreSQL. PostgreSQL apps seemed to reflect a knowledge of relational databases and standards, while MySQL apps tended to be quick and dirty and hard to port. Did I say dirty? I meant ugly.
So, that explains my old bias towards apps written for PostgreSQL. As for Koha in particular, I think I first looked at it back in 2004 (a few years before I rejoined the library world). That's a long way from 2.2.8. The problem back then was the use of SQL92 keywords as field names, table names, etc., that would severely hinder porting attempts to any other database -- I think I remember reading this post to the mailing list, actually: http://osdir.com/ml/misc.koha/2004-07/msg00027.html
But looking at the koha.mysql file in CVS HEAD, I see that tables are defined with TYPE=MyISAM, and that (as an engine that does not respect transactions) immediately sets off my spidey senses for any application that relies on transactions. Like, say, circulating books, or recording bill payments. Ah yes, and there are oodles of column definitions for integer types that have DEFAULT '0' -- that's sloppy DML that MySQL and PostgreSQL accept, but there's no need for it -- get rid of the quotes around integer values. There are date columns that default to '0000-00-00' (a completely nonsensical date) rather than NULL -- why?
So, that's the sort of thing that, in the past, has turned me off MySQL applications. It looks like schema design by copy and paste from other MySQL applications, rather than from understanding SQL and designing for portability. Yes, it works, but that doesn't mean that I have to like it.
In a quick comparison, peeking at some Evergreen DML shows that integer column defaults are set with non-quoted integer values (yay), default values appear to make sense, and of course the PostgreSQL engine respects transactions. Evergreen also makes ample use of views (a good thing) and user-defined functions. The latter will introduce some additional work in porting, but that's a reasonable trade-off for making the database engine work more efficiently.
So yes, work would be required to port either Evergreen or Koha to a different database. Now that Koha 2.2.8 is out with MySQL 5.0 support, the really annoying effort of having to rename columns or tables and every line of code that accesses them wouldn't be necessary. But as I said in my summary, my reaction was based on my first look at Koha back in 2004. Now that you've enticed me to take another look, I can see that some things have improved in the database layer but many things remain the same.
Just a FYI in HEAD the file you want to look at is in installer/ called kohastructure.sql.
Which uses innodb, and default dates as NULL.
Chris
From a quick glance, you're right, the schema appears to be headed in the right direction. Removing the quotes from the integer values would be nice, and SET SQL_MODE='ANSI_QUOTES' so that you can use double-quotes for SQL name delimiters would be even nicer, but that's just my pro-standards bias showing through.
Thanks for the pointer, Chris. I look forward to giving Koha a whirl some time this summer and will report back with my findings!