Coffee|Code : Dan Scott - Evergreen

Making the Evergreen catalogue mobile-friendly via responsive CSS

dan@coffeecode.net (Dan Scott) — Sun, 21 Apr 2013 22:48:40 -0400

Back in November the Evergreen community was discussing the desire for a mobile catalogue, and expressed a strong opinion that the right way forward would be to teach the current catalogue to be mobile-friendly by applying principles of responsive design. In fact, I stated:

Almost all of this can be achieved via CSS, possibly with some changes to the underlying HTML (e.g. tables to divs or whatever so that "Place Hold" appears under the bib info instead of way over to the right).

I have this bad habit of talking more than doing. So when I saw the Beanstalk mobile catalogue resurrected again at the Evergreen 2013 lightning talks, it bugged me that I still hadn't put any effort into a proof of concept of what was possible with CSS media queries. Thus, today, on the last day of my holidays, I spent a few hours trying things out on our development server and came up with this rough branch to work towards making the exact same HTML that we serve up for desktops provide an experience similar to that of the Beanstalk generation of catalogues for mobile, just via CSS.

As you can see from the commits, I made one change to the HTML to define a viewport, and added one set of CSS rules wrapped in a media query; in essence:

...
<head>
...
<meta content="initial-scale=1.0,width=device-width" name="viewport">
<style>
@media only screen and (max-width: 600px) {
    #header {
        padding: 0px;
        margin: 0px;
    }
    .facet_sidebar {
        display: none;
    }
    ...
}
</style>
<head>
...

Results and trade-offs

Here are a few example URLs from our test server (which is slow, and might get wiped any day, so test them quickly if you have a mobile device around!):

Search results - sacrificed facets, per-item actions, and the language picker
Record details - sacrificed per-item actions, flattened the item table vertically

In general, I removed a lot of the frippery from the header, while trying to retain the most valuable pieces. However, some bits are broken: Another Search doesn't actually let you do another search because the search bar is totally hidden. Other bits haven't been touched (Advanced search is still overwhelming, and My Account, while functional, could be much prettier.

What I've done so far is oriented towards our 2.3-ish lightly customized Laurentian skin (we force full details in search results, for example) but the principles should be applicable to an out-of-the-box Evergreen catalogue. In working through some of the challenges, I've determined that I was pretty much on target back in November; with a few HTML tweaks that would improve the layout for desktops as well, we could keep the per-item actions and facets around, but just move them to a different location.

Less talk, more action

So who's with me? What we have to gain is a single set of HTML to support for TPAC, and a single set of CSS, all available from the same URL, rather than trying to maintain overlays and monkeying about with mobile-vs-desktop URLs and the like. Feel free to dig in and start pushing branches with improvements over my rough attempts and let's make this thing happen for Evergreen 2.5.

With thanks to Firefox...

I would be remiss if I did not mention the marvellous Responsive Design View introduced in Firefox 15, along with the Style Editor; together, these tools (built into Firefox) made my developing and testing work so much easier.

If you want to live on the cutting edge of Firefox, you want Aurora - go and get it

Structured data: making metadata matter for machines

dan@coffeecode.net (Dan Scott) — Fri, 12 Apr 2013 15:11:47 -0400

Update 2013-04-18: Now with video of the presentation, thanks to the awesome #egcon2013 volunteers!

I've been attending the Evergreen 2013 Conference in beautiful Vancouver. This morning, I was honoured to be able to give a presentation on some of the work I've been doing on implementing linked data via schema.org in Evergreen. I think I did a good job of explaining the potential value of linked data and arguing for improving Evergreen's schema.org publishing ninja skills.

My slides, with a reasonable number of useful speaker notes to provide context, are available in LibreOffice format.[1]

In addition, the amazing organizers of the conference also streamed most[2] of the talk and the recording will be available on the conference web site in a week or two.

Footnotes

I felt pretty dirty not using HTML5 + RDFa Lite to actually mark the whole thing up; there was some question close to the time of the conference as to whether anything but PPT or perhaps PDF would be an acceptable format... a concern that was subsequently removed, but a little too late to be worthwhile changing course.
The room was standing-room only (well, sitting-on-the-floor-room only), and one of the organizers accidentally sat on and unplugged the Ethernet cable, so something like ten minutes were lost. Heh!

Introducing SQL to Evergreen administrators, round two

dan@coffeecode.net (Dan Scott) — Fri, 15 Feb 2013 21:32:08 -0500

Three years ago I was asked to create and deliver a two-day course introducing SQL to Evergreen users. Things went well and I was able to share the resulting materials with the Evergreen and PostgreSQL community. Perhaps one of my happiest moments at the Evergreen conference last year was when one of the participants in that course, told me that many of his fellow participants were still successfully writing SQL queries and getting work done. Huzzah!

Time goes by and another group, OHIONET, was running into difficulties getting started with PostgreSQL and Evergreen. They asked me if I would be willing to give the same sort of training I had given a few years back. "Sure", I said, thinking it would be a great opportunity to polish the materials and add some updates to cover new features in PostgreSQL and Evergreen. We also opted to skip the travel and do an entirely virtual training session via Google Hangouts, which worked out rather nicely (but that's a different story).

As it turned out, I probably ended up putting about four days worth of effort (crammed into lots of late nights, weekends, and vacation days) into overhauling the instruction materials. But the results were worth it, in my opinion; I'm rather proud of the content, and while I believe it stands up on its own, the guidance that I was able to provide during the live instruction sessions was well-received by the participants.

Thus, I am pleased to be able to offer to the broader community the latest version of the Introduction to SQL for Evergreen Administrators, under a Creative Commons Attribution-ShareAlike 3.0 (Unported) license.

Reference documentation--30 pages introducing SQL with examples drawn from the Evergreen schema: (HTML) (PDF) (ePub) (AsciiDoc)
Presentation: (LibreOffice Impress) (PDF)
Solutions to exercises: (Day 1) (Day 2)

So, a huge thanks to OHIONET for giving me the impetus to overhaul this material, and for giving me a chance to introduce them to the wonders of SQL with PostgreSQL, and to the inner workings of the Evergreen schema. It was a blast! And thanks for agreeing to let me share these materials with the broader community.

Leaving SELinux in enforcing mode with Evergreen on Fedora 17

dan@coffeecode.net (Dan Scott) — Sun, 02 Sep 2012 01:26:59 -0400

Ever since I switched over to Fedora a few years back (hi Fedora 13!), I've been guilty of a dirty secret: to run Evergreen, I've had to run setenforce 0 to disable the most excellent SELinux security policies before I could start up the Apache web server to serve up the Evergreen goodness. This worked for development purposes, but tonight something snapped and I decided that it was no longer acceptable to throw away a great layer of operating system security simply for the sake of hacking on Evergreen. So... I stepped into the world of what had formerly seemed to be inscrutable SELinux concepts, and came out with something that seems to work (at least for my fairly limited purposes thus far of searching the TPAC catalogue).

This was a pretty iterative process that involved trying to start the httpd.service, then checking /var/log/messages and /var/log/audit/audit.log for clues as to why httpd.service was either not starting, or (once I passed that hurdle) was simply returning internal server errors.

First, due to my recent experience with running a web.py script under Fedora, I had learned that the httpd SELinux policy had a number of booleans for enforcing or allowing particular behaviours, so I immediately ran the following command to enable httpd to connect to the network:

setsebool httpd_can_network_connect on

I then needed to change the labels on many of the OpenSRF and Evergreen files that were installed and which Fedora gave a default type of unconfined_t, which is understandably restrictive:

# Mark web content as, well, web content
chcon -R --type=httpd_sys_content_t /openils/lib/javascript
chcon -R --type=httpd_sys_content_t /openils/var/web
chcon -R --type=httpd_sys_content_t /openils/var/templates*
chcon -R --type=httpd_sys_content_t /openils/var/data
chcon -R --type=httpd_sys_content_t /openils/var/xsl
chcon --type=httpd_sys_content_t /openils/conf/opensrf_core.xml
chcon --type=httpd_sys_content_t /openils/conf/fm_IDL.xml 

# Mark the custom Apache modules 
chcon --user=system_u --type=httpd_modules_t /usr/lib64/httpd/modules/mod_xmlent.so 
chcon --user=system_u --type=httpd_modules_t /usr/lib64/httpd/modules/osrf_*

# Mark the dynamic libraries we need to load
# "-h" changes the context of symlinks as well as files
chcon -h --type=lib_t /openils/lib/*

# Mark executable scripts
chcon -t httpd_sys_script_exec_t /openils/bin/openurl_map.pl 
chcon -t httpd_sys_script_exec_t /openils/bin/offline-blocked-list.pl 

# Might not have been necessary
chcon -R --user=system_u /usr/local/share/perl5/
chcon --user=system_u /etc/httpd/conf.d/eg.conf 
chcon --user=system_u /etc/httpd/startup.pl 
chcon --user=system_u /etc/httpd/eg_vhost.conf 
chcon -R --user=system_u /etc/httpd/ssl/

Note: I'm aware that simply running chcon won't survive a relabelling of the files. We really need to turn this into a policy, or alternately use semanage to make the changes permanent...

Next, I opted to finally start running Apache as the stock apache:apache user/group rather than as the opensrf user. This turned out to require only a few steps:

Change the User setting in /etc/httpd/conf/httpd.conf back to apache, reverting the change we made following the default install documentation.
To avoid errors writing to the /openils/var/log directory, cut over to using syslog - which, on Fedora, is provided by rsyslogd.
1. Copy the very handy Open-ILS/examples/evergreen-rsyslog.conf file that Bill Erickson created into /etc/rsyslog.d/
2. Restart the rsyslogd service with systemctl restart rsyslog.service.
3. Edit /etc/httpd/eg_vhost.conf and /openils/conf/opensrf_core.xml to use syslog instead of writing to log files.
4. Restart the OpenSRF services.
One more restart of the httpd service and I was in business.

So this is a start. I think this has broader implications than for just Fedora; we should stop using the opensrf user to run the Apache service in the default configuration on all distributions (we've discussed this several times in the past, but never really done anything about it).

I hope to update the README accordingly, and I also hope to take the SELinux work a step further to provide a modified policy so that Fedora and Red Hat (and derivative) distributions can offer a more secure environment for running Evergreen.

Oh, and some handy resources:

Enabling mod_wsgi with LDAP access under Fedora 17

dan@coffeecode.net (Dan Scott) — Wed, 11 Jul 2012 10:23:47 -0400

Continuing my path of new problem to solve = opportunity to try something new, I opted to give web.py a shot as a Web front-end for an existing script I had put together to provision users in our Evergreen library system based on their LDAP entry. The goal was to provide access to the functionality of the script, without having me as a single point of failure... something I have intended to put in place for a long time, but which jumped up in priority once I went on vacation and received a few requests (surprise, surprise).

Creating a web.py front end was easy enough. It was a bit more challenging to put it into production, because my production box for this task runs Fedora 17, and that means SELinux. In the past, my knee-jerk reaction during development would be to setenforce 0 and be done with it, but exposing it to more than just me at the terminal means taking some care. So, fortunately, it was pretty easy to sort out (thanks largely to the assistance gleaned from this Packtpub.com article, minus the compiling mod_wsgi from source bits).

The pertinent bits for my case were:

Install mod_wsgi and web.py: yum install mod_wsgi python-webpy
Configure /etc/httpd/conf/httpd.conf to include the appropriate WSGIScriptAlias line
Fix the SELinux label on the WSGI files: chcon -R httpd_user_content_t patron-load
Allow Apache to connect to an LDAP server: setsebool -P httpd_can_connect_ldap=1

And poof: my server still has the protection of SELinux, and my desired functionality works. Yay!