Fetching item availability from Evergreen using the OpenSRF HTTP gateway

This is a preview of one part of my upcoming session at the OLA SuperConference, Evergreen Exposed: Hacking the open source library system. In the Conifer implementation of Evergreen, at least one of the partners plans to use a decoupled discovery layer rather than the Evergreen OPAC. So we needed to answer the typical question "How do I retrieve the availability of copies for a given work at my institution?" Note that this mini-tutorial is based entirely on OpenSRF 1.0 / Evergreen 1.4; OpenSRF 0.9 will generate different JSON output, and the URL for the OpenSRF gateway will be different.

Learning from the old masters: how the Evergreen OPAC does it

The Evergreen OPAC itself relies heavily on JavaScript to dynamically flesh out item details and retrieve item status, so it's actually pretty easy to work out how to do this without even delving too deeply into OpenSRF. First, let's use the Firebug Mozilla extension to follow network requests for a given "title details" page in the OPAC search results for the title: The new world guide to beer. Open up Firebug, enable network monitoring for the OPAC site, and watch the requests flood past for the title details page. We can see that there are a number of POST requests to http://dev.gapines.org/osrf-gateway-v1:

• POST request #1 parameters
  • method = open-ils.search.biblio.record.mods_slim.retrieve
  • service = open-ils.search
  • locale = en-US
  • param = 8526
  This is how we retrieve the title / author / ISBN and other bibliographic details of interest for display; as we're talking about a decoupled discovery layer, we won't need to worry about this piece of the puzzle.
• POST request #2 parameters
  • method = open-ils.search.config.copy_status.retrieve.all
  • service = open-ils.search
  • locale = en-US
  This is how we retrieve the list of all possible copy statuses that have been configured for this Evergreen system; here's the response (truncated for legibility):

  {
     "status" : 200,
     "payload" : [
        [
           {
              "__c" : "ccs",
              "__p" : [
                 null, null, null, "f", 3, "Lost", "f"
              ]
           },
           {
              "__c" : "ccs",
              "__p" : [
                 null, null, null, "t", 0, "Available", "t"
              ]
           },
           {
              "__c" : "ccs",
              "__p" : [
                 null, null, null, "t", 1, "Checked out", "t"
              ]
           },
           {
              "__c" : "ccs",
              "__p" : [
                 null, null, null, "f", 2, "Bindery", "t"
              ]
           }
        ]
     ]
  }
  

  We're getting a response in JavaScript Object Notation (JSON) format - the nice, compact, easy-to-read data interchange format that almost every programming language under the sun can interpret and generate. Yay!
• POST request #3 parameters
  • method = open-ils.search.biblio.copy_counts.summary.retrieve
  • service = open-ils.search
  • locale = en-US
  • param = 8526
  • param = 1
  • param = 0
  This is how we retrieve the call numbers, copies, and copy status for a given title. We pass in the the TCN input parameter ("8526"), the numeric ID of the organization being searched ("1" = "every branch"), and the depth of the organization ("0" = top of the hierarchy). The response for this request is:

  {
     "status" : 200,
     "payload" : [
        [
           [
              "127",
              "663.42 JACKSON, MICHAEL",
              {
                 "0" : 1
              }
           ],
           [
              "130",
              "663.42 JACKSON, MICHAEL",
              {
                 "0" : 1
              }
           ],
           [
              "125",
              "663.42 JACKSON, MICHAEL",
              {
                 "0" : 1
              }
           ],
           [
              "34",
              "R 641.23 JACKSON, MICHAEL",
              {
                 "0" : 1
              }
           ]
        ]
     ]
  }
  

Interpreting the HTTP requests and responses

Okay, so we've found a couple of requests that are pertinent to our goal. And you might be able to guess that the fifth element of the __p entry in the copy status response is the numeric identifier for the copy status, while the sixth element is the copy status name (which, as of OpenSRF 1.0 / Evergreen 1.4, if you pass a different locale value can return a translated value).

You might even be able to guess that the response from the copy_counts.summary request returns an array of responses consisting of the organization ID, the call number, and a hash of copy status and the respective counts for each copy status. And you would be guessing correctly. But why guess, when you can get an authoritative interpretation by looking up the class hint (the __c value in the copy_status response of "ccs") in Evergreen's intermediate definition language file /openils/conf/fm_IDL.xml:

<class id="ccs" controller="open-ils.cstore" 
  oils_obj:fieldmapper="config::copy_status" oils_persist:tablename="config.copy_status">
  <fields oils_persist:primary="id" oils_persist:sequence="config.copy_status_id_seq">
    <field name="isnew" oils_obj:array_position="0" oils_persist:virtual="true" />
    <field name="ischanged" oils_obj:array_position="1" oils_persist:virtual="true" />
    <field name="isdeleted" oils_obj:array_position="2" oils_persist:virtual="true" />
    <field name="holdable" oils_obj:array_position="3" 
      oils_persist:virtual="false" reporter:datatype="bool"/>
    <field name="id" oils_obj:array_position="4" 
      oils_persist:virtual="false" reporter:selector="name" reporter:datatype="id"/>
    <field name="name" oils_obj:array_position="5" 
      oils_persist:virtual="false"  reporter:datatype="text" oils_persist:i18n="true"/>
    <field name="opac_visible" oils_obj:array_position="6" 
      oils_persist:virtual="false" reporter:datatype="bool"/>
  </fields>

So now, by taking our first steps into Evergreen's object persistence model, we can determine authoritatively that the order of values in the __p array maps to "isnew", "ischanged", "isdeleted", "holdable", "id", "name", and "opac_visible". As for the response from the copy_counts.summary call, well, these are not Evergreen objects (they don't have a __c class hint) - but you can use the OpenSRF shell "srfsh" introspect command to view the documentation for the applicable method:

bash$ srfsh
srfsh# introspect open-ils.search
... (truncated for legibility) ...
Received Data: {
  "__c":"OpenILS_Application",
  "__p":{
    "api_level":1,
    "stream":0,
    "object_hint":"OpenILS_Application_Search_Biblio",
    "package":"OpenILS::Application::Search::Biblio",
    "remote":0,
    "api_name":"open-ils.search.biblio.copy_counts.summary.retrieve",
    "signature":{
      "params":[
        
      ],
      "desc":"returns an array of these: [
         org_id,
         callnumber_label,
         ,
         ,
        ...
      ]       where statusx is a copy status name.  the statuses are sorted by id.",
      "return":{
        "desc":null,
        "type":null,
        "class":null
      }      
    },
    "server_class":"open-ils.search",
    "notes":"\treturns an array of these:\n\t\t[
       org_id,
       callnumber_label,
       ,
       ,
      ...
    ]    \n\t\twhere statusx is a copy status name.  the statuses are sorted\n\t\tby id.\n",
    "method":"copy_count_summary",
    "argc":0
  }  

The introspect output is a bit rough - it's really intended for the doxygen API help interface - but it's good enough for our purposes. If we want to dig into what's going on under the covers, we can follow the package_name value "OpenILS::Application::Search::Biblio" to read the source code for the OpenILS::Application::Search::Biblio Perl module, and look up the method "copy_count_summary" as indicated by the "method" value in the introspect output. That reveals that the input arguments are "($self, $client, $rid, $org, $depth)". Every OpenSRF method automatically receives $self and $client as the first two arguments, so $rid (record ID), $org (organization unit ID), and $depth (organization unit depth) are the variables over which we have control.

Zeroing in on the copies for a particular library or library system

If we want to retrieve the visible copies for just a single organization unit in the entire Evergreen system, we just have to adjust the values of the organization unit ID and organization unit depth parameters accordingly. If we ask for the visible copies for just org_unit ID "125" at depth "2", we narrow down our results to a single hit:

{
   "status" : 200,
   "payload" : [
      [
         [
            "125",
            "663.42 JACKSON, MICHAEL",
            {
               "0" : 1
            }
         ]
      ]
   ]
}

So, with all of that ammunition at your disposal, you can write an Evergreen copy status lookup in any decoupled discovery layer that supports HTTP POST or GET requests. Which should be pretty much any discovery layer, right?

Frequently used tools and methods for Evergreen / OpenSRF hacking

Note, the first: you can easily play with different parameter values for the HTTP POST requests using the json_xs command to pretty print the JSON response:

curl -d service=open-ils.search   -d locale=en-US \
  -d method=open-ils.search.biblio.copy_counts.summary.retrieve \
  -d param=8526 -d param=1 -d param=0 \
  http://dev.gapines.org/osrf-gateway-v1 | json_xs -t json-pretty

Note, the second: the OpenSRF gateway also supports GET requests; simply concatenate the request parameters in a single URL like this