The Iron Fist of Interoperability
“Any customer can have a car painted any colour that he wants so long as it is black.” — Henry Ford, My Life and Work (1922)
It’s not easy to get people to agree on anything. This is why joining a standards committee is generally considered as something akin to being sent to a Siberian work camp. Every party will have competing priorities, ideals and agendas and the resulting work generally comes out worse the wear after all of the stretching and mending from all sides.
At the same time, without some kind of agreement, there is no hope of adoption. It is really an unenviable position.
So with this in mind, I have great awe and respect for the Digital Library Federation ILS and Discovery System Task Force for wading into these waters. While the task force is not a standards body, per se, that they are trying to promote interoperability through the recommendation of a specification makes it seems like splitting hairs to not cast them in the same lot.
The recommendation draft is certainly welcome to library developers who have been craving something, anything, to help unlock the data from their proprietary systems (even though, as Marshall Breeding pointed out in the Library 2.0 Gang podcast on the subject, it’s about a year late). The current draft lays out desirable pseudocode-type methods and then gives options of existing, off the shelf standards and protocols that could be used to enable the functionality that is defined.
The problem here is that they generally give multiple options for achieving the goal of any given method. So this means that any ILS vendor can choose from a variety of protocols for implementing the spec and that a different vendor can choose alternate standards for the exact same functionality. The most striking example of this would be a GetAvailability service (basically, “what is the current status and location of a given item”) which the recommendation says could be implemented via NCIP, a REST interface, SRU or OpenURL.
The point of a standardized API isn’t to make it simple for the vendors to implement. The point is to make it simple for developers to implement. The more options that the developer has to account for, the more complicated the client library must be to access the API. This then gets to be rather chicken or egg. If there are no programming language specific client libraries to access the API, there will be a slower rate of adoption (especially if non-library programmers have to learn the basics of SRU, OpenURL or NCIP). If the spec is too complicated or allows for too much variation in protocols or output formats, it will be hard to find volunteers to build the domain-specific languages to help spread the proliferation of library data in other kinds of applications.
This is not an argument against using library standards. An OpenURL interface on a library catalog to resolve holdings data would be incredibly useful. Being able to harvest bibliographic records via OAI-PMH seems like a no-brainer.
However, it’s the combination of these where things begin to break down. Imagine this hypothetical scenario:
- SirsiDynix conforms to the recommendation by providing an OAI-PMH interface to their bibliographic and authority records and NCIP for item availability.
- VTLS conforms to the recommendation by providing an SRU interface that has the capability of exposing all bibliographic records with the option to filter by date and a proprietary RESTful interface for item availability.
- Talis conforms to the recommendation by providing a SPARQL endpoint for bibliographic records and a proprietary RESTful interface for item availability (with different syntax/responses than VTLS).
- Ex Libris conforms to the recommendation by providing OpenURL interfaces for bibliographic, authority and holdings records.
These are entirely fictitious, of course, and somewhat facetious. I wouldn’t expect Ex Libris to use OpenURL for everything, nor would Talis just say, “here’s SPARQL, have at it!”. After all, Platform stores already have an OAI-PMH interface. Replace the names with any other vendor’s name, it’s the point that they could do the above and all claim compliance.
Now imagine being a developer trying to write an application that uses data from the ILS. Maybe the main library has a Voyager catalog and the law library uses Innovative. Maybe the library is part of a consortium with libraries that have Aleph, Polaris and Unicorn. Now let’s say that the developer doesn’t actually work for the library, but for the central IT department and he or she is trying integrate these services into courseware or a city government portal. If all of these disparate systems use different protocols to access the same kinds of data, the odds lessen greatly that many of these catalogs will ever make it to the portal or VLE.
With the Jangle project, we’re trying to eliminate as much of this flexibility in implementation as possible. It is a difficult balance, certainly, to prescribe a consistent interface while also accounting for extensibility. But the point here is consistency. One of the reasons we chose the Atom Publishing Protocol to interact with Jangle is because we think it will provide the lower level functionality needed to manage the data in a simple and consistent way. On top of the AtomPub interface, simple OAI-PMH, OpenURL or NCIP servers can be built, using the AtomPub REST interface, to ensure that our library services can interact with existing library standards based applications. At the same time, developers can use common client libraries (such as those for Google’s GData or WordPress, for example) to have a congruous means of accessing different kinds of data and services. By only allowing Atom, we can focus on interacting with the data instead of requiring developers to focus on the protocols.
After all, sometimes to get from point A to point B, you just need a car. The color doesn’t matter all that much.













May 15th, 2008 at 2:04 pm
[…] Singer at Panlibus discusses a draft recommendation from the Digital Library Federation ILS and Discovery System Task Force and notes that while […]
July 10th, 2008 at 1:41 pm
There is nothing wrong with having multiple APIs for different purpose. There already are a couple of relevant APIs for libraries and for some purposes a simple API is missing. Instead of creating a totally new API for everything I think it is better to fix “good” APIs that exist, burry the “bad” APIs that are crap, and fill the gaps with new ones, that are not designed from a librarians point of view but from the viewpoint of simple usage - but maybe that’s also the idea behind Jangle?
July 10th, 2008 at 2:00 pm
[…] anstatt für jedes System die Anbindung neu anzupassen. Jangle wird von Ross Singer (Talis) vorangetrieben und steht anscheinend sowohl in Konkurrenz als auch in Kooperation mit der DLF working group on […]
July 10th, 2008 at 3:02 pm
Jakob, you’re right, of course. The downside of the ILS-DI recommendation, as it currently exists, is that there is no agreement on what protocol should be used for any given method.
So, take GetAvailability, for instance. You could use a RESTful interface, NCIP service or an OpenURL resolver. It could return a result in NCIP, MFHD, ISO 20775 or the ’simpleAvailability’ schema that they have created.
To follow the API, using the recommendations, you’d have: OAI-PMH, SRU, and some REST interface (for Harvest, Search and GetAvailability). Which means you need server interfaces and client support. This means the local developer also needs to be able to account for each of these protocols and know when it’s appropriate to use which. And keep up with the differences in syntax.
Instead, it seems like it would have made more sense to try figure out one technology that could work for everything. Leaving aside my own self-interest in AtomPub, SRU could have done this. Rob Sanderson has spoken in the past about having SRU eliminates the need for OAI-PMH (there’s an assumption you already have SRU in that argument, of course), since you could, effectively, ’search’ for the same results you’d get via OAI-PMH (through dc.date, probably).
The same could hold for GetAvailability.
Where things break down is where you get into ‘write’ methods (self-renewal, etc.), since, obviously SRU has no provision for that. But, at the core functionality that they are currently asking for, it seems like recommending one protocol (and then examples of how it would work) will keep us from trotting into an interoperability quagmire.
And, naturally, I think AtomPub offers some way out there…