Open Library Environment Project – is SOA right?
OLE – The Open Library Environment Project has been around for about a year now, and I am guilty of not monitoring as closely as I would have liked to. So the opportunity to listen to their recent webcast seemed a great way to get up to speed again.
Following the instructions on the OLE Project site to replay the webcast, led me to one of the most unusual webcast playback experience I’ve had for a while. To see the slides you have to click through to a service run by Adobe Acrobat, which provides a good representation of the webcast environment, complete with chat traffic in real time. The problem then is that you have to use the telephone system to get the audio. This is not a cheap exercise for those of us having to dial international – at least with Skype Out you can keep the costs down a bit. Synchronising the listening with the viewing is then a bit of a challenge, especially if you have to pause and restart.
Anyway enough about the experience – what about the content?
What is clear is that the Mellon Funded Project has got a great deal of attention and significant partners from academic and national libraries. They also have a challenging and worthy goal, which they are taking significant early steps towards:
“By the end of our project, we will have a design for a next-generation library system using Service Oriented Architecture. We also will have built a community of interest that can be tapped to help build the OLE framework.”
The webcast inevitably, especially in the QA section, swung between the low-level detail, the strategic approach, and things like privacy which are more the policy concern of potential implementing libraries than the project itself.
Having listened to it, it is clear that they are working on an assumption that implementing libraries would have to throw their current investment in commercial or open source systems away and build all this from scratch – this being based on experience with the current generation of systems not being capable of integrating easily, or not dealing with electronic resources. That is a heck of a large chunk to bite off, even if you pull in things like circulation and cataloguing from other projects.
Experience also calls me to strongly question the emphasis on Service Oriented Architecture (SOA), that is if SOA is being used as generally understood as against a generic term for systems being connected via web-style calls.
A bit of background on that ‘experience’ I mention – There are [in general terms] two approaches to Web Services – tightly coupled SOA, and loosely coupled REST based services. The difference being that a SOA developer/integrator trying to embed the service in to their application needs access to web service descriptions and other enterprise integration tools. Whereas in the RESTful world, integration calls can often be tested using a web browser, and integrators/developers need no more development tools than they currently use.
Both SOA & REST have their benefits and their, sometimes religious, proponents. With our first use of SOAP (the underlying messaging protocol for SOA) back in the late 1990’s I have been using both of these competing approaches for some time. Talis over that time has developed and rolled out and established a significant user community for a product known as Talis Keystone. Keystone is a web service integration component designed to enable external enterprise services (Student Registries, Finance Systems, Student Portals, e-payment services, CRM systems, etc.) to easily and reliably integrate library system data and functionality into their workflow.
Keystone is now in use in many Talis customer libraries, and with some from libraries with a system from another vendor, in the UK. Successful integrations have been completed with products such as: Aggresso, Civica, Oracle, and SAP finance systems; Microsoft Sharepoint, uPortal, Moodle, and Blackboard learning and portal environments; and WorldPay e-payment services. Integration with systems from other suppliers are already in the pipeline.
From day one, Talis Keystone has had the capability to support both SOA and RESTful integration. It maybe useful for projects such as OLE to reflect on the experience in rolling out these integrations, and the take-up of the REST and SOA options. The vast majority of these integrations have taken the RESTful approach, with only one or two going for SOA. There are many reasons for this, but they all fall under the heading of there being a much lower barrier to implementing REST than SOAP. Pragmatically I am of the opinion that lack of SOA capability would not have prevented any of these integrations taking place, whereas if SOA was the the only choice many would not have been undertaken at all.
I/We would be more than happy to share some of these experiences in implementing and rolling out a product that addresses many of the concerns of the OLE Project.
October 3rd, 2008 at 6:45 pm
I’m never sure what people mean by “SOA.” Sometimes it seems to be used to merely indicate the _philosophy_ — individual software components carrying out particular functions, that can be combined in a ‘plug and play’ manner.
That much, I’m sure is what we need in library software.
Other times it seems to be used to mean a very particular architectural approach to carrying this out. It’s a rather complicated approach that I haven’t really had time to understand fully, but when I’ve taken a look… does this particular “SOA” approach come out of the Java community? Because, like much Java stuff I look at, it seems to be overly engineered, overly complicated, overly hard to implement, overly flexible in it’s configuration without sane defaults forcing you to take care of lots of details that should just be assumed. So I share your lack of enthusiasm about this kind of particular “SOA”.
But I’m never sure if that’s really what people mean by when they “SOA”.
October 3rd, 2008 at 7:40 pm
As you say Jon ‘I’m never sure if that’s really what people mean by when they “SOA”.’
I get the impression that the OLE Project is at least leaning in the ‘what came out of the Java community’ direction. I hope that my impression is wrong and they are at least looking at both REST and SOA.
October 3rd, 2008 at 9:34 pm
In our initial scoping activities, we have intentionally embraced a principles-based understanding of services orientation that does not restrict itself to SOAP, REST, or any particular implementation technology. Since we are still taking design, a service is a reflection of an atomistic component of how a system operates; defining it does not dictate the particular technology used to implement it.
My impression is that OLE Project embraces SOA as a design philosophy. Architects and developers are free to select whatever technology they like to construct the services (including REST).
I’ve tried to state this in various ways. Is there a better way? How would you state this to avoid confusion?
October 3rd, 2008 at 10:50 pm
That’s good to hear.
As is clear from this conversation, and my misunderstanding, this is not easy to clarify.
I tend to follow the hierarchy of description that Wikipedia does with Web Services as the umbrella concept which then includes SOA (being the tightly coupled WDSL, UDDI, etc. option) and REST (the loosely coupled approach which our experience has shown to be most often adopted).
My recommendation would therefore to be to state simply and explicitly that OLE embraces both forms of Web Services. As designs evolve, any examples should be provided in at least RESTful form (as the barrier to understanding, for those new to Web services, tends to be lower than SOA).
October 4th, 2008 at 12:04 am
Just to confuse things more, while Evergreen uses and supports web services, its SOA is not web-based, but comes from an OpenSRF based implementation that does scalable message passing via XMPP, the same protocol that Jabber uses. Asking OLE to embrace web services, while good, is still pushing specific technology. The underlying SOA can be something different, whether it be OpenSRF, something SOAP or REST based, or the Java-based backbone I hear other Mellon-funded community source projects are working on. In line with this, the Wikipedia article for SOA lists web services as an approach, but states that service oriented architecture is not tied to a specific technology. In my mind, the main thing about SOA is that it’s modularity made scalable (in many different directions).
– Jason
October 7th, 2008 at 11:51 am
I think we need to be a bit clearer on what SOA is in relation to REST, SOAP and the rest of the WS-* stack.
SOA is an architectural approach which embraces a number of industry best practices to solve high-level architectural problems. A lot is written about SOA but for me it embraces the key principles of componentised, reusable, loosely coupled and incrementally adoptable “business” services.
None of these are new ideas but an umbrella term like SOA is beneficial as it gives us a reference point and something to work towards. SOA promises ubiquitous and seamless real-time interoperability between disparate systems in an organisation, delivering massive efficiency saving and end-user benefits.
However, the adoption of a full SOA stack is enormously challenging for an organisation. I’ve found that the real challenge in implementing an SOA strategy in countless organisations is not a technical challenge but a cultural one. Why are systems in disparate and disconnected silos? Departments have emerged, grown, changed and implemented their own systems and business processes and systems independently of each other over many years and it will take more than some technical glue to get them to marry up.
SOA is not the silver bullet which will address interdepartmental process and social integration – although it can certainly do the teccy stuff and implementation projects can begin to pragmatically address the business operation issues.
I made a point of highlighting the word ‘business’ earlier, in “business services”. There is much confusion in the word “service” in the context of SOA and Web Services. For me the word service means different things in each of these areas:
Service Oriented Architecture (SOA) – Service here refers to business services. Such as circulation, or cataloguing or stock management in the library world. This is not a technical service moreover a core operational activity.
Web Services - Services here refers to technical services: these are a standards based, machine interfaces to glue systems together. Such as a web service to create a borrower/patron on an LMS/ILS. Web Services come in different flavours, such as XML-RPC, SOAP (WS-*), REST and others…
It is an unfortunate coincidence that SOA and SOAP are such similar acronyms as they mean different things. SOAP – Simple Object Access Protocol is not SOA, although SOAP can certainly make SOA a reality – but so could REST.
To this end you can implement SOA without Web Services; it is perfectly feasible to have a componentised, reusable, loosely coupled business services delivered over SIP2 for example. However the emergence of Web Services has enabled the industry to take interoperability outside of domain specific protocols to embrace a more widely adopted standards based (XML/HTTP) approach.
To echo Richard’s comments: experience has taught us that REST web services (not BIG Web Services such as SOAP) provide a pragmatic and sensible way to address the technical issues and allow us to concentrate on the real challenge of integration and SOA adoption - that of tackling the reasons the systems are in silos in the first place.
As the Technical Lead for Talis Keystone I am too excited by this initiative and second Richard’s offer to share our experiences in this fascinating area…
-Andy
Andy Latham
Senior Solutions Consultant
Talis Information Ltd
October 10th, 2008 at 4:29 pm
Andy, your point hits the nail on the head and is one of my primary concerns with the OLE project, as it stands. Merely stating that it will be built using SOA-principles does not really say anything. They might as well say they will use “technology” to build a new ILMS framework. I would think that any plan to reengineer a technology from scratch would take a “small pieces loosely joined” approach; that point goes almost without saying. But how you’re actually going to work around existing cultures, business practices and workflows and turn them into individual, independent operating systems is the real hurdle.
Also, although I agree that SOA is technically a superset of ROA, I do think there tends to be a bit of a philosophical difference between dividing the world up into “services” as opposed to dividing the world up into “resources”. They aren’t mutually exclusive (you can have both, for instance, or, as you have pointed out in the past, your ‘resource’ can actually represent a ’service’ of sorts), but they do tend to have their differences.