Why SOA Fails?

26 Apr 2011

Service Oriented Architecture is not a bad idea. Quite the contrary. What is bad about it is its way of implementation and an unbelievable hype.

Usual SOA implementation starts with purchase of an "infrastructure", which is usually some combination of ESB and process manager (usually BPEL-based). The next step is an attempt to connect existing services to ESB and "orchestrate" them using BPEL. While attempting to connect the very first real service it becomes quite clear that this is much harder that one would expect from product datasheets. Existing services - even though they are based on "web services" - are not quite ready for SOA. They need to be tweaked, schemas need to be modified to conform to the requirements of the ESB, new headers need to be added and so on. This usually result in wrapping existing services with yet another service layer to connect them to "SOA". But the ridiculous part just begins here. Now the services needs to be "orchestrated", e.g. the result of one service needs to be passed as an input to another service. But they have incompatible data formats. Now the deployment usually takes two different courses. First is an attempt to abuse BPEL to transform data formats. The result is unreadable, complex and unmaintainable "integration" mess expressed in BPEL. Second usual course is to create a special "conversion" service that just transforms data formats. This results in explosion of services: for each pair of cooperating services that is a new "adaption" service. Complex and unmaintainable integration mess again. Even if this task miraculously succeeds, there is another major problem ahead. The original services were neither independent nor idempotent, they usually have lot of (undocumented) side effects. Therefore they just cannot be freely reused and recombined into business processes. So the next necessary step is to re-engineer all the services (while maintaining backwards compatibility, of course). This is very costly and slow process, but until it is done the benefits of SOA are more than questionable.

What is the real benefit of such "SOA" deployment? I can't think of any. The mess is still there. It may seem that it is just in one place and therefore is easier to maintain. But that's just an illusion. There are service wrappers an transformation services that are usually not in one "place". And even if it is, it is so difficult to navigate that any real benefit of centralization is lost.

What are the downsides of such "SOA" deployment? First of all there is an investment to purchase SOA "infrastructure" and to set it up. Secondly, there is an investment to convert existing non-SOA integration mess to a SOA integration mess. Thirdly, the "infrastructure" is yet another moving part that can fail. It becomes a business-critical piece and needs additional (substantial) cost to set up high availability and resilience. It also takes a lot of energy and attention that could be invested in much better way, it disrupts usual business and builds a false of hope of better future. And I haven't even mentioned "advanced" problems such as synchronicity and consistency. Clearly and plainly, such SOA deployment is a waste of time. Waste of huge amount of time.

How to make it better? Just remember what SOA is: Service-Oriented Architecture. The focus should be on services not the infrastructure. The better way of SOA deployment is to start from there. Try to assess what services are already there, how well they are defined, whether they can be reused and whether they actually need to be reused at all. Every experienced developer knows that reusability does not come for free. It is actually quite costly quality. Therefore focus on services that need to be reused and re-engineer them to be reusable. This is best done as part of natural system upgrades and replacements. Try to gradually develop a common data model for your business so there will be less requirements to convert data from one service dialect to another. Some services may need more than one "upgrade" to get into a reusable form. This is not a fast process, so the number of services in SOA will grow very slowly. At the beginning of such SOA initiative the easiest way of "orchestrating" the services is to use any way that you are familiar with, e.g. a simple Java web application. Just make sure you can modify the orchestration code quite easily, as many adjustments will be needed as the things slowly settle down. Having an internal employee or a very flexible partner to do that job would be probably a good choice. As the number of services will be initially low and ability to reuse them will be quite limited, such "orchestration" code will be acceptably maintainable even if some things are hardcoded. This may be a good solution for first few years. Once the number of services grows then it is a good time to think about ESB and BPEL (or alternative technologies that most likely will be available in the future). At that time there will already be a considerable number of services therefore the cost of infrastructure could be easily justified. This service-oriented process to SOA deployment will be less expensive, less disruptive and will continually bring benefits proportional to investments.

Service-Oriented Architecture is nothing new. It is just an ordinary architecture. The architect works with systems instead of components. The architect works with services instead of component interfaces. Apart from that, it is still just an software architecture. Vast majority of principles and experiences applicable to intra-system architecture are reusable also for extra-system SOA. And that is probably the most important part of Service-Oriented Architecture.