With the arrival of microservices, the lines between systems integration and application are increasingly becoming blurred. Previously, a traditional enterprise application as a single big ‘monolith’ would perhaps make some network calls to an integration layer to connect with other external single monolithic applications.
These days microservices advocate intelligent APIs and that microservices should just be big enough “to fit in your head”. This leads to many more 'integration' points in a traditional business 'application' between microservices micro-application. A lot more than what we encountered in additional enterprise systems integration.
In this blog, I am going to investigate if traditional ESB type approaches to integration still apply when working with microservices and are Microservices patterns applicable to traditional systems integration?
What are the Boundaries of the Application?
I have always perceived microservices for intra-application integration as opposed to inter-application integration. However, now we are finding that it is difficult to separate intra-application from inter-application especially in Greenfields Microservices environments.
For example, a single website might have been thought of as a single application and would have been implemented as a single archive file. Since the website’ is now broken down into discrete microservices. the question that we might ask ourselves becomes:
Is each microservice an “application” or do all the microservices together form the application? The more important (rhetorical) question is: Does it even matter?
Embedded Integration Best Practice Patterns
Instead of palming off best practice Service-Oriented Architecture (SOA) Integration Patterns to a central integration layer, a Microservices approach ensures these best practices are embedded into each microservice. For example, (the principles of Service Virtualisation, Service Autonomy, Service Discovery) etc. In a Microservices approach, ESB patterns are not handled by a central ESB, but distributed as best practice patterns in different feature teams, with enforcement through smart, visible APIs.
Integration between APIs is a very important aspect of the microservices approach, It is not seen as a different responsibility or someone else’s problem. It is part of the core way of the Microservices thinking, API first.
Microservices is forcing applications to be built as distributed systems instead of a single big monolith. The microservices approach of distributed processing has been a hallmark of integration layers following a SOA approach: each service has it’s own isolated responsibility and can easily be changed without impacting any of the other services. As a sidenote, it is much simpler for SOA environments to become fully microservices compliant than monolithic applications.
What about Mixed Legacy and Greenfield Environments?
The traditional ESB system integration approaches are useful when encountering a ‘legacy’ environment where there are many large 'monolithic' enterprise applications. Using Microservices patterns for integration in a ‘greenfields’ environment seems to be most fitting. However, what about an environment where both are present: many large monoliths and many ‘new’ microservices?
One approach is API domain encapsulation. Microservice emulation of the monolith via multiple ‘smart endpoints’ allows other microservices to interact with it in the same way they would with any other microservice, via APIs & lightweight messaging. These ‘system-layer’ Microservices can facilitate the traditional role of integration with these legacy enterprise monoliths by exposing encapsulation behind APIs and facilitating the microservice gap.
Some of the challenges with this approach are:
- Eventual consistency compensation in the enterprise monolith.
- Transforming bulky and complicated ‘canonical’ or proprietary web service (WSDLs) to discrete Domain Driven Design modelled APIs.
- Choreography between Microservices and the enterprise monolith.
- Fault tolerance or fall back in the enterprise monolith.
- Receiving “events” from the monolith via APIs and lightweight messaging.
With the growth of Microservices, the lines between integration and application have become harder to identify. A Microservice approach isn’t a silver bullet. It might not be appropriate for all use cases. It is however forcing a rethink of traditional systems integration.
Is it necessary to have an ESB in a Microservices environment? Probably not. Is Microservices a good fit for an integration layer between traditional legacy enterprise monoliths? Possibly, it depends on the situation.“Enterprise Service Bus” is fundamentally a set of patterns normally implemented by a software product. So why can’t it be implemented using a Microservices approach? If good SOA principles and patterns are used, it would be far easier to implement a Microservices based approach.
In summary, Microservices not only forces further thinking on systems integration, but it also requires a re-think of data-integration approaches, and master data management. We can no longer ignore microservices across these paradigms, it does demand our attention.