With the arrival of microservices, the lines between systems integration and applications are becoming increasingly blurred. Previously, a traditional enterprise application as a single big monolith would perhaps make some network calls to an integration layer to connect with other external single monolithic applications. These days a microservices approach advocates having intelligent APIs and building microservices just be big enough “to fit in your head”. This leads to many more integration points in a traditional business application between microservices, certainly more than what we encountered in a traditional enterprise systems integration.
In this blog, I am going to investigate if traditional Enterprise Service Bus (ESB) type approaches to integration still apply when working with microservices and whether microservices patterns applicable to traditional systems integration?
What are the Boundaries of the Application?
I have perceived microservices as applicable for intra-application integration rather than inter-application integration. However, now we are finding that it is difficult to separate intra-application from inter-application, especially in greenfield microservices environments. For example, a single website might have been thought of as a single application and would have been implemented as a single archive file. Since the website is now broken down into discrete microservices should we ask ourselves the following questions:
*Is each microservice an application or do all the microservices together form the application? Where would an integration layer be applied? Is it a good fit between microservices and intra-application? And more importantly, does it even matter?
Embedded Integration Best Practice Patterns
Instead of palming off best practice Service-Oriented Architecture (SOA) Integration Patterns to a central integration layer, a microservices approach ensures these best practices are embedded into each microservice. For example, using a microservices approach, ESB patterns are not handled by a central ESB, but distributed as best practice patterns in different feature teams, with enforcement through smart, visible APIs. This approach leverages some of the principles of Service Virtualisation, Service Autonomy and Service Discovery.
Microservices are forcing applications to be built as distributed systems instead of a single big monolith. The microservices approach of distributed processing has been a hallmark of integration layers using a SOA approach. Each service has its own isolated responsibility and can easily be changed without impacting any other services. It is therefore much simpler for SOA environments to become fully microservices compliant than monolithic applications. This leads us to ask, if a mature microservices environment embeds integration best practices is a separate integration required?
What about Mixed Legacy and Greenfield Environments?
The traditional ESB system integration approaches are useful when encountering a legacy environment where there are many large monolithic enterprise applications. Using microservices patterns for integration in a greenfield environment seems to be most fitting. However, what about an environment where both are present: many large monoliths and many new microservices?
One approach is API domain encapsulation. microservice emulation of the monolith via multiple ‘smart endpoints’ allows other microservices to interact with it in the same way they would with any other microservice, via APIs & lightweight messaging. These system-layer microservices can facilitate the traditional role of integration with these legacy enterprise monoliths by exposing encapsulation behind APIs and facilitating the microservice gap.
Some of the challenges with this approach are:
- Eventual consistency compensation in the enterprise monolith.
- Transforming bulky and complicated 'canonical’or proprietary web service (WSDLs) to discrete Domain Driven Design modelled APIs.
- Choreography between microservices and the enterprise monolith.
- Fault tolerance or fall back in the enterprise monolith.
- Receiving events from the monolith via APIs and lightweight messaging.
With the growth of microservices, the lines between integration and application have become harder to identify. A microservices approach is not a golden ticket, and might not be appropriate for all use cases. It is however, forcing a rethink of traditional systems integration.
Is it necessary to have an ESB in a microservices environment? Probably not. Are microservices a good fit for an integration layer between traditional legacy enterprise monoliths? Possibly, it depends on the situation. An ESB is fundamentally a set of patterns normally implemented by a software product. So why can’t it be implemented using a microservices approach? If good SOA principles and patterns are used, it would be far easier to implement a microservices based approach. Perhaps, there are more important questions to consider when assessing options for your situation. For example, what is my existing investment? Or what is the team’s skillset? Therefore we can no longer ignore microservices across these paradigms, it demands our full attention.