Cloud-based integration technologies continue to evolve. Where two-three years ago enterprise-grade cloud integration had a select few choices led by technologies such as MuleSoft, Dell Boomi, Informatica and Snap Logic, there is now an array of services and iPaaS (Integration Platform as a Service) available to build robust and scalable solutions. In my opinion one of the most significant developments has been the maturing of integration services from global cloud providers such as Amazon Web Services (AWS) and Microsoft Azure. These ecosystems are now providing all iPaaS services and can enable businesses to build affordable, fit-for-purpose integration solutions – a concept I covered in my previous blog Using AWS for Enterprise Grade Integration. We have been doing integration implementation using AWS for some time now and more recently we have implemented solutions using Microsoft Azure platforms as well.
In this blog I will compare the enterprise integration capabilities of AWS and Azure, discuss best fit use cases and share my experiences for minimising costs, reducing complexity and improving quality.
Comparing AWS & Azure for Integration
When starting to look at integration with cloud-based tools, a reference architecture can go a long way as it gives a good indication of how the cloud provider built their ecosystem to be used for this purpose. Microsoft Azure provide a well-defined reference architecture for enterprise integration – this is well worth a read and can be a great starting point for architects. AWS has lots of blog-style content and whitepapers on various integration architectures.
These days both AWS and Azure provide similar tooling and capabilities for integration, but the differences are there and can affect suitability for different scenarios. Solution architects need to know what these differences are in order to design for different business challenges, so in this blog I will compare the two cloud providers across a few dimensions, as I’ve experienced, specifically for the purposes of integration: serverless, APIs, orchestration, DevOps and persistence.
Serverless, APIs and General Features
As the stronger, earlier cloud player in the market, AWS do all the basics of technical aspects like IT cloud computing very well (Reliability, Availability, Scalability, Performance, Visibility, Repeatability, etc.), and also offer a huge variety of tools & features. For example, AWS have easier cost-efficient serverless API gateway High Availability (HA) options, it is easier to roll out a private API gateway via the click of a button or config option in template, and their devops capability has all the features you’d want to roll out new features quickly in their environment.
Azure also cover the basics well, though perhaps not as maturely. For example it takes a lot more effort to roll out a private gateway, a serverless private API gateway is not an option (at the time of this blog’s writing), and their devops needs a few features that would make the feature development lifecycle faster. The tradeoff is that Azure offers more depth than AWS in certain specialised areas useful in the integration lifecycle – for example, Logic Apps provides a no-code, configuration, feature-rich orchestration & connector capability with a nice, visual, easy-to-use IDE. Azure also has loads of great easy-to-use integration connectors & patterns with its Power platform, which helps to accelerate integrations with platforms such as Dynamics, Cloudflow, and PowerBI. Of course, one of the major selling points for Azure in general is its seamless interaction with the rest of the Microsoft suite, particularly attractive as most businesses we encounter are already operating Office 365 or Active Directory.
The conversation changes, however, when you talk about serverless. In our experience a well-architected serverless approach saves on cost, reduces complexity (e.g. using AWS API Gateway instead of installing and maintaining API gateway software on server infrastructure, or lambda vs tomcat on computing infrastructure, or SQS vs ActiveMQ on server infrastructure), and improves quality (e.g. no server infrastructure to upgrade or that can fail). This is a major advantage for AWS, who have a particularly comprehensive serverless offering – their API offering, for example, is serverless no matter what options or features you choose. AWS SAM (an open-source framework for building serverless applications) is also a great tool in the Amazon space, which has very good localhost developer capabilities enabling an easier & faster developer lifecycle.
Azure provide some serverless API options but if you want advanced features, such as Azure AD integration (e.g. for OAuth 2.0 authentication), you need to choose one of the tiers (SKUs) that spin up actual per hour CPUs (increased cost, complexity & maintenance). For APIs, Azure has a ‘higher’ level API capability with tools such as a developer portal, but this comes at a higher cost.
Technical Resources and RPA
Working with AWS can become trickier and more expensive when you factor in technical resources. Its extensive ecosystem can prove highly challenging for non-technical or inexperienced users.
Working with Azure on the other hand is a bit more accessible when doing the basics of standard enterprise integration patterns. However, it can quickly become very technical when, for example, a logic app connector or action does not exist and you need to use an Azure Node.js or C# Function to e.g. generate public/private RSA256 2048bit key pairs or sign a Json Web Token using a Private key.
It is also worth mentioning that Microsoft has an RPA offering called Power Automate desktop flow with some really good Azure integration services interaction patterns out of the box, whereas AWS does not have an RPA offering yet (to see more on using Power Automate in an enterprise scenario, check out the blog by my colleague Kevin Henshall here). Having RPA available helps integration designers integrate more easily with legacy applications that do not have well defined interfaces – RPA is often the only viable option to integrate in these situations.
Orchestration and Low Code/Codeless
Although AWS has a wider range of more ‘technical’ products (requiring people with more technical skills), Azure has a wider range of ‘codeless’ options, e.g. for workflow orchestration. AWS Step Functions is a great tool which is created via json configuration (and still requires a lot of lambda backing for anything beyond the rudimentary), but Azure provides a rich orchestration capability through Logic Apps. This also comes with a drag ‘n drop editor as an alternative to json configuration. Additionally, Azure Logic Apps has a large out-of-the-box connector library enabling less dependence on the much more technical Azure Functions. Beyond this there is an even less technical orchestration tool, Power Automate Cloudflow, which is based on Logic Apps and easily interacts with Azure integration services. This tool makes it easy for non-technical users to create some business processes. The flip side of this is that it is easy for the costs to increase significantly (e.g. when using Logic Apps by enabling an Integration Service Environment (ISE), or an integration account), so one has to be careful to architect the correct approach to minimise costs while at the same time fulfilling the technical and business requirements. For example, instead of running ‘inline code’ in a Logic App, it might be cheaper depending on volumes to do what is required in an Azure Node.js or C# function.
Since Azure re-worked their DevOps approach a few years ago both Azure and AWS now provide very good and similar automated devops capabilities with git triggered pipelines controlling deployment to their cloud environments using native, out-of-the-box adaptors. Seeing as AWS has been using this approach for longer it is to be expected that they have more mature capabilities, although the capabilities provided by Azure are more than sufficient for enterprise grade integration using their reference architecture approach.
With some integration patterns we sometimes need to persist e.g. events, auditing information, correlation data, reference data mappings. These days we prefer a fit-for-purpose persistence, e.g. Kinesis for event streaming or Azure data lake storage gen2, but for more generic purposes we prefer a NoSQL serverless persistence approach. In this approach we avoid complex relational schemas in exchange for small narrow focus microservices style built-for-a-specific-purpose persistence e.g. persistence for the purpose of a ‘single view of the customer’ separate from persistence for the purpose of customer event auditing. Both AWS and Azure provide good options in this space, with AWS DynamoDB and Azure CosmosDB which have good user interfaces & connector/library support. The added benefit of Azure CosmosDB is that it can expose a relational interface (ODBC driver) to enable, for example, PowerBI to report on data within CosmosDB.
Configuration-based connectors speed up integration delivery in connecting your apps, data, devices and services together because they’re easy to use – for developers and non-coders alike. Azure Logic Apps has one of the most extensive lists of connectors of any integration platform out there, with over 200+ currently available for popular SaaS apps, including Dropbox, Dynamics, Outlook, Twitter, and Salesforce.
Logic Apps also provides a growing list of connectors that allow developers to build integration flows and get data from and push data to many different protocols, SaaS applications, other Azure and Power Apps Services, and on-premises systems.
AWS is increasing its connector library every day (e.g. range of Step Function tasks, or Event Grid connection options), but relies more on Lambda for wrapping interfaces via e.g. Node.js packages.
Right-Fit Use Cases
Every situation is different and has different considerations, and the type of architectural approach makes a huge difference. However, there are certain scenarios where I would recommend the use of one vs the other of these two cloud providers. Where a business might be looking to lower software costs, the cheaper, more robust AWS serverless tools work extremely well, especially with an event-driven approach. Likewise, for event streaming, large volume integration, or particularly technically complex integration requirements, AWS’s range of technical features provides a good fit. Azure can be a better choice for environments with less technical and simpler integration requirements, saving on resource costs and complexity. Azure is also a good choice for large existing Microsoft ecosystems, or where there are custom legacy systems without well-defined interfaces, making the desktop flow capabilities particularly important.
None of these scenarios I mentioned are perfect as there are always many dimensions (such as speed of implementation, cost efficiency, architectural approach, complexity, skillset of resources, ease of support, ability to change, etc.) to consider when choosing an approach for a specific integration challenge. The important thing is that you weigh up the options across these key factors knowing the strengths of each provider. Once you have chosen your preferred provider, commit, give everything and don’t hesitate. Both providers are the best cloud providers out there and are great options to do integration with, so there are no ‘wrong’ choices, only implementation dimension differences for your specific situation.