Every successful digital transformation initiative begins with data integration because all digital companies are data-driven. The consensus is that the future of integration is through a hybrid, cloud-based platform model, also known as integration Platform-as-a-Service (iPaaS).
The name is a little misleading as iPaaS comprises several platforms and is “a set of automated tools that integrate software applications deployed in different environments. Large businesses that run enterprise-level systems often use iPaaS to integrate applications and data that live on-premises and in both public and private clouds,” according to TechTarget.
It has to be hybrid in nature because integrations need to span legacy applications, systems, data and integrations, and the cloud-native environment as enterprises’ cloud migrations gather pace. iPaaS can connect applications that are run locally (on-premise), in the cloud, or across clouds.
A platform approach is an obvious choice because it is the most successful method yet devised to bring disparate technologies together and scale them using APIs. These APIs can be replicated and reused limitless times – think LEGO® bricks – allowing speedy, automated configuration rather than customization, which is forever a time-consuming, manual task to maintain and update.
Enterprises around the globe spend trillions of Dollars or Euros on IT every year. Estimates suggest that 38% of IT teams’ time is spent on designing, building, and testing new custom integrations between systems and data. The labor involved in each custom integration costs an average of $3.6 million. Large organizations could have thousands or tens of thousands of integration points, and custom integrations don’t scale.
For integration to play its pivotal role in digitalization, it must itself be transformed. iPaaS is an increasingly popular choice because it can impact many aspects of businesses by supporting autonomous integrations that reduce cost, eradicating many repetitive, manual tasks. Verified Market Research says, "The growing awareness and adoption of iPaaS among business enterprises is due to the promise of its cost-effectiveness and use of cloud-based systems to manage soaring quantities of data."
Its recent Integration Platform As A Service (IPAAS) Market report found the global iPaaS market was worth $2.57 billion in 2020 and predicts this will rise to $27.1 billion by 2028, with a compound annual growth rate of 37.2% between 2021 to 2028.
Despite these strong business and operational cases for modernizing integration, technology is only part of the transformation challenge and not an automatic guarantee of success. This white paper looks at how iPaaS can make a big difference and how enterprises can make the best possible return on their investment (ROI).
Introduction - Why iPaaS is increasingly the modern integration solution of choice
Integration is a Herculean task for enterprises that are in the throes of digital transformation themselves. It involves thousands or tens of thousands of integration points, multiple generations of systems, and a mishmash of integration technologies and approaches and their transition to a cloud-native environment. No surprise that most are finding it challenging.
- Data silos are a persistent challenge for 90% of organizations
- Companies have more applications than ever (976, up by 133 since last year’s benchmark) but are less successful at integrating them (only 28% are integrated, slightly less than last year) as they grapple with siloed data, legacy code, and systems
- There is an acute skills shortage
- On average, IT departments are being asked to deliver 40% more projects than in the previous year
- 52% of IT projects were not delivered on time in the 12 months preceding the research
- Existing infrastructure is acting as a bottleneck
- More than a quarter (26%) of business leaders demand a company-wide API strategy
- The number of firms using APIs to build integrations has not increased since the previous year’s report, although, on average, 35% of organizations’ revenue is derived from APIs and related implementations
- Companies are recognizing that the combination of API-led connectivity and automation can deliver better employee and customer experiences
- The growing number of applications combined with current custom integration practices are an increasing technical debt, with organizations spending more than $3.6 million on average for the labor involved in custom integration versus $3.5 million the previous year
- 88% of respondents said that integration challenges are slowing or hindering digital transformation to some degree
- Respondents estimated that 38% of it teams’ time is spent on designing, building, and testing new custom integrations between systems and data
To fulfill its pivotal role in digitalization, integration must itself be transformed to take advantage of new technologies and models to secure a better return on investment. This is where iPaaS comes in, which Gartner defines as "a suite of cloud services enabling development, execution and governance of integration flows connecting any combination of on-premises and cloud-based processes, services, applications and data within individual or across multiple organizations."
iPaaS and the platform landscape
There are three primary types of cloud computing services: Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), and Infrastructure-as-a-Service (IaaS).
Where does iPaaS sit in this platform landscape and within an organization? A simple way to think about it is that workers within business functions use SaaS-based applications hosted in the cloud by a third party. iPaaS can be used to simplify hybrid SaaS deployments that have legacy elements. Business workers also interact with information systems (IS) run by the IT department.
The IS – or a subset of them – run by the IT department connect to the iPaaS, which is linked to both PaaS and IaaS. PaaS is a set of cloud-based tools that developers use to oversee services and for specific development projects so that the developments don’t cause disruption, whereas iPaaS combines a number of platforms and delivers benefits on a wider scale than PaaS.
IaaS provides virtualized computing resources across the internet. The cloud provider manages elements like storage, servers, and network resources and makes them available to users via virtual machines across the internet.
Usually, an iPaaS platform comes with an integration toolkit, ready-made connectors, and the option of building custom ones. It can process and cleanse data, putting it into modern formats like JSON and XML, and supports common protocols like FTP, HTTP, Open Data Protocol and Advanced Messaging Queuing Protocol.
It is built to handle scale and multiple integrations concurrently, whether integrations need to be carried out in real-time, near real-time, or batch processed. It monitors performance, identifying failures, latency, use of resources, workflows, and more.
iPaaS contains business rules, maps, and transformations to help with applications’ development and integration flows – iPaaS supports SaaS and local applications. It also controls access and uses security measures such as data encryption and single sign-on integrations.
Some provide development kits to modernize legacy applications and additional functions like integration with social platforms or managing business data. IT departments can carry out integration themselves, but often it makes more sense to lift that burden and use an iPaaS provider with a proven track record, as we explore in Section 2.
Whichever option enterprises choose, unless they adopt modern practices and tools, integration will sabotage companies’ wider digitalization initiative: at best it will hold it back and at worst, cause at least parts of it to fail. Even so, careful planning is needed to ensure maximum ROI from iPaaS.
Top tips for maximizing iPaaS ROI
As we have seen, the business and operational cases for iPaaS are clearly attractive, but they do not automatically guarantee ROI. There are many potential pitfalls, so careful planning is required to maximize it. Here are our top six ROI recommendations.
Consider letting an iPaaS provider take the strain
There’s a lot to think about when it comes to integration, as the issues briefly outlined in this paper show. The situation is going to become more challenging as the number of applications and integrations increase and as more applications, data, and systems are moved into cloud-native environments.
From devising an integration strategy as part of the broader digitalization plan, to plotting tactical actions that enable digital transformation and leveraging DevOps methodologies and AI, there is a lot to consider, and much of it is new territory.
The IT department could, in theory at least, handle the integrations. Still, often it makes more sense in terms of time, money, and resources to put setting up, operating, maintaining, and/or updating an iPaaS into the hands of an experienced third-party provider.
Here are the things a reputable iPaaS provider, with a proven track record, should be able to do for you:
Don’t stay on the bus too long
Needing to ‘replatform’ applications is a big driver of digital transformation. Traditional monolithic applications are expensive and difficult to operate, maintain and evolve, and tend to silo data. Organizations looking to ‘replatform’ are sometimes uncertain why they should choose iPaaS over the longer-established enterprise service bus (ESB).
ESB’s architecture is designed for on-premise systems and applications rather than cloud-based ones, although some ESBs have new features to make them more cloud friendly. The biggest difference between ESB and iPaaS is flexibility: ESB is designed for vertical integration and cannot support multi-tenancy, whereas iPaaS has a horizontal approach that enables interaction between many parties.
Also, it is designed to deal with ad hoc integrations, which contribute to business agility, whereas ESBs are not.
In cloud-native architectures, large applications comprise loosely coupled microservices, which are small, autonomous, self-contained pieces of software that are organized around business domains.
This allows each part to be monitored, tested, and updated without affecting the others, whereas updating traditional applications meant engaging with the whole application. Hence microservices can deliver more business and operational speed and agility because organizations can update a technology stack without needing a specific technical framework to build and use microservices, they just follow principles and best practices resulting in a lightweight, adaptable architecture.
Understandably, everyone wants to wring every scrap of value they can from assets that have served them well, such as ESB. Ultimately the move to iPaaS will be inevitable for most, if not all, enterprises.
However, it is likely that many companies will run ESB and iPaaS in tandem during their cloud migration and possibly afterward for some legacy applications.
It’s important to have a timetable in place for ‘getting off the bus’ at the right time for various applications, well before the limitations of ESB disrupt the transformation journey.
Avoid messing up with microservices
Microservices bring many benefits, but only if they are deployed correctly. Implemented wrongly, they can prove both expensive and inefficient. Microservices are not the best option in every situation. Start with the desired business outcomes and figure out the best way to deliver them rather than starting with possible uses for new technology.
Developers write code for the business logic of a microservice. This works especially well for use cases with self-contained logic like applying tax rules or discounts. However, many use cases involve several systems of records, and, as with APIs, governance is key as different teams tend to reinvent the wheel with similar solutions. This duplication is expensive and slow way of working overall.
Another issue is that each microservice needs its own infrastructure because each can scale independently, and this too can quickly become extremely costly. Also, services need to communicate with each other, but a poorly designed service will likely involve many remote calls, which creates latency in the network and processing.
One of the big drivers of digital transformation is automation, but inconsistent approaches (like reinventing the wheel as outlined above) and customization instead of repeatable configuration hamper or make automation impossible.
Companies need to align their microservices strategy with their integration plan to avoid ending up with expensive microservices-based solutions that are less efficient than the traditional monolithic applications.
Complementing iPaaS’ capabilities
Users should be able to complement an iPaaS’ capabilities with other tools, where appropriate. For example, many customer journeys and use cases cut across multiple systems of record, connect to partner systems, and draw data from cloud-based services. This likely requires complex composition and orchestration with multiple integration points, which iPaaS is designed to provide.
However, there are some use cases that are mostly self-contained and only rely on data from external systems in some circumstances. Here a micro-integrator, such as Coupler, is a better option because it can leverage the visual design flow of iPaaS and the attributes of a microservices framework, such as containerization and scalability. In addition, its low-code approach means less skilled developers can use it, expanding an organization’s talent pool for integration.
The diagram shows how Coupler integrates a legacy system with microservices built on containers (Docker) and is deployed using a container orchestration platform (Kubernetes).
API management and governance
A significant strength of iPaaS is that it can accommodate the integrations needed for legacy and cloud-native environments, for internal operations, or to expose capabilities and services to third parties, including suppliers, customers, and partners. As we saw in the Introduction, this is really important when, on average, 35% of organizations’ revenue is derived from APIs and related implementations.
To get the full benefits, it is essential to reuse APIs wherever possible and deploy them in a uniform way (to avoid tweaking and difficult retro-engineering later). An iPaaS implementer should be able to help customers all the way through the API lifecycle, as shown below.
The last stage is frequently neglected and costs organizations dear: APIs need to be maintained and updated across the board. Consistency is the key, and different versions of APIs result in fragmentation which hinders automation. Just what a difference this makes is illustrated by BT reducing its partner onboarding time by 95%.
Ease of use, accessibility, and consistency will become ever more important, Gartner says, because as the volume of integrations grow, increasingly it will become a part of many non-specialists’ everyday responsibilities rather than the job of technical specialists.
Some iPaaS providers include API management and governance, but not all of them. As ROI is deeply impacted by how efficiently and effectively APIs are reused, this should be a major consideration. Governance must help make the organization more agile, not slow it down. Torry Harris Integration Solutions (THIS) launched an integration governance and empowerment framework to enable cohesive integration across large enterprises. It is designed to ensure all elements are connected, rationalized, and organized to provide the consistent guidance and incentives that executives and business unit leaders require.
It should be applied in conjunction with an API manager, such as the award-winning DigitMarketTM API Manager which has micro-gateway capabilities and can support any custom gateway behavior. Each micro-gateway has a subset of APIs that are managed by it. The DigitMarketTM API Manager gateway can be deployed across a set of containers and managed using Kubernetes due to its distributed architecture.
Istio or similar service meshes can be used to control the flow of traffic. The HTTP listener node can be replaced with a service mesh where extreme scalability is needed through the configuration of and enhanced deployment architecture for the gateway.
DigitMarketTM API Manager also supports pre-built micro-gateway artifacts like usage policies, API bundling through packs, security policies, and other common gateway functions.
Establish the total costs of using iPaaS – and not using it
iPaaS involves training, recruiting, and paying developers. If customized coding is required, such as to accommodate legacy systems, then extra developer resources will be needed.
It’s essential to accurately assess how long a developer takes to complete different integration tasks. This provides a benchmark from which to deduce how many hours of human effort have been saved by automating tedious, repetitive manual tasks.
Saving hours of human effort is only part of the story: what you do with those hours is even more important. The idea is to use them productively for higher-value tasks. This should progressively become the case the longer the DevOps methodologies of continuous integration/continuous testing/continuous delivery or deployment (CI/CT/CD) are in use and constantly refined by machine learning.
Companies also need to consider the cost of tools, and an enterprise might have to call in specialist help to keep the integration process on track.
Other elements to consider:
Note: Decisions about which systems to connect to what others, when, and how need to be part of the transformation strategy and mapped out in detail.
A piecemeal approach will result in fragmentation, which defeats the object and hinders automation. The strategy must span legacy, current and future systems and applications, through consistency, transparency, availability, and ease of use.