Whitepaper - Migrating to a hybrid integration platform (HIP) makes your cloud journey easy

Abstract

Digital transformations tend to focus on the desired high-level benefits such as greater efficiency and profitability, more operational and business agility, and better customer experience. This top-level view overlooks a major, pivotal element of digitalisation which is foundational to the success of all transformation efforts – integration.

Integration is what pulls technology initiatives together into a cohesive, actionable strategy. It is especially important as companies of all sizes and types evolve into extended enterprises – that is delivering services, solutions and products via an ecosystem of partners through platform-based business and operational models.

These models are the most efficient ways ever found of bringing together buyers and sellers, as shown by the fact that many of the world’s most valuable companies are platform-based.

This shift to platform models is only part of the story though: Enterprises are running their IT systems across many different infrastructures, from on-premise, to public, private, hybrid and multi-cloud. Integration is what enables them to work seamlessly, and the systems to become infrastructure agnostic.

Done correctly, integration ensures business outcomes are greater than the sum of the smoothly interoperable parts. Done badly, it becomes an inhibitor, preventing the recycling and reuse of IT building blocks, and stopping internal capabilities from being consumed by external – and possibly even internal – parties.

No wonder that in 2018, Gartner predicted, “Through 2020, integration work will account for 50% of the time and cost of building a digital platform”. This white paper explores how a platform-based approach to integration itself and the democratization of integration are the keys to success.

Gartner’s solution to many faceted challenge of transforming integration as an enabler of wider digitalization efforts is the hybrid integration platform or HIP. The ‘hybrid’ in HIP describes the essential co-existence of legacy and modern integration technologies as part of digital transformation.

Over time, enterprises will transition from integration technologies that have been around for a decade or more, progressively moving to the newer tools and approaches, which form the building blocks of modern integrations. Not all legacy technologies will be modernized, but their retirement still needs careful oversight.

Specifically, newer integration platform-as-a-service (iPaaS) and cloud-native integration models deliver better return on investment from integration transformation.

As interconnections with partners in extended enterprises proliferate, so do the number of integrations, which will evolve to become a part of everyday business life for many more people, rather than the province of specialists. A hybrid integration platform provides easy access to user-friendly tools, boosting reuse and consistency.

Integration transformation is integral to the success of any enterprise’s digitalization and specialist help is recommended, as outlined in this paper. The paper sets out to demonstrate that although transformation attempts fail, and typically at considerable cost, it doesn’t have to be that way.

Introduction

Integration is often overlooked in the rush of corporate enthusiasm for digital transformation but it is a large, inherent part of any digitalization initiative. That means that integration itself needs to be transformed as part of and to enable the wider digital transformation. Already IT teams globally are spending over a third of their time on integration projects, according to the MuleSoft’s 2021 Connectivity Benchmark Report, and custom integrations are costing large enterprises on average £2.5 million ($27.55 million) each in annual labor.

As digital initiatives accelerate, integration is a critical factor in determining the success and speed of digital transformation across every industry sector. What’s more, integration’s importance and impact are expanding all the time as:

Integration is no longer solely about enabling and improving internal processes and interoperability. It is essential to enable companies to evolve into extended enterprises, offering services, products and solutions to customers as part of an ecosystem. Many businesses in every sector are moving towards this model because as the world’s largest platform- and ecosystem-based companies show (such as Apple, Amazon, Facebook, and Google), it is the most efficient way yet found to bring buyers and sellers of products and services together.

Partners in ecosystems interact via platforms that bring buyers, sellers and end users together, enabling B2B and B2B2X transactions. The platforms also help organizations open their internal capabilities to third parties via open APIs.

Applications run across different kinds of infrastructure and straddle boundaries, such as a company’s own data centres and public, private or hybrid cloud, or even multi-cloud. The backend of an organization typically runs traditional back-office systems and applications in parallel with those on the cloud.

Increasingly enterprises are looking to leverage the full capabilities of running applications in a cloud-native environment, rather than simply doing a ‘lift and shift’ of legacy applications to the cloud.

Integration is also at the heart of other defining characteristic of digital companies, such as the fact they are data-driven, which means being able to pull data from varied, multiple sources to extract intelligence from them with analytics, and, increasingly, AI.

Integration is an intrinsic element of automation, which is most useful when implemented from end to end, rather in islands. Automation can only ever be as good as and as extensive as the integration that underpins it.

During digital transformation initiatives, integration often becomes a bottleneck with demand outstripping enterprises’ capacity. Also, the scope of traditional integration tools and approaches are unequal to the new tasks asked of them because they weren’t designed for those purposes, which makes progress slow, expensive and unsustainable.

As Massimo Pezzini, Research Vice President and Gartner Fellow (now Emeritus), said back in 2018, “The complex challenges posed by digital business transformation require a radical change in the integration technology platform and in the way organizations deal with integration.”

1. Shooting for transformation from the hybrid integration platform

Gartner’s solution to ensuring integration remains central to digital transformation and is itself able to evolve to meet changing needs is a hybrid integration platform or HIP. This approach has gained wide acceptance in the market.

What should the HIP offer?

It is likely that some application teams are already updating and extending their tools to support the migration to cloud services, mobile apps, IoT and AI, for instance. Typically, they are using integration platform-as-a-service (iPaaS), often with an API management platform, and these two elements form the core of hybrid integration platform.

In Gartner’s view, a successful hybrid integration platform needs to include these four dimensions:

• Personas or constituents, such as integration specialists, ad hoc integrators, citizen integrators and digital integrators
• Integration domains like applications, data, processes and B2B
• Endpoints ranging from on-premise devices to cloud, mobile and IoT devices
• Deployment models – on-premise, multiple cloud environments, a hybrid of on-premise and multiple clouds, and embedded IoT devices.

Letting go

Gartner warns that IT departments should understand that digital transformations tend to reduce their control over IT-related initiatives when considering future scenarios, which in any case are hard to predict.

This unpredictability becomes more acute when different parts of the organization begin experimenting with new approaches, from software-as-a-service (SaaS) for Human Resources to IoT in R&D, say. Data might become the province of a Chief Digital Officer, who is learning by doing how and where to deploy AI technologies, such as machine learning, to best effect.

Although separate, these are all parts of the same digitalization drive and need a degree of integration into core IT systems – and perhaps to interwork with each other too even though the IT department might not know about them – and it is likely many systems were not intended or designed to interoperate with others.

2. The anatomy of a hybrid integration platform and how it helps

Many enterprises are committed to taking the HIP path to transformation, but don’t know where to start, what to prioritize, to identify what should remain as legacy integration and what should be modernized, or what their overall integration landscape should look like. On top of grappling with new technologies and the greater urgency for digital transformation that has been underscored by the pandemic, it is too difficult to attempt digitalization without specialist help. This includes integration transformation, which at the heart of the wider effort, but is unchartered territory for most.

The ‘hybrid’ in HIP describes the essential co-existence of legacy and modern integration technologies as part of digital transformation. The schematic in Fig.1 below shows the technologies that have been around for a decade or more in the left-hand column and the blue boxes show their modern counterparts. Over time, there will be a progressive move to the newer tools and approaches, which form the building blocks of modern integrations. Remember that not all legacy technologies are destined to be modernized – some will become obsolete and be retired.

Building blocks: digital-ready, modern, hybrid integration landscape

This is complex situation that needs to be carefully managed and timed, so integration transformation is a crucial, detailed discipline in its own right, as well as being tightly woven into the IT fabric.

Integration transformation must be addressed on a case-by-case basis, as no two enterprises are at the same starting point, with the same transformation priorities or schedules and desired outcomes, but at the end of the process they will have a fit for purpose HIP.

As we saw in the last section, transforming integration by creating a HIP is a big mindset and organizational change, as well as a technology transition. The best way to approach any highly challenging and complicated task is to break it down into manageable, contiguous steps. Transforming integration with a HIP is no exception because while it needs to be treated a key element of digitalization, at the same time it is intrinsic to the entire effort.

This is reflected in the specialist services offered by Torry Harris Integration Solutions (THIS) shown in the graphic below. The following are specific to integration transformation and the HIP approach:

• Consulting on integration transformation and architecture
• Setting up governance for integration
• Setting up the hybrid integration model and migration services

The others also exist independently in the wider digitalization landscape as the HIP is multi-dimensional, hence elements such as extending third parties’ full stack products using microservices and transforming legacy applications to cloud-native are part of it. Even so, in this context the specialist services are approached with the primary on the integration transformation.

A good analogy for THIS’ approach is that it is like a travel portal. Customers can use it simply to book a flight – the equivalent of only taking the consulting option for instance, or purchase any combination of flight, hotel, car hire, insurance, tickets for local attractions, and more.

THIS’ eight bundles of services can be consumed individually, or in any combination, depending on what an enterprise needs, what in-house expertise it has, the stage it is at, its desired outcomes and other variables. In composite, the eight bundles of services cover the key areas needed to build and run a successful HIP for integration transformation.

This white paper focuses on the three that are specific to hybrid integration platform.

3. Developing an integration strategy and transformation architecture

Where to start with developing an integration strategy and transformation architecture? The first step THIS recommends as part of its consulting on transforming to a HIP is to assess the maturity of an organization’s integration tools, approach and resources. It offers consulting services to help clients all the way from this assessment to building a roadmap.

The integration maturity model matrix

As the table (Fig.2) above shows, the matrix includes identifying gaps, and not just for current needs but for likely future needs as digitalization progresses. Once the strengths and gaps are recognized, the next stage is to develop an integration transformation roadmap to figure out priorities and the timing to meet the needs of the wider transformation effort.

The roadmap helps make the business case for integration transformation overall, as well as in specific instances, by mapping business capabilities to integration components. This in turn helps to define the migration strategy for integration transformation. The strategy must include governance and an empowerment framework help with the democratization of integration, as discussed in the last section, instead of remaining the province of technical experts – a scarce and expensive resource.

The overarching architecture for integration

The integration architecture is needed to consistently deliver business objectives. The factors that must be considered include:

• integration of both applications and data
• asynchronous and synchronous integration scenarios
• hybrid integrations
• integrating user interfaces
• using decision trees and integration rules that determine when to use which integration method and tools – as shown in the example below.

Master decision tree

Leveraging THIS’ architecture frameworks and decisions trees can accelerate progress substantially. The company offers workshops tailored to specific clients to help them construct the major components of their governance model or for a comprehensive review of their current set-up.

THIS also can help in setting an integration center of excellence (CoE) as a reference point and knowledge base for the entire client organization, which is constantly updated and available to all. The CoE can help with issues like comparing and selecting products and components, such as:

• infrastructure platform-as-a-service (iPaaS)
• microservices framework
• enterprise service bus (ESB) software
• micro-integrators
• containers and orchestration
• monitoring
• automation and more.

The final element in the THIS’ consultation services for transforming integration is developing a DevOps strategy to put the plan into action. Here a key factor is creating a dedicated roadmap for adoption which is tightly aligned with governance: It doesn’t matter how creative or brilliant your integration solutions are, if they are not widely and consistently reused, then digital transformation will fall short of delivering the expected benefits – and possibly catastrophically short.

4. Setting up governance for integration

Governance is critical to the success of all transformation, and arguably even more important when you have a HIP as teams may not be aware of how the company is going about transforming its legacy IT to a modern stack and how to maintain the right level of hybrid integration. Having witnessed and addressed these issues first-hand with clients, THIS launched its integration governance and empowerment framework in May 2021.

Good governance sets up the structure and processes required to produce the desired results for all stakeholders, making the best use of an enterprise’s resources at any given time. The schematic below shows what a mature governance framework looks like.

Shuba Sridhar, VP – Strategic Initiatives at THIS, points out, "There is no predefined governance structure that intrinsically delivers business benefits…We analyze how clients measure performance, and what their capabilities and constraints are. We then establish a realistic, fit-for-purpose team with clearly defined roles and responsibilities."

As a foundation, governance must identify three key profiles:

• The beneficiary is the individual or team that assumes the responsibility for delivering the expected benefits of digitalization to the intended selected community of users.

• The enabler is the team tasked with modifying or adding elements to legacy assets to meet the enterprise’s digital needs.

• The catalyst is the team that aligns the beneficiary’s priorities with the enabler’s change roadmap. To put it another way, the catalyst is responsible for delivering the integrated experience.

The governance and empowerment framework facilitates good governance, allowing the initiative to evolve, and iteratively present best practices based on results. The framework is also designed to enable cohesive integration across the enterprise so that all elements are connected, rationalized and organized to provide consistent guidance and incentives that executives and leaders of business units require.

API governance is key part of the integration and empowerment framework as APIs operate at many levels. The top level embeds APIs in enterprise-wide architecture governance, and policies and process enforcement flows from that architecture into IT governance.

At the program level API governance covers areas like API portfolio management (see the diagram below), which determines the inputs and drivers, such as controlling a domain-driven approaches and approaches based on customers’ journeys.

Then API governance must oversee demand management for different types of demands and channels and sources, and identify API best practices and guidelines for interface design.

There are business as well as technical angles to API governance, including funding and charging models for the APIs, monetization models and measuring customers’ satisfaction with the APIs. You can read about The Digital (API) Economy here to learn more about business and operational models.

It’s all about aligning the API and IT portfolios, as shown below.

If you don’t get governance, you won’t get transformation

Integration governance typically factors in all aspects of governance, from the duties of the decision-making authorities to the integration team’s structure and the team members’ day-to-day integration responsibilities that underpin the new B2B, B2C and B2B2C scenarios.

THIS’ whitepaper, Why Governance is the key to ROI in Digital Transformation, looks at five key success factors for digital transformation in which good governance has a foundational role. It explores failure and success stories, and offers recommendations to achieve customer transformation objectives.

Sridhar notes, “The role of Integration and API governance is to balance competing objectives for the benefit of all stakeholders’ interests. The purpose of governance is to align the interests of all stakeholders as closely as possible to the objectives of the organization’s integration-driven digital programs”.

5. Setting up a hybrid integration model and migration services

THIS can help clients to design and develop technical building blocks they need based on the HIP model, in context to their organization – an example is shown in the diagram below. As mentioned previously there is no one-size solution fits all.

The schematic below (Fig. 6) shows a reference architecture of a HIP with legacy integration components such as the enterprise service bus (ESB) and webservices alongside modern integration components such as containers, microservices, and an API gateway. The right-hand side shows DevOps as an element that cuts across many blocks, plus API management and a digital ecosystem, which is the key driver for creating the HIP – the HIP facilitates the transition to an extended enterprise. No two enterprise’s landscapes will be identical, so each has to design a model of what they want and use migration services to get them there in the most efficient and effective way.

Hybrid Integration Platform

Part of this exercise is to identify the candidates for HIP’s different building blocks in the context of what an enterprise has now – some parts of the organization might have progressed to modern technologies like iPaaS and microservices, while others are soldiering on with webservices, ESBs and so on.

The migration strategy isn’t just about which legacy technologies need modernizing or retiring, but how fit for purpose they are, business priorities and synching with other transformation activities and goals, including timing. Integration transformation is the engine house of digitalization, it cannot be looked at in isolation.

Once the migration strategy is in place, then THIS can help to implement it, then offering support and operational services for the migrated integration technologies within the hybrid integration platform.

Addendum

THIS offers the following products and associated services that can be used for integration transformation: