Abstract
Defining, much less executing, cloud integration isn’t straightforward. At the simplest level, it means bringing together multiple IT environments – on-premise, public and private cloud, a hybrid of the two, and multiple public clouds – multi-cloud.
The purpose of cloud integration is to create a cohesive IT setup, regardless of the underlying environments to provide:
- Flexibility regarding how data is shared, stored, processed and accessed;
- Greater operational efficiency and business agility through easier access to data and faster data analysis to launch and tweak new offers, for example;
- Better customer experience due to better joined-up processes, for instance;
- Modern could-based applications
- Scalability to accommodate rapid changes without having to build capacity that is only used in peak times; and
- Combined cloud applications and on-premises systems to create a whole greater than the sum of its parts.
Cloud migration and integration are not only about doing what you do already but better and for less money. These are desirable, but don’t leverage cloud’s potential of facilitating collaboration between teams by enabling them to combine, process, analyse and act on data in ways that were not previously possible. Integration is integral to accomplishing this, but is a challenging undertaking technically, with many moving parts.
Further, cloud integration is a critical aspect of cloud migration, which is itself part of an enterprise’s overall digital transformation jigsaw, and needs to be viewed in context, not isolation.
Introduction
Enterprises move data, applications and IT functions to the cloud with the intent of hosting them in the best IT environment. This is based on factors like cost, scalability, performance and security, adopting operational and business models such as Software-as-a Service (SaaS). Moving workloads to the cloud should be done as part of an enterprise’s digitalization strategy that includes the impact on staff and their roles, and realigning resources as well as technology.
Without proper strategy and management for cloud migration, lots of things can go wrong – and quickly. For example, cost is a major driver of moving to the cloud, but while pay-as-you-go cloud services can offer greater flexibility and reduce hardware costs, the overall cost could be much higher than unwary users bargained for.
TechTarget reckons that in 2020, 45% of organizations that ‘lifted and shifted’ applications into the cloud, overprovisioned by as much as 55% in the first 18 months, overspending by up to 70% during that time. Note also lift and shift does not leverage the full advantage of cloud’s native, containerized tech.
Arguably, integration is an even more serious issue than overspending, as well as potentially contributing to it and compounding the issue. In the scramble by businesses of all sizes to gain the advantages of cloud, and especially Software-as-a-Service (SaaS) offerings, they have replicated the age-old IT problem of silos.
While the SaaS delivery model makes it easy to access software, the rapidly rising number of SaaS-based applications has created serious integration challenges, in terms of the volume of integrations and their complexity.
Done right, cloud integration ensures business outcomes are greater than the sum of smoothly interoperable parts. Done badly, it prevents the recycling and reuse of IT building blocks and open APIs, and stops internal capabilities and data from being consumed by external – and possibly even internal – parties.
1. Cloud migration
There are many, varied reasons for enterprises wanting to leverage cloud-based resources, including:
- to support real-time applications and services;
- to increase automation in the back office and customer-facing systems;
- to better enable mobile applications; and
- to deploy and scale IT more easily in general.
The CIO must consider:
|
WHAT are you trying to achieve? |
e.g. introduce more back-office automation |
|
WHY do you want to do it? |
e.g. enable self-service to improve customer experience and reduce operational cost |
|
WHERE is the integration needed? |
e.g. identify interdependencies of systems, processes, data and applications, and that not all of them are suited to a cloud environment, for instance, where low latency is required |
|
HOW to execute? |
This is not just about achieving a good solution NOW, but one that can flex and adapt to meet changing needs. HOW cloud integration fits into the wider integration and digital transformation effort is also foundational to success. |
Choosing the best option
There are various approaches to cloud migration – the trick is identifying the one that best suits an enterprise’s current needs and offers flexibility for the future. Partnering with an integration specialist can help assess an enterprise’s current situation and clarify the best options to achieve its goals.
For example, legacy virtual machines and base infrastructure can be migrated to multiple cloud providers or they can co-exist with hyperscalers, depending on factors like timeframe, cost, performance, business goals and more.
Alternatively, applications and data can be moved from one cloud provider to another after end-to-end impact analysis of dependencies, potential effects on customers, migration challenges, fitment issues and so on. Applications can also be re-engineered and re-cast to fit when shifting from one provider to another, so an application built on AWS could be moved across to Microsoft Azure, and likewise data could be migrated from one provider’s cloud to another.
The success of all of these approaches and the many other possible variations on the theme depend on cloud integration.
2. Enabling and accelerating cloud integration with iPaaS
The growing number of SaaS applications has created a big integration challenge, often replicating IT’s age-old problem of siloes. Organizations are turning to what Gartner calls Enterprise Integration Platform as a Service (EiPaaS) as strategic alternatives to classic integration platform software in its Magic Quadrant, published in September 2021.
According to Gartner, “An enterprise integration platform as a service (eiPaaS) is a suite of cloud services that addresses a variety of scenarios, including application and data integration, as well as some combination of process, ecosystems, mobile, AI-enabled systems and IoT integration, as well as API management and digital integration hub capabilities”.
EiPaaSs offer a combination of integration technology functions as a suite of cloud. EiPaaSs are public, stand-alone products that subscribers use directly, not integration capabilities embedded in another offering (such as a SaaS or application Platform-as-a Service (PaaS).
Gartner notes that many software engineering leaders attempt to standardize on a single EiPaaS to minimize complexity and contain costs, but says they may benefit from using multiple offerings to address different use cases or lines of business.
Market research published in August 2021 by The Insight Partners forecasts iPaaS will grow at 21.2% CAGR between 2021 and 2028, at which time the market will be worth more than $8.844 billion.
Despite its continued growth, the EiPaaS market is fragmented, fast-evolving and overcrowded, according to Gartner, which makes it difficult for buyers to evaluate and select vendors. The research house expects “ongoing disruption in this market as it continues to consolidate”.
Case study: Meeting the iPaaS needs of one of France’s biggest energy firms
The energy company wanted a strategic integration platform to gain a single view of the organization and enable easy sharing of information with the business plus customers, partners and suppliers. The platform had to embrace ESB, API management and iPaaS platforms, all defined by a strong governance model.
Torry Harris Integration Solutions (THIS) defined and developed cloud integration architecture, to address diverse needs such as high-volume data integration, managed file transfers (MFT), batch and streaming data. We setup the iPaaS platform for both Cloud to Cloud and hybrid (on-premise applications and AWS). The iPaaS matched the defined set of integration patterns, integration styles and data volumes, including EDI (Electronic Data Interchange). The iPaaS was setup to enable different business units to easily develop integration flows on their own based on “Citizen Integration” paradigm. The nature of this platform and its management challenged its business users faced with onboarding partners and needing visibility of all transactions.
Making informed choices means the energy company can transfer EDI and other files through the MFT platform, giving the business more agility to onboard partners’ integration with file transfers. The business teams also gained visibility and traceability of data passed to and from partners. The project also strengthened the overall integration standards as part of the enterprise architecture.
iPaaS is a substantial part of cloud integration, but needs to be synched with other activities, as we explore next.
3. How to complement and augment iPaaS
Seeing iPaaS as a key part of the whole picture and understanding how it fits in with other elements is key to its success. This includes timing, budget and allocating the right level of resources as well as engagement with your iPaaS partners to keep everything in synch.
Setting up a hybrid and multi-cloud architecture
Establishing the right architecture for hybrid cloud and multi-cloud scenarios is a critical success factor for working with AWS, Google Cloud Platform and Microsoft’s Azure. Multi-cloud management platforms – see this white paper – that provide a single point of control, administration and governance are key, and there are many tools that can help execute the appropriate architecture.
For instance, automation tooling for multi-cloud deployments accelerates and simplifies the implementation process. A good example is Deplomatic, an API-first, automation framework which can create and be deployed onto any cloud platform in a single click. Tooling including Ansible, Terraform and others can speed up development of infrastructure as a code.
AutomatonTM is a no-code tool that automates test for data interfaces, APIs and user interface components. It includes modules to connect to external data source and validate changes to data at the source level.
Here are some examples of how partnering with a specialist integration partner and leveraging its expertise with tools can bring benefits.
| No | Torry Harris tool | Area of Automation | Annual Savings |
| 1 | Deplomatic | Deployment automation framework that also provides environment governance and workflow capability. It drives cost efficiency through avoidance of overprovisioning of Cloud resources. | Has reduced environment governance cost for customers by 35% |
| 2 | Coupler | Microservices development framework through visual flow design approach. Ideal for Integrationcentric Microservices. | Reduces development effort up to 40% compared to a codebased approach for implementation. |
| 3 | AutoStub® | Stubbing Automation tool to automatically generate datadriven API stubs for test automation and facilitate CI/CD | Reduces development time by upto 20% |
| 4 | AutomatonTM | Test Automation tool that provides a visual tool to model your APIs and other application test scenarios | Reduces test automation effort by 30% |
| 5 | DigitMarketTM MicroGateway | Microgateway to securely expose your Microservices through a decentralized policy management | Reduces operational cost by 15% |
Some IT departments are not convinced about the need for separate API management when, after all, an iPaaS is API-driven, but they are complementary.
iPaaS enables an organization to exchange data between systems and expands connectivity across it to improve access to data. It supports the flow of data between on-premise and SaaS, data warehouses and lakes, IoT devices and other endpoints. APIs makes all this possible, exposing data from each source intelligently and securely, allowing customer relationship management systems to be integrated with enterprise resource planning, say.
Creating and managing APIs are not the same thing: Without API management, organizations risk data breaches, disrupted business processes, inefficient operations and failing to leverage the full value of their APIs. For example, a developer portal can help in API discovery, self-service and help set-up new digital revenue channels.
You can find more about API management here.
Transforming applications to a cloud-native model
Integration of new, cloud-based capabilities is essential. Otherwise, a digital platform can become just another box on the organizational chart instead of a transformational engine that powers your company.
Legacy applications are often what differentiates a company from its competitors. Modernising applications helps to re-engineer legacy and end-of-life applications. How to get more value out of legacy by transforming them to a cloud-native model includes:
- Combining and integrating them in new ways
- Making them more accessible to customers and partners
- Mixing them with new capabilities
- Making them more scalable.
Case study: UK telco benefits from far-sighted integration strategy
THIS first engaged with this customer about 20 years ago, contributing to its integration strategy, including the rationalization of services during mergers. This created a new brand and won iCMG’s award for the best SOA implementation in 2012.
THIS was engaged as part of a transformation project after the merger to deliver, manage and govern the digital services platform within the AWS public cloud. Its responsibilities included triaging and management of incidents, automating data import scripts to manage orders and extend product search through optimized configuration within the Hybris (now part of SAP) e-commerce solution.
The integration of Hybris and the Adobe Experience Manager (AEM) content management system improved interaction with customers. Setting up compatibility rules within Hybris also extended product management, and THIS also established change, incident and problem management.
Finally, the integration specialist devised a complete support model for the entire digital services team and integrated on-shore and off-shore resources into teams to minimize costs and maximize business contact and engagement.
Cloud-enabling the data platform
Cloud-enabling the data platform brings together two of the biggest potential forces for digital transformation, so it’s important to get it right through a sound strategy, deployment roadmap and constant monitoring of progress and goals – not least because the latter often changes in the rapidly moving digital world.
Data integration brings data from different systems together to increase its value to any business. Without data integration, there is no way to access the data gathered in one system from another, or to combine data sets so they can be processed, analyzed and acted on.
|
Volume
Companies generate immense amounts of data from sources including their networks, services and customers, as well as gather it from other sources like social media |
Variety
The data comes in many formats, is often incomplete, siloed, incompatible – for instance, structured and unstructured. |
Veracity
Data should be a ‘single source of truth’ – the cleansing needed to get it to that point is not necessarily simple. |
Velocity
We know the speed that data is produced at rises all the time. The question is but how fast a firm can act on it in real time or close to it. |
|
Value derived from integrating data to gain insights, improve CX, efficiency, profitability & TTM
|
|||
The aim of moving data to the cloud and intelligent integration is to access and apply the massive computational capabilities and storage capacity to data, affordably. This requires data warehousing, data lakes, data engineering, data science, data application development, and securely sharing and consuming data.
You can read much more about approaches to and benefits of cloud data integration for the enterprise and find more resources here.
3. In summary
It is hard to overstate the importance of a comprehensive and forward-looking cloud integration strategy and its intelligent execution. An extreme example of how market conditions can change, and the level of adaptability required is the Indian mobile market. THIS worked with one of India’s largest mobile service providers since 2014, originally helping it migrate from an IBM to an Oracle stack.
Since then, the country’s mobile market has shrunk from seven to three main mobile operators after Reliance Jio unleashed a ferocious price war upon its entry to the market in 2016. THIS is engaged in taking this customer’s vision to the next level, creating ways of enhancing value to help it continue to compete successfully.
300 million subscribers
World's Third Largest Telecom in terms of Customer Base
- Customer selected Torry Harris to transform their Integration into a SOA model from point to point Integration model.
- Wanted to launch new offers quickly (time to market)
- Wanted to standardise the data model to TMForum’s eTom model enable smoother BSS transformation in the future.
- Transition from large vendors. Customer wanted Agile and flexible partners.
Cloud integration is a rapidly evolving, multi-faceted discipline that is foundational to the success of migrating data, applications and IT systems to the cloud and wider digital transformation initiatives. The boom in running applications using SaaS has made iPaaS the preferred approach to cloud integration because multiple SaaSs have replicated traditional IT siloes. However, the rapidly expanding, fragmented iPaaS market is hard to navigate and a specialist integration partner is advisable.
iPaaS is key to cloud integration, but not the whole story. It needs to be addressed in parallel with hybrid and multi-cloud architectures, transforming applications to the cloud-native model and cloud enabling data platforms. Enterprises can take advantage of the range of tooling to accelerate and automate progress in them all, but again expert guidance is invaluable in choosing the right tools and gaining their benefits quickly.
