The pandemic put the boosters under digital transformation for most companies, whether they had already embarked on the digitalization journey or not. As soon as lockdowns began, it became apparent that the key to businesses’ survival was supporting a remote workforce to serve customers who couldn’t interact in the usual ways.
Yet while the importance of being digital and the need to invest in digital capabilities have never been more evident, most companies are still not where they want and need to be, digitally speaking. According to a Gartner report that pre-dates the COVID-19 outbreak, over 75% of businesses thought their DevOps activities were progressing too slowly, which means digital transformation efforts are also lagging because the two are interdependent.
In 2022 and beyond, making rapid changes to a business is synonymous with being able to make changes to software quickly. Yet changes to software often result in incompatibilities and other issues, making it harder to carry out further changes while keeping teams aligned around standard designs and architectures.
This alignment and consistency in hybrid integration are foundational to speed and replicable integrations and deployments, avoiding high degrees of customization, which requires manual effort and is often troublesome in the future, requiring retro-engineering. By hybrid integration, we mean the ability to connect applications, data, files, and business partners across cloud and on-premises systems –across personas, domains, endpoints, and deployment models.
This white paper looks at the top eight trends in DevOps for 2022 and how they can improve the use of DevOps to achieve business and operational goals:
- The benefits of no-code DevOps orchestration
- Shifting left makes a lot of things go right
- Managing the metadata
- The rise – or maybe ‘fall’ – of DevSecOps
- Better service through serverless computing
- Microservices will grow, but the brakes are still on
- Kubernetes – development versus ops
- AI will be a driver of DevOps
It then addresses how companies can leverage these trends to best effect their organizations.
The Gartner report that pre-dates the COVID-19 outbreak found over 75% of businesses thought their DevOps activities were progressing too slowly, which means digital transformation efforts are also lagging because the two are interdependent. Businesses’ rapid responses to changes are enabled by fast changes to software – which is largely why DevOps practices have been ever more widely adopted.
Yet DevOps is not reaching its full potential, as the report indicates, listing the top five issues responsible:
- Initiatives tend to kick off without an organization nailing down what business outcomes it wants to achieve;
- The necessary organizational changes are not properly managed – and tools don’t fix cultural issues;
- Lack of collaboration with ALL stakeholders so that activities are limited to infrastructure and operations teams;
- Attempting to do too much too fast – start small, build on success, outruns big bang approaches; and
- A disconnect between what can realistically be achieved and ill-informed expectations.
It is easy to see how events could have exacerbated all these factors since early 2020. Adjusting to remote working and new ways of serving customers were huge challenges in themselves without bringing about additional cultural change to accommodate new technologies and associated practices at the same time.
In addition, there is a big shortage of people with the right skills. While it’s true that tools for developers and to help with the adoption of the cloud are becoming more widely used, the number of tools is also proliferating and adding to the complexity.
All of which is why the trends outlined in the next section are likely to be the top ones in 2022 and beyond.
DevOps is accepted as an indispensable software development methodology that can bring benefits such as faster delivery, better quality, and higher customer satisfaction. No wonder that according to this survey, 83% of respondents implemented DevOps practices in 2021.
Further, the report revealed that highly evolved firms are more likely to have implemented extensive automation, and 90% of respondents with highly developed DevOps practices reported that their teams had automated most repetitive tasks.
Source: Puppet 2021 State of DevOps Report
So whether organizations are seeking to follow suit and follow those leaders with automating the most repetitive jobs or they are ready to move to the next phase of DevOps deployment, these trends are expected to shape DevOps going forward. Automated deployment is a major theme throughout to attain consistency, robustness, and speed for hybrid integration.
The benefits of no-code DevOps orchestration
Software delivery is complicated against the backdrop of rapid digitalization and the adoption of multi-cloud ecosystems. Engineers use a variety of tools for different stages of software’s development lifecycle:
It can be expensive and time-consuming for companies to connect and integrate all these tools and build and manage the software pipelines, from ideation to delivery of software. A developers’ time can be eaten up in writing scripts and glue code to deal with the range of tools, making the skills shortage more acute.
No-code DevOps orchestration enables DevOp teams and engineers to instantly provision and integrates their preferred constant integration/constant delivery and deployment (CI/CD) and DevOps tools. Using a common framework, they can also build scalable no-code declaratively, in minutes, suitable for a range of use cases.
Declarative programs describe their desired results without explicitly listing the commands or steps needed to reach that outcome. This means developers spend their time building core products that translate into faster market times instead of struggling with the tools and pipelines.
Shifting left makes a lot of things go right
Shifting both security and quality ‘left’ during development is not new – nor widely adopted. Shift-left is a practice designed to find and avoid faults early in the software delivery cycle through earlier testing.
It looks likely there will be a lot more shifting left shortly as the number of attacks on software soar. Using a DevOps orchestration framework to Integrate security, quality, approval thresholds, and gates into pipelines declaratively addresses several potentially problematic issues. They range from assuring compliance with appropriate access levels and approvals when deploying software to minimizing bugs and securing pipelines. The set of practices and tools for automating compliance checks is called ‘Compliance as a Code’
Security can be further strengthened by automating code along container and infrastructure scans and proactive threat and vulnerability checks by integrating the right security tools. Automating checks like unit and integration tests in pipelines reduce problems and outages after deployment, resulting in a better customer experience.
Finally, a shift-left approach can substantially improve collaboration between the engineering teams responsible for development, security, and quality.
Managing the metadata
Software delivery becomes more complex as applications, platforms, and cloud ecosystems multiply, which increase the risk of poor oversight and reporting – and DevOps teams are often running behind the bus regarding visibility and insights across such ecosystems.
No-code orchestration helps by enabling the abstraction, correlation, and contextualization of metadata and logs from the different tools to flag actionable insights for different personas. Real-time, unified logs are a game changer: They reduce troubleshooting efforts and time to resolution. The ability to trace and visualize transactions across different points in your integrated IT landscape is broadly called ‘System Observability’.
In addition, cross-function metrics and scorecards can assist in tracking and optimizing efficiencies, security, and compliance. At the same time, machine learning can generate predictive insights to highlight potential problems and prevent them or apply self-healing techniques.
The rise – or maybe ‘fall’– of DevSecOps
DevSecOps is defined by Gartner as “the integration of security into emerging agile IT and DevOps development as seamlessly and as transparently as possible. Ideally, this is done without reducing the agility or speed of developers or requiring them to leave their development toolchain environment.”
Many predict the rapid expansion of DevSecOps due to the rising threat levels as cloudification progresses and provides more opportunities for cyber attacks. The shift from DevOps to DevSecOps is expected to gain momentum in 2022, with more companies embedding security controls early in the software development lifecycle. This enables DevOps teams to continuously monitor and remediate security defects during development phases, improving the speed of delivery and quality.
Indeed, some pundits expect security to become a natural, integral part of DevOps throughout instead of a separate branch of the discipline, which makes good sense.
Better service through serverless computing
Serverless computing is not new, but it’s taken more than ten years for organizations to trust the notion of serverless frameworks due to worries about the lack of industry support and return on investment. The advantages of serverless are beginning to win them over, though, helped by the high CapEx and OpEx of server-based infrastructures.
According to AWS, “Serverless technologies feature automatic scaling, built-in high availability, and a pay-for-use billing model to increase agility and optimize costs. These technologies also eliminate infrastructure management tasks like capacity provisioning and patching, so you can focus on writing code that serves your customers.”
New products are now coming onto the market that support and simplify the adoption of serverless computing. Michelle Gienow, Technical Sourcer at Cockroach Labs, was quoted predicting, “Serverless will eventually become just an implementation detail, providing inherent scalability and reliability — and automating routine operational tasks that few developers enjoy.” She thinks this evolution will take a big step forward in 2022.
Microservices will grow, but the brakes are still on…
As serverless computing gains more widespread adoption, so will Microservices. The two are tightly interlinked, and both applications are broken into independent units to give large teams more flexibility and access to a range of tools. Microservices are essential building blocks of a Cloud Native model, which intrinsically weaves DevOps into the development lifecycle of Cloud Native applications.
When implemented successfully, Microservices offer enterprises greater scalability and agility than traditional applications: Developers can scale segments of a service to match business requirements instead of trying to scale the whole application. When something goes wrong, the use of Microservices means the fault is contained and does not affect the entire application. Chaos Engineering practices leverage automation in your Cloud Native environment to inject faults in your network, file system, operating system to ensure your Microservices and other components handle errors gracefully. These tools are built into your DevOps lifecycle and largely contribute towards SRE – Site Reliability Engineering.
However, poor implementation of Microservices can lead to all sorts of issues, as outlined here in the Muddles with Microservices section of this blog. For these reasons, the growth of Microservices is likely to be slower in smaller companies due to the perceived risks of losing data, rotten reliability, and security issues. This also presents small companies with an excellent opportunity to gain the advantages of being a first or early mover – see Section 2.
Kubernetes – development versus ops
Like Microservices, Kubernetes is integral to the evolution of serverless computing. The Microservices-based software is widely used for orchestrating containers, and its use has expanded across many sectors in the last five years or so. This year it is forecast to expand further into software development, as opposed to being primarily viewed as an operations mechanism for provisioning and managing cloud infrastructure.
AI will be a driver of DevOps
AI is expected to bring significant changes to the DevOps ecosystem through streamlining and speeding up every phase of the software development lifecycle – primarily through automation. This opens up a category of tools called ‘AI-Ops,’ which leverage AI and Machine Learning to detect patterns in system failures, attack vectors, service response anomalies, and many other metrics.
Automation reduces or removes the need for human intervention in processes from code changes to deployment, greatly relieving the DevOps teams’ burden through mechanisms such as a closed loop.
So what now? How do these trends help your organization?
Interesting as these top trends in DevOps are, the thing that really matters is how an organization leverages them to benefit its operations and business. Rapid, consistent, high-quality software delivery requires thoughtful, disciplined, intentional design of processes, tooling, teams, and organizational culture.
These six steps are essential - proper preparation up-front will pay off massively in the longer term in transforming your software delivery. Note that these steps are not necessarily sequential, so much as intrinsic to each other, so it’s like playing 3D chess in terms of complex interdependencies that might not be immediately obvious.
For this, the six steps should not be seen as a linear ‘tick list’ so much as an on-going circular refinement, mimicking what an organization undertaking software delivery automation is striving to achieve – continual improvement. The non-linear progression of these steps is also because organizations have to adapt to changing markets and other factors constantly – there should be no such thing as a ‘finished’ roadmap.
The on-going optimization of delivery automation involves continuous evolution of architecture, governance, tools and infrastructure, scripts and frameworks. Constant refactoring and retuning is a fact of life in automated software delivery, and staff should be well prepared for it. Alternatively, enterprises could address this using a managed service from a partner with a successful track record in providing such service.
Find out where are you now – and be honest
Every organization needs to accurately assess its current delivery environment before making decisions about the next steps. There is no substitute for having a precise and honest picture of the teams, tools, architecture, culture and governance already in place. Short cuts could lead to costly delays later instead of taking automated software delivery to the next level. The aim is to gain sustainable and increasing speed.
A partner with a track record in this kind of assessment can save organizations a lot of time and resources by providing tools such as assessment questionnaires refined by long practice – asking the right people the right questions is key to success.
Identify where improvements are needed
Be thorough in looking at where improvements are needed, whether team structures, tools, architecture patterns, key performance indicators (KPIs), service level agreements (SLAs), or governance policies and processes. Defining KPIs, SLAs, and other metrics is almost a science in itself, and calling on expert help is a good way of ensuring an organization covers all the bases while accelerating the processes of definition and implementation.
Automated deployment (CI/CD) can be applied to any software architecture. Still, it is most effective when combined with the modern modular approach to software design for cloud-native platforms and Microservices. If an organization lacks these skills or does not have them in sufficient numbers, it should look to an experienced partner that can accelerate progress from day one.
Don’t try to run before you know where you’re going
Every organization needs a roadmap to improve software delivery with DevOps. Still, no two are exactly the same because every single one has a different starting point, priorities, and ultimate goals. Even so, an experienced, expert partner can speed up and simplify the process with tried and tested assessment questionnaires and roadmap templates, along with a curated list of the most appropriate third-party DevOps tools and frameworks.
The aim is to create a roadmap for a staged evolution to rapid, controlled software delivery that will support accelerating rates of change in the business now and in the future. Again, a proven DevOps reference implementation can provide a very helpful guide.
The roadmap should not have technology as the main driver, but outcomes and must take into consideration people, skills, and organizational culture (see below). If you are changing the way people work, you need them to understand and support those changes and have the necessary skills and training to implement them.
Choose carefully, implement intelligently
Selecting and implementing initial delivery automation tools, practices and scripts is all about the best way to integrate CI/CD into the development lifecycle. Never lose sight of the fact that the aim is Agile development, with scripting, pipelines, and code repositories as major components.
Examples of tools that can reduce time and effort include Deplomatic, Automaton, and AutoStub from Torry Harris Integration Solutions (THIS), which leverage APIs. These interfaces are a pragmatic way for any organization to securely expose data in a controlled manner and underpin modern software architecture.
Deplomatic provides automation tooling for multi-cloud deployments to accelerate and simplify implementation. It is an API-first automation framework that can be created and deployed onto any cloud platform with a single click. It can be used with tooling, including Ansible, Terraform, and others, to speed up development of infrastructure as a code. It can reduce governance costs by 35%; ensuring deployments are less error-prone and more repeatable.
AutomatonTM is a multiple award-winning, no-code tool that automates test for data interfaces, APIs and user interface components, and all the other elements of an application. It can reduce test effort by 30%. Users can run tests without coding knowledge. It includes modules to connect to external data sources and validate changes to data at the source level.
AutoStub® can speed up API development by up to 20%. It reduces build time by designing, prototyping, documenting, and testing APIs using a functional ‘mock’ that allows developers to work with APIs before they are fully implemented.
Governance is an essential enabler
Governance is critical to the success of automating the delivery of software, but it is often overlooked instead of being regarded and implemented as a key enabler. Naturally, governance should be automated and built into delivery pipelines and processes.
Governance is often seen as something that potentially gums up the works rather than as a key enabler. Still, an experienced, expert partner can offer design patterns for Agile best practices for governance, along with a governance architecture and a bunch of tools to lighten the load while increasing effectiveness.
Compliance as a code is a highly valuable approach to automating governance. Different industry sectors must comply with specific regulations that vary from country to country or region to region. Healthcare is an obvious example, as well as general regulation and legislation, such as GDPR.
Expertise in chaos engineering is invaluable, as real life is unpredictable despite all best efforts. TechTarget describes it as “the process of testing a distributed computing system to ensure that it can withstand unexpected disruptions. It relies on concepts underlying chaos theory, which focus on random and unpredictable behavior.”
THIS offers 4Sight, an AIOps-based analytics tool to assist with governance, as well as DevOps scripting tools and frameworks in the interests of consistency and reusability, plus an API Conformance Engine. It also offers third-party tools like Chaos Monkey and continuous monitoring tools like Grafana Labs and Elastic Stack.
In 2022, accelerating, security, and automating the DevOps software delivery pipeline will be a priority for all organizations that intend to progress their digital transformation. As we have seen in this white paper, there are a host of technologies and methodologies – some new and others that are established but not yet widely adopted – to help them in their endeavors.
Yet success does not depend on the technologies themselves so much on how organizations harness them. This includes having a well-defined strategy built around how to move from where they are now to gaining the desired business outcomes through hybrid integration DevOps.
Each organization needs a comprehensible but flexible roadmap suited to its unique circumstances and priorities, supported robust overlooked governance, and great attention to evolving the culture so that the organization is ‘fit for the purpose’ of deriving full operational and business benefits of the DevOps approach.
All of this is, which needs to happen in parallel, is a lot to ask of any organization. Those that are smart and forward-thinking will recognize this and choose to work with an experienced, expert partner to help and advise them.