Use ITSM to Measure DevOps Success

In this blog, we look at how organizations with an existing ITSM reporting capability can use it to support the adoption of DevOps practices.

The reality for many companies is that they will have an IT operations or IT service management (ITSM) organization in place at the same time as they embark upon new DevOps approaches for product development and delivery to customers. Companies in this situation are reporting that while DevOps and ITSM clash in some ways, they can also complement each other too.

One way to benefit from their complementary aspects is to use the company’s existing ITSM ecosystem to “shine a light” on DevOps performance. And as DevOps is focused on delivering products as a flow-oriented value stream, the logical starting point in ITSM is the change and release management processes.

Why Shine an ITSM Light on DevOps?

Seasoned ITSM professionals might shudder when they hear people say things such as “failure is acceptable” or “continuously release software every eleven seconds into production” – it goes against what is seen as 'traditional' ITSM thinking. It then gets worse when they perceive these dangerous phrases to come from a new “cultural movement” that is, in essence, a dangerous black box to them; a threat to the ITSM status quo and its process-based controls.

This new situation is an IT reality that’s not going to go away though. These dialogues are happening in enterprises all around the world and it’s often made worse when played out in what Dr. Ron Westrum calls pathological or bureaucratic organizations:

• Pathological organizations are characterized by large amounts of fear and threat. People often hoard information, withhold it for political reasons, or distort it to make themselves look better
• Bureaucratic organizations protect departments. Those in the department want to maintain their ‘‘turf,’’ insist on their own rules, and generally do things “by the book” – their book

Both of these organization types will be slow to realize the benefits of, and to adopt, DevOps across both app dev and IT operations.

However, linking up metrics will effectively shine a light into the DevOps black box so that ITSM people can see that DevOps is not as dangerous as first thought. And, that it’s actually beneficial and possible to collaborate, and to change the organization to be high performing and generative. Generative organizations focus on the mission – “How do we accomplish our goal?” – and everything is subordinated to good performance; to doing what is supposed to be done.

ITSM Vs. DevOps Metrics

ITSM and DevOps both use a wealth of KPIs and metrics, but you can’t boil the ocean from day one. So instead, it’s good to be focused, with the change and release (management) processes a good place to start. These ITSM activities are close to DevOps activities whereas service level management and SLAs, say, will be reliant upon so much more than just DevOps processes.

These metrics flow both ways between ITSM and DevOps:

1. The DevOps process can get ticket numbers from the ITSM change system and update tickets automatically from engines such as the continuous integration system
2. The ITSM processes are a source of information for the DevOps process. For instance, a change ticket is approved, which triggers the continuous integration system

Lighting Up Change Metrics

DevOps is a delivery and flow-oriented approach that uses practices such as continuous delivery to build a very robust change and release system. Within ITSM, the change management process wants to be given confidence that changes will be successful in terms of implementing new features while not causing outages.

So, it’s important to start with a few change metrics and to make sure that the automated links between DevOps and ITSM systems work, e.g. the continuous integration server with the ITSM ticketing system.

After these initial integrated metrics are set up, it’s also possible to link up other parts of the DevOps value stream such as user interface testing and customer A/B testing. By integrating these with the ITSM change process more insights become available. As with anything in IT in 2016, this should be a simple and automated process where possible.

Lighting Up Release Metrics

There will be multiple software releases per day in a DevOps environment. The releases can be triggered by anyone in the system, be that a developer, a tester, or someone in operations. It’s a single release process that has a number of automated steps that, over time, becomes very robust and trustworthy.

Each step in the release process is an attempt to prove that the software works and is of the right quality. And each of the steps can produce metrics that can be pushed into the ITSM world to provide insights into system performance, especially to help improve quality and performance over time.

There are of course many more release metrics available and it’s possible to be overwhelmed by them. Therefore, the DevOps and ITSM teams should work together to identify the leading and lagging indicators that best help them to address current issues without creating too many reports.

The Outcomes of Linking ITSM and DevOps Metrics

There are three measurable outcomes from linking metrics that have been reported in DevOps research, indicating that an organization is high performing (when these metrics show such levels of success):

• Agility metrics: 30x more frequent code deployments and 8000x shorter code deployment lead time
• Reliability metrics: 2x the change success rate and 12x faster mean time to repair (MTTR)
• Organizational performance metrics: 2x more likely to exceed productivity, market share, and profitability goals; and 50% higher market capitalization growth over three years

These high performance indicators aside, linking DevOps and ITSM metrics allows organizations to overcome internal fears and conflict, to see the benefit of DevOps, and to start working as a generative organization towards a common goal.