DevOps can help improve business culture, speed up time to market, reduce errors, and more. This year make sure you’re tracking key performance indicators (KPIs) to demonstrate how DevOps is improving business outcomes and where additional opportunities to eliminate bottlenecks exist.

Determine Your Baseline

Before implementing a new tool or process, first understand how well your development and deployment processes are going today. You can identify bottlenecks with a value stream map. Take this time to document how long development and deployment processes are taking, how many defects are detected before and after deployment, and whether the app’s uptime and performance are meeting expectations. Collecting this baseline now allows you to compare with KPIs later to see if the change achieved the desired outcome.

If you have existing KPIs that your organization is measuring, you might continue to collect those before and after making changes to your development and deployment processes.

"Value" Metrics vs "Counting" Metrics

In a 2019 Forrester Report on DevOps Quality Metrics That Matter, Forrester interviewed over 600 enterprise leaders to understand what metrics mature DevOps practices measured compared to lower performing practices. Although the report reviews 75 different DevOps metrics, not all KPIs provide value.

As an example, the report shows the difference between measuring the total number of tests executed versus measuring the test case coverage. The first metric is a "counting" metric and can be easily manipulated by adding redundant passing tests focused on a narrow feature set, leaving large portion of the application untested. The second metric measures how well tests cover the application’s functional requirements. This is a far more valuable KPI to measure if your goal is to reduce defects in releases and improve user experience.

DevOps KPIs to Consider

Over the next few months we will take a closer look at specific KPIs across each phase of the software development lifecycle (SDLC). We work with customers who adopt DevOps and DevOps tools to reduce software release cycles, improve application quality through automation, and save both time and effort.

A Word of Caution

Choosing the right number and type of metrics to measure is a balancing act between actionable data and overload. Steer clear of vanity metrics used to make teams appear successful but do not provide meaningful, actionable data to continue improving. At best these numbers can be distracting, at worst they can mask real issues and cause teams to miss opportunities.

Meanwhile, reporting on and sending alerts for too many metrics causes "alert fatigue." If your team receives a constant barrage of alerts, they will start ignoring even the more critical ones. Reporting on too few events means a longer time (and more lost work effort) before course-correcting an issue.

Finally, do not use metrics to either reward or punish employees. You’ve recruited intelligent people and they will learn to game the system in their favor and to avoid blame.

At ReleaseTEAM, we know people, processes, and tools work together in a successful DevOps team. Contact us to find out how we can work with your people to establish processes, implement the right tools, and identify which KPIs to measure for your objectives.