Over the past couple of months, we’ve discussed the metrics used to ascertain application quality, software release cycles, and how to choose the right metrics. In this month’s blog, we will take a closer look at how to measure customer satisfaction and whether you’re writing the right software.

If you’re writing and deploying software fast, but customers are unhappy, you may need to pay attention to these four metrics.

Here’s a quick reference chart:

Number of Customer Tickets

Customers will submit tickets for a variety of reasons, including:

• Bugs in the software
• A mismatch between their mental model and the designers/developers’ implementation

For tickets that report bugs, ensure that your QA process is robust. Increase your test coverage to prevent similar bugs from being deployed to production. This may increase your software development cycle time, or you may have been testing the wrong sections of code for a specific release type.

For tickets that uncover a mismatch in how the users expect your software to work and how it actually works: review your requirements gathering to ensure your developers are creating the right software and test for usability. Include test cases and testers with different backgrounds to understand better how your products will be used in the wild.

App/System Availability

You may have created an application that everyone wants to use, but if it keeps crashing, they will abandon it quickly. Test your production infrastructure and your deployment processes to ensure that downtime isn’t preventing your customers from using your application.

Service Level Agreements (SLAs)

Service Level Agreements set expectations between you and your customer on everything from application uptime, ticket resolution times, and define incident severities. Paying penalties for failing to meet customer SLAs can cut into your profitability, but it also erodes customer trust in your product.

The customer has agreed to a specific set of SLAs because they believe those criteria are essential, and it reassures them that using your application isn’t risky. If you’re not meeting those SLAs, examine whether you have written the right software requirements. In some cases, the customer may have chosen the wrong metrics for the application’s purpose and might be willing to negotiate different SLAs as long as their needs are being met.

Customer Usage

You’ve successfully deployed a new feature, it’s always available, and you’re not receiving many customer tickets. It’s time to celebrate, right? Not quite — if it turns out the low number of customer tickets is because no one is using your software.

There are several reasons why users may not be using your new software feature: lack of awareness, preference for another product, bad initial experience, or they simply don’t need that feature.

Measure how often features are being used and by what user profile to better understand where to invest future development. If users are aware of a feature and do not find it useful, it might be a candidate to trim in a future release.

Summary

Track the four customer experience metrics above to ensure your investment in DevOps results in software that your users want to use. Metrics like customer usage help inform which features to abandon or further develop while monitoring customer tickets help improve your design and QA processes.