Blog
/
Developer Productivity
/
How to measure developer experience (with metrics!)

How to measure developer experience (with metrics!)

Alyssa Towns
Writer
July 12, 2022
Updated on:

How to measure developer experience (with metrics!)
Photo by 

Developers are the powerhouses and necessary backbones of technology and software companies. Their technical skills and abilities allow businesses to create products to sell to customers; without them, organizations may not be able to fulfill their revenue goals successfully. Since they’re so crucial, it’s essential to provide developers with the best experience possible so they can stay motivated, focused, and most importantly, so teams can retain them. That’s where developer experience metrics and measurements can help. 

But first, what is developer experience (DX)? Similar to user experience (UX), this growing topic in DevOps refers to how working with a particular technical product makes developers feel when they interact or work with it — or the UX for programmers, essentially. This could look like a developer interacting with an API or SDK as a consumer or an internal “customer” developing your software product. DX emphasizes developers as the end-users and focuses on the sum of positive and negative feelings they experience. 

Good DX boosts efficiency, enables innovation, and keeps developers happy and committed to their work, allowing them to deliver business value. Some organizations have DX teams who prioritize and focus on improving the experience for programmers.

In this article, we’ll give you an overview of:

  • Metrics for developer efficiency 
  • Measuring developer experience
  • KPIs to consider 
  • Tips for maximizing developer experience

Let’s dive in!

3 developer efficiency metrics

There’s a significant difference between getting things done and doing things efficiently. Developers have a lot of work on their plates, and in order to do their best work and spend time on the most critical tasks, they need to use their time effectively. There’s no one-size-fits-all approach to measuring developer productivity, but there are some metrics you can use to understand developer output across teams and as individuals. Metrics like hours worked, lines of code and numbers of bugs fixed don’t necessarily account for quality and efficient practices. Instead, consider the following key metrics for tracking developer efficiency: 

Lead time 

In development, lead time is the amount of time it takes from the beginning of a project through to its completion and delivery. Understanding your team’s lead time is helpful in providing accurate turnaround time estimates. Additionally, if lead time improves and quality remains high, it may speak to the fact that your team is working more efficiently. Or it could indicate that other productivity improvements positively impacted the team’s efforts.

Automation

According to GitLab, automation and collaboration are key for operational efficiencies and accelerating deliveries. Great DX includes the right architecture and tools to support it, including automation where possible. Highly-skilled developers shouldn’t have to waste their time and energy on mundane tasks and instead should leverage automation in their workflows so they can spend more time focusing on the development process. Consider measuring the number of automated processes implemented in a given period.

Velocity

Teams use developer velocity to understand how much work a developer can complete in a set period. In many instances, it’s the sum of total activities and efforts required to deliver a final feature or product. Measuring velocity can help teams identify common barriers consuming their time, including lack of continuous integration and deployment, context switching, distractions, and gaps in processes. Use velocity metrics to identify areas of opportunity for improvement. 

3 measurements of developer experience 

Like UX, the Harness team recommends using the following three pillars to measure developer experience. The measures below offer a view into developers’ daily experiences.

Findability 

Findability is the ease with which software engineers can locate information, systems, or tools they need to do their work. When information is easily findable, it’s easy to locate or identify. Creating a good DX involves making essential information easy to find to provide the best user experience. This includes providing logical and contextual search structures and functionality, organized documentation, and unrestricted access to tools as needed. Pay attention to developers’ feedback and notice any recurring complaints or frustrations related to findability. You can use the feedback to improve the team’s processes.

Usability 

Usability testing refers to how easy it is for developers to use code, documentation, or other tools and products related to their work. Gather feedback through an internal team survey to better understand how easy or difficult it is for developers to use these elements. Example questions to ask to gauge usability include:

  • Is it easy for you to use the codebase?
  • Do our tools provide the support you need to complete your work? 
  • Do the APIs offer you the information you need?
  • Are you comfortable with our programming language or framework? 

Credibility 

The credibility pillar refers to whether your developers trust the product and its benefits. In development, organizations can build credibility with developers by creating a stable and reliable service or product, providing adequate resources to reduce burnout, and supporting tools and resources to get the job done. It’s important to note that high output and good results don’t always mean there’s a sense of trust with developers, particularly if their work environment is wearing them down, retention is high, or they’re paying considerable technical debts.

Developer experience KPIs

There are tons of KPIs to choose from related to developer experience, and it’s important to identify KPIs that fit your business and will resonate with your software development team. Below are some examples for employers with development teams:

  • Employee turnover and retention rates
  • Employee engagement
  • Sprint and release burnout 
  • Cycle time 
  • Code churn

And a few examples in the context of developers as consumers (accessing APIs and more):

  • Number of developers that land on the development portal for API access 
  • Click-through and bounce rates 
  • Adoption and usage of tools 
  • Pull requests on open source projects
  • Developer satisfaction 

How to maximize developer experience

You can implement these tips immediately to maximize the developer experience in no time. 

Foster cultures of psychological safety 

Creating a psychologically safe environment is a cornerstone of productive teams, including development teams. The must-have requirements for psychological safety are transparency, communication, equal talk time, and room for mistakes. These elements create environments where engineering teams can iterate, innovate, motivate one another, and keep work-related stress at bay. These environments allow developers to share ideas, raise concerns, expose defects without fear, and feel empowered to take risks.

Build or encourage joining developer communities 

Developer communities provide spaces for developers to share their experiences, failures, successes, and achievements with others in the industry. Sharing in a community format allows developers to teach and learn from one another, inspires action, and sparks collaboration and innovation. Other benefits include providing a knowledge repository, quicker problem-solving, more eyes on bugs, and professional growth and development. Consider building an internal developer community for your team, or encourage them to join one of the many existing communities available, including GitHub, Stack Overflow, or Hashnode.

Use the SPACE framework to paint a complete picture 

The subject of developer experience is complex, and a solid understanding of the developers’ journey requires more than examining a small handful of metrics. To provide the best experience, look at multiple factors and how they play into the entire journey. The SPACE framework combines many metrics to understand better and influence developer productivity. The framework is as follows:

  • Satisfaction and well-being: Refers to how fulfilled developers feel and how happy and healthy they are. To understand developers’ satisfaction, consider measuring employee satisfaction, developer efficacy, and burnout.
  • Performance: Refers to the outcome of a system of processes, not the output. High output doesn’t always equate to quality outcomes. Metrics in this category include reliability, absence of bugs, customer satisfaction, adoption and retention, and feature usage.
  • Activity: Developer activity is a count of actions or outputs — in this case, activities developers perform to complete their work. Some examples of developer activities to consider measuring include a count of design documents, number of pull requests, count of deployment and releases, and number of incidences addressed.
  • Communication and collaboration: Describes how development teams work together and support each other in their work. Measurements might include onboarding time for new team members and the discoverability of essential documentation.
  • Efficiency and flow: The ability to complete work with minimal (or without) interruptions and delays — or how much Focus Time developers have to get their job done. Measurements in this category might include the total number of interruptions and how they impact development, the number of handoffs in a process necessary to complete a task, and the number of weekly Focus Time hours or time spent writing code.

Development managers can use the SPACE framework to measure teams and individuals to understand productivity and developer experience more broadly.

Implement a tool to stay on top of developer experience

Various engineering management systems are available to help you stay on top of and constantly improve your developer experience. Tools like WayDev, LinearB, and Haystack track developer metrics that you can use to better understand output alongside the developer experience. 

Going forward

DX refers to how developers feel when interacting with code, either as an internal customer (an employee) or a technical user through an API or SDK (a consumer). Great DX optimizes developers’ workflows, work environments, and more, so they can have a positive experience. Examining lead time, automation, and velocity can help shed insight on developer productivity. Similar to UX, findability, usability, and credibility are measurements you can use to understand the developer experience. You can leverage many KPIs to improve your developer experience for both internal developers and external consumers. Psychological safety, involvement in developer communities, holistic frameworks such as SPACE, and engineering management systems can help maximize your developer experience. 

Read next: 1. How to Build and Hire a DevEx team 2. What is a time orchestration platform?

About the author

Alyssa Towns

Alyssa Towns is a freelance writer for Clockwise based in Denver, CO. She works in communications and change management. She primarily writes productivity and career-adjacent content and has bylines in G2, The Everygirl, Insider, and other publications. When she isn't writing, Alyssa enjoys trying new restaurants with her husband, playing with her Bengal cats, adventuring outdoors, or reading a book from her TBR list.

Optimize your work day with AI powered calendar automation.

Sign up for free

Make your schedule work for you

More from Clockwise