In the realm of public service, success hinges not only on the delivery of programs and services but also on the outcomes they produce for those we serve. As customer experience (CX) practitioners, we can measure both performance and outcomes to get a fuller picture of our impact. We have an opportunity to weave together three types of customer experience data: user research, customer feedback, and operational data. This article focuses on the power of operational metrics to tell us more about the customer experience.
Measuring Experience
As people working in customer experience, how do we show the outcomes of our work? Measurement reveals the effectiveness of designs in achieving desired outcomes, whether that’s for our customers or operations. In the field of CX, it’s common to see metrics like satisfaction that focus on customer perception. However, satisfaction metrics can only go so far in telling us what customers are experiencing. Operational metrics can ground us in another source of insight on customer experience and help create standards for service performance.
Operational metrics provide valuable insights into the quantitative aspects of program and product performance. These metrics encompass factors such as response times, resource utilization, and compliance metrics. For example, the DHS CX Metrics Dashboard invited components to share metrics around wait and service processing times (“time to next step or decision”). Another example metric is the volume of requests for information or participation, which can point to demand or utilization.
Operational metrics also have their limits. They can be crucial to measuring efficiency and effectiveness but on their own they may not offer a nuanced understanding of how end-users interact with DHS policies, programs, and products. What’s more, they don’t necessarily measure what matters most to customers. The only way to determine this is through customer (user) research. Metrics are most powerful when the decision of what to measure stems from a strategic vision built on customer understanding.
Operational metrics are most valuable when paired with a deep, qualitative understanding of the customer experience – such as situations, motivations, interactions, and sentiments. Qualitative research on the customer experience can reveal areas of frustration and opportunities to improve. Qualitative research can also help show the stories behind the data – giving an idea of the “why” behind patterns. By incorporating this knowledge, we gain a deeper understanding of how policies, programs, and products are perceived and used by our intended audience. This allows us to identify the right metrics that reflect users' actual experience and measure against our desired outcomes and goals.
Measuring Design Outcomes
How do we know whether we are measuring the right thing? While measuring performance is important, for metrics to be meaningful they must also address design outcomes. Design outcomes refer to the intended goals and objectives of our end products. For example, the intended result of simplifying an administrative form is to reduce the overall administrative burden. Depending on what customer research indicates, the measurable design outcomes may include the time it takes to complete the form, reduction in the error rate, eliminating rework, and reducing overall processing time. The fundamental idea is to measure the things that actually matter based on what we know of our customers’ experiences.
Measuring design outcomes offers several key benefits:
- Comprehensive performance insights: We gain a comprehensive understanding of how programs and products are performing from both a quantitative and qualitative perspective, painting a fuller picture of the customer experience.
- Human-centered decision making: This holistic understanding allows us to prioritize end-users' needs and preferences, ensuring that programs and products meet those needs and expectations.
- Identification of improvement opportunities: Holistic measurement enables us to identify areas for improvement and optimization, leading to enhanced performance, efficiency, and effectiveness over time.
- Enhanced accountability and transparency: We can demonstrate accountability and transparency in delivering value to stakeholders and, most importantly, to those who use or are affected by our policies, products, and programs.
Establishing a Holistic Measurement Strategy
To effectively identify and use experiential metrics to measure and shape design outcomes, follow these key steps:
- Understand organizational goals: What are the desired outcomes of your policy, program, or product? How will it affect people's lives if you do the best job of designing and delivering your end product? Questions like these may help you define your organizational goals and objectives. These could include improving compliance, increasing product adoption, reducing administrative burden, or enhancing trust.
- Conduct research and identify user journeys: Conduct qualitative research to understand the customer experience of interacting with existing services. What experiences do your customers expect of you? Check your assumptions: Are the outcomes that you have identified things that customers desire for themselves? Identify the critical pain points, frustrations, and satisfactions within user journeys. This qualitative data will help form a strategy and identify potential metrics that reflect user experiences.
- Set a strategy and vision: Based on the research you do with customers and stakeholders, form an experience strategy and a vision for what you hope to achieve. Where along the customer journey do you want to lift people from frustration to relief, or even delight? If you combine the improvements you are hoping to make to customer experience, what is the end-state vision?
- Select appropriate metrics: CX metrics should come from a deep understanding of one's customer. Base your metrics on your vision of how you seek to improve the customer experience – grounded in user research. Use metrics to roadmap your way to that vision, with progress metrics to inform and drive outcomes along the way.
- Use your strategy, vision, and customer understanding to establish an outcomes-based framework of measurement. Identify key metrics and operational data that directly measure against the customer-focused outcomes and organizational goals you have established. These metrics should be actionable, relevant to user needs, and reflective of your organizational goals and desired outcomes.
- Example operational metrics include time to complete a task or process, as well as rates of error, completion, and compliance. These can be paired with customer feedback data such as satisfaction or trust scores. User research, including usability testing, can help to observe tangible change over time. Remember, measurement at its core is the observation of change. It doesn’t necessarily have to come from large volumes of data or an analytics tool.
- Different metrics serve different purposes. Success metrics will let your organization know when you have achieved the vision. Progress metrics mark incremental wins and let you know you are heading in the right direction. Problem-value metrics show the cost of not acting on customer experience and can be articulated in units of time or money, for example.
- Analyze data holistically: Evaluate your design based on your identified metrics that genuinely reflect the experience and measure against desired outcomes. This holistic approach will yield a comprehensive understanding of performance and identify the right areas for improvement.
- Iterate and adapt: Continuously iterate and adapt based on insights gained from analyzing a holistic metrics strategy, driving continuous improvement and innovation.
It can be helpful to consider the role of metrics at the beginning of a project and without predetermining the metrics themselves. Establishing a base of understanding people’s experiences will help you set your strategy and measurement focus. By combining quantitative and qualitative insights, we can make informed decisions, prioritize user needs, and drive continuous improvement to better serve the public, our employees, and our partners.