Building the Customer Feedback Repository at UserTesting

Summary:

This case study will follow how I developed the Customer Feedback Repository and Overview Dashboard at UserTesting, which empowered Product and Engineering teams to efficiently manage user feedback, gauge user experience, and prioritize critical user issues. This was achieved through:

  • Compiling all of the customer feedback into a centralized library

  • Reviewing and refining over 1000 feedback tickets to ensure they were clear and actionable

  • Introducing KPIs to measure each customer feedback’s impact and frustration level

  • Organizing all the customer feedback by product team and identifying duplicate or blockage-causing frustrations

With the standardized process I established for managing customer feedback and the implementation of new metrics, teams were able to consistently update the feedback library with the latest user insights, allowing them to track the evolution of the user experience over time.

This case study will focus on the issues caused by UserTesting’s lack of a customer feedback repository, the process of building the repository in Jira, and the impact it had on enhancing the overall user experience.

UserTesting is a company that aims to empower other organizations to empathize with their customers and build amazing experiences through UserTesting’s Human Insight Platform. I had the opportunity to work with UserTesting’s co-founders, Darrell Benatar and Dave Garr, in developing the Customer Feedback Repository. This repository was designed to empower product teams by providing a centralized library to identify and measure the current user experience, align on and prioritize impactful issues, and track the evolution of the user experience over time. 

The Problem Space - Managing Customer Feedback Without a Standardized Process or Metric

UserTesting managed its customer feedback through Jira ServiceDesk (Jira), where users would report issues or express frustrations encountered while using the product. This feedback was typically reported to support agents via conversations or emails, which were then entered into Jira through Zendesk. Additionally, UserTesting's internal team members had the capability to provide feedback and comments via an internal feedback page, which also fed into Jira's system.

In theory, all customer feedback was documented in Jira as tickets, providing product teams with a unified view of the user experience landscape and guiding them in prioritizing their tasks effectively.

Additional Context - UserTesting’s Users vs. Customers:

  • UserTesting’s customer base are companies that subscribe to the platform to conduct studies, the main users being researchers, designers, or product managers

  • UserTesting’s user base encompasses its customers (companies), test participants (general consumers), and internal employees

Second, not all customer feedback found its way into Jira. For instance, UserTesting conducted benchmark studies and received feedback reports from customers. While Product Managers and their teams had access to these reports, the findings and customer feedback were often stored in isolated folders or spreadsheets, never making its way into Jira.

However, there were two major issues with how UserTesting gathered feedback:

First, there was no standardized procedure for UserTesting agents to submit customer feedback tickets. The majority of tickets lacked clear, actionable feedback from users, often resembling the following example:

These issues made it challenging and time-consuming for stakeholders to align on the current user experience, measure the experience effectively (a form of "debt" to UserTesting's customers), and manage and prioritize users’ issues.

Building the Repository

Given that Jira already provided the infrastructure for the library, to first build the library, I had a 3 step approach: consolidate and clean all the customer feedback that UserTesting had gathered into the repository. Next I assigned relevant metrics, and finished by organizing these entries by their respective product teams. 

Step 1: Consolidating and cleaning

Gathering all of the customer feedback involved working with product managers, researchers, support agents, and any other relevant stakeholders to compile any ad hoc studies or customer reports. 

Each ticket’s summary and description required standardization for clarity, actionability, and traceability. To accomplish this, I meticulously reviewed and refined over 1,000 customer feedback reports and tickets to enhance comprehension and usability. 

Through my work, I gained insights into the lifecycle of the customer feedback tickets, including their submission process and adoption by Product Managers into their backlog. This allowed me to construct a model showcasing the journey of these tickets from inception to resolution, highlighting process gaps and identifying pain points in using the tickets.

Step 2: Assigning KPI metrics

In order to be aligned on the current user experience, we needed metrics for the product. I collaborated with the leadership team to design and implement a metric to measure each ticket’s impact level and Customer Frustration Index that was accepted across the Product organization. This allowed for consistent grading of all tickets and future customer feedback submissions. 

To maintain consistency across the board, I reviewed and updated the metrics for all 1000+ tickets.

Step 3: Organizing the tickets

The final step was to organize the customer feedback tickets by identifying duplicates, related tickets, or those causing blockages, and assigning them to their appropriate product team. 

To achieve this, I needed a strong understanding of how the Insights Platform worked, team structures, and interrelations within the product ecosystem. This understanding was vital for efficiently categorizing and managing the numerous tickets, especially given the frequent duplicates and interconnected ticket dependencies.

At this point, the repository was up and running, but it wasn’t actionable yet.

Making the Repository Actionable

The repository had been built, but it was still a dense word wall for Product Managers to decipher what should be prioritized first. 

Using my mixed-methodology research skills and understanding of customers’ feedback, I weaved quant and qual data together to build a Customer Feedback Dashboard to showcase a comprehensive user journey and experience map of the Insights Platform. 

  • The Customer Feedback Dashboard also displayed the overall Customer Frustration Level and prioritized user issues impacting the experience. 

  • Teams could also filter the Dashboard to focus on their specific product areas

With my initial design, Dave and Darrell developed it into an MVP Dashboard that Product teams could use. 

Impact

With the repository established, a standardized process for submitting new customer feedback, and the creation of the dashboard, there was now a reliable source of truth that all business stakeholders could refer to. The Product teams could now be aligned on how the Insights Platform is performing by the KPIs we had implemented and prioritize which features should be worked on first. 

As a result, the Repository and the Dashboard enhanced the product and engineering teams’ ability to measure and showcase the impact of their work. Teams could track the evolution of the user experience by addressing each issue or frustration and continually update the library with the latest user feedback.

Final Remarks

This project was well received across the company - imagine my surprise when other departments aside from the Product and Engineering teams began reaching out to me to thank me for my work! It turned out that this task was something everyone desperately wanted to get done but no one had the time for. 

Finally, I’m still “shook” that I had the amazing opportunity to work with the co-founders of UserTesting. By simply being in meetings with them or putting our heads together to tackle a problem, I was able to observe and learn how a good leader inspires his or her team, as well as how they break down and conquer challenges. 

UserTesting’s bubbly work culture and admirable work discipline is a force to be reckoned with. Although my project had come to an end, I know my journey with this amazing team is not over and I look forward to when our paths will cross again!