Hiring managers 😉, product designers, UX researchers, and journalists often share a common pain point: the daunting task of analyzing interviews. Despite the availability of tools like ChatGPT, this process remains challenging and susceptible to bias. The volume of customer interviews often overwhelms teams, making it difficult to extract meaningful insights.

To tackle this challenge head-on, I took the lead as the sole designer at powerUp to develop Customer Discovery AI. This groundbreaking tool uses artificial intelligence to streamline the analysis of extensive customer interview data. It generates comprehensive reports similar to those produced by seasoned researchers, all without requiring any additional effort.

Our primary objective was to identify a compelling value proposition. Remarkably, within just three months we transformed an unvalidated product into a validated minimum viable product (MVP) that attracted a growing number of active users.

In this case study, we’ll dive into a project where effective collaboration across various disciplines played a pivotal role in shaping product decisions and achieving outstanding results.

This is the story of Customer Discovery AI.

Upon joining powerUp, I immersed myself in their SaaS platform designed to help corporate clients streamline and manage their innovation data. However, we soon encountered a roadblock – our project thesis was unvalidated due to lack of traction.

During a post-mortem research, we made a crucial discovery that would impact future developments: while the legacy product held potential value for corporate managers, it imposed a significant burden on regular product team members. These individuals were required to manually format their data to align with a new workflow, without experiencing direct benefits in return. Consequently, the overall product became a tough sell.

In light of these insights, we made a strategic decision to pivot our efforts towards addressing fundamental challenges regular product team members often face at the beginning of the discovery process.

In January 2023, we began the process of pivoting the project. Proactively, we engaged in direct conversations with potential users and conducted 30+ discovery interviews. Initially, these interviews involved participants from our own network. Through this process, we gathered insights that led us to formulate the assumption that: "Product teams struggle to manage their research data."

To validate our assumption with a broader population and gain initial insights into the target audience's interest, we employed a guerrilla tactic by running Ad campaigns on LinkedIn. This approach revealed that the segment "Product teams" encompassed a wide range of professionals, making it challenging to address their specific pain points effectively.

To refine our customer segment, we cast a wider net using an automated CRM to reach out to potential users. This helped us capture saturation and surface trends, allowing us to narrow down the customer segment.

Notably, we observed that well-established companies with high Daily Active Users (DAU) had dedicated research teams responsible for data analysis. Conversely, in smaller startups, it was often the Product Managers who handled qualitative analysis.

Armed with these insights, I have made a strategic design decision to narrow our customer segment. By focusing on Product Managers at smaller startups, we identified a more specific group experiencing a more pressing pain point and the desire to automate parts of the process.

To further validate the problem faced by Product Managers at small startups, we implemented a proactive approach. Through an automated system, we connected with potential leads and invited them to engage in quick video calls. The exceptionally high response rate indicated that we were indeed addressing a genuine pain point.

To facilitate effective communication of the identified problem, I developed a prototype. This prototype aimed to tackle the pain point of Product Managers struggling with qualitative analysis. Our strategy of showcasing a prototype proved to be highly effective. Users actively engaged in conversations, expressing their desires for specific features and providing valuable reflections on their current work processes.

→ Link to the prototype

It's important to note that the initial purpose of this prototype was not to test a specific solution but rather to engage customers in reflecting on their pain points. This exercise allowed us to gain a deeper understanding of the problem at hand.

After analyzing the conducted interviews, our collaborative team, which includes the researcher, Product Owner (P.O.), and myself, extracted and prioritized key insights to inform the project's direction. We reached the following key conclusions:

01. Inefficiency of Qualitative Analysis Process: Users consistently expressed frustration with the slow and labor-intensive nature of qualitative analysis. The process relied heavily on human involvement, often taking days or weeks to complete.

02. Trust in AI and User Control: Trust in AI-generated insights was an important factor, and users emphasized the need for transparency and the ability to validate and customize the results.

03. Timing of Qualitative Research: Participants highlighted that qualitative research is typically conducted during the customer/problem discovery phase. This early-stage is the most resource intensive part of the research.

Most importantly, trough a series of 30 interviews, we discovered that 88% of the participants were interested in testing a MVP with their real data.

In my role as the product design specialist, I carefully assessed all available information and made the decision to build a hyper-focused MVP that would address a specific pain point.

→ Link to the prototype

Despite receiving numerous feature requests, the rationale behind this decision stemmed from the need to maintain focus and avoid feature creep, particularly during the early stages of product development. While those requested features could indeed enhance the product's maturity, our primary objective was to establish a strong foundation by addressing the core pain points. It's easy to lose sight of the main problem when customers express desires for specific features.

After presenting the documentation and reasoning behind the solution - backed by data - I obtained buy-in and invaluable collaboration from the team, allowing us to proceed with designing and crafting handovers for a private MVP centered around a single use case.

The primary objective of the MVP was to test the time-saving aspect of the value proposition and address a specific task: extracting insights from customer interviews. All the evidence we had gathered pointed to the significance of this pain point.

In order to expedite development, we made the decision to utilize only the Core UI components from the legacy platform. Although this presented a challenge due to the significant divergence in UX, I collaborated closely with the engineering team to overcome this obstacle and successfully deliver a viable MVP in less than a month.

To facilitate prompt testing without relying on engineers, we developed a collaborative prompt engineering interface for the team's use.

During the initial phase of 20 moderated tests, we identified some issues. Users found it hard to understand the "generate insight report" flow and desired more flexibility in their research goals.

To maintain momentum, we quickly iterated on the MVP using the ongoing flow of insights. By using our own tool, we were able to distill insights more easily from our interviews.

In this iteration, we addressed the flexibility issue by adding the ability to ask questions to one or all of the research files.

→ Link to the prototype

After conducting 40+ live demos, users perceived values such as time-saving, removal of bias in note-taking, and quick retrieval of interviewee responses. However, we also noted a list of unmet needs that require key improvements:

• Users expected insights to be grouped by research question or goal;
• Users wanted to turn qualitative data into quantitative insights for better product direction;
• Quotes, as evidence, were considered the most valuable part of the insights;

Our next step was to plan for the near future, taking into account these learnings from the MVP phase.

At this stage, we have successfully validated our customer segment and value proposition. However, we faced two pressing challenges: our current solution did not fully address all customer needs, and other AI analysis products were emerging in the market. These challenges further emphasized the need for a clear differentiator and intellectual property (IP) to establish our tool as a viable business.

To address these challenges and provide a guiding framework, I leveraged our previous findings to establish a short-term vision based on three key principles.

The aim of this vision was to align the team and empower them to pro-actively pursue solutions that would improve metrics related to those principles.

The primary issue with qualitative research was the significant time and resources required to conduct it. Therefore, our primary focus became saving people time. It was essential to ensure that overwhelmed project managers did not need to learn a new tool for a specific purpose. Instead, we aimed to integrate our functionality seamlessly into their existing transcript and documentation workflows, following a "no-touch approach." Each iteration of the MVP was designed to bring us one step closer to achieving this vision.

Project managers constantly faced the challenge of navigating multiple contexts and experiencing cognitive overload. Our goal was to alleviate this burden by reducing cognitive overload and enabling users to achieve better results more efficiently. By reducing the amount of manual information processing required to derive value, we aimed to streamline their workflows and improve their overall experience.

During customer interviews, trust emerged as a significant concern when it came to AI analysis. To address this, I designed features that allowed users to easily verify and trace back every output provided by our tool to its origin within their own data. By providing transparency and a clear audit trail, we aimed to instill trust and confidence in the insights generated by our AI-powered tool.

By adhering to these evidence-based principles of speed, de-clutter, and accountability, we established a strong foundation for our product's development. These principles not only guided our decision-making process but also served as a basis for ongoing validation and improvement, ensuring that our tool addressed critical user needs and stood out in the competitive landscape.

In order to address the identified pain points and leverage the evidence-based principles, I have developed a comprehensive "final" concept and created a high-fidelity prototype that effectively communicates the value proposition to potential customers and investors. To make our product stand out in the market, I have incorporated a brand component. Additionally, I have meticulously chosen typography and colors to enhance readability and declutter the interface, ensuring a visually appealing design.

→ Link to prototype

The key feature of the prototype focuses on speed by providing users with the ability to seamlessly integrate with various qualitative data providers such as Otter, Fireflies, and Zoom trough Zapier and IFTTT. This integration allows for efficient data collection and analysis, saving users valuable time and effort.

To further improve speed and streamline the workflow, I have implemented a syncing feature that enables users to set up research questions once. The system then automatically asks these questions to newly synced files, while the analysis is sent directly to their email. This approach eliminates the need for repetitive clicks and ensures that cumulative information is added to existing questions effortlessly.

To convert qualitative data into quantitative insights, I have introduced the use of quotes as an indication of prevalence. This empowers users to track recurring themes effectively. Additionally, a sentiment analysis feature has been implemented to aid in scanning and comprehending the overall sentiment expressed in the data.

By crafting and overviewing the implementation of these designs and considering various disciplines throughout the project, I have successfully influenced product decisions and achieved exceptional results. Despite being involved in multiple aspects of the project, I have consistently delivered high-quality design handovers that excel both visually and functionally.

Within just one month after launch, we achieved 200 onboarded unique users, all without a marketing budget. Despite not implementing the new improvements described in our vision, our product demonstrated its value and garnered a 10% retention rate at the five-week mark. This success provided us with substantial evidence to validate our value proposition and gather initial feedback on users' willingness to pay.

Personally, this experience was incredibly fulfilling as it allowed me to connect with users, contribute to the development of a product that effectively addresses real pain points, and gain valuable insights on how to pivot and refine our approach. The project showcased the power of collaboration and the impact that a multidisciplinary team can have in creating a high-quality solution. I am proud of the outcomes achieved and the valuable lessons learned throughout this process.

Thanks for reading,

Tutu