In a recent post we outlined the early stages of our user research and evaluation project with Jobs for the Future (JFF) and the International Rescue Committee (IRC). Building on that, we also shared a post on the principles that drive our approach to user research.

Now, we offer a comprehensive guide for those who are embarking on user research for the first time, or those looking to refine their current practices.*

1. Define the Scope 🔍

Every successful project starts with a clear definition of its scope. This involves establishing the “Who, What, When, Where, Why, and How” before taking any further steps. By doing so, we create a shared understanding with our clients, which is vital for ensuring that everyone is on the same page. This clarity enables us to guide the project effectively, making informed decisions at every stage. Without a well-defined scope, projects can easily lose focus, leading to wasted time and resources.

2. Identify Stakeholder Groups 👫

Identifying the relevant stakeholders is the next crucial step. Stakeholders are those who have an interest in the outcome of the project or who may be affected by it. For our JFF/IRC project, we concentrated on engaging employers, IRC staff, and, where possible, IRC clients. Each group brought unique insights that enriched our understanding of the Job Readiness Credential’s effectiveness. Engaging a broad spectrum of stakeholders ensures that the research reflects a wide range of experiences and perspectives, making the findings more robust and actionable.

3. Create Surveys (Quantitative Data) 📊

Once stakeholder groups are identified, we design surveys tailored to each group to gather quantitative data. This data is essential for identifying trends and patterns that might not be immediately obvious. For example, in our work on this project, we developed a survey for employers that asked them to rate various aspects of the Job Readiness Credential, including its design and usability. The data collected provided a solid foundation for further analysis, allowing us to make evidence-based recommendations. Surveys are a powerful tool for collecting data at scale, providing a broad overview that can guide more detailed exploration.

4. Develop a User Research Guide (Qualitative Data) 📝

Quantitative data provides a broad picture, but to gain a deeper understanding, we complement it with qualitative research. This involves developing detailed guides for our interviews, which help us stay focused while remaining open to unexpected insights. The qualitative data gathered through interviews adds depth and context to our findings. For instance, while a survey might tell us that a particular feature is unpopular, an interview can reveal the underlying reasons why. By combining quantitative and qualitative data, we create a more comprehensive picture of the user experience.

5. Create Transcripts 🎤

Conducting interviews is only the beginning. To extract meaningful insights, we transcribe the conversations using tools like Sonix.ai. These transcripts are then reviewed and edited to ensure clarity and accuracy. This step is crucial because it allows us to focus on the most relevant insights without losing any nuance. By thoroughly reviewing the transcripts, we ensure that the final analysis accurately reflects the participants’ perspectives. This meticulous approach to data processing is what allows us to provide our clients with reliable and actionable insights.

6. Apply Snowball Sampling ❄️

In some cases, the initial set of stakeholders might not be sufficient to capture the full range of perspectives. This is where snowball sampling comes in. By asking participants to recommend others who might have relevant insights, we can expand our research to include a wider range of voices. For example, in the JFF/IRC project, we explored the possibility of engaging a representative from Indeed, the HRTech platform. This approach allows us to gather additional perspectives that might otherwise be overlooked, ensuring that our research is as comprehensive as possible

7. Use AI to Gain Initial Insights 🤖

With the increasing availability of AI tools, we’ve integrated them into our process to generate early insights from the data we collect. Tools like GPT-4o and Claude help us quickly identify key themes and patterns in the data. These AI-generated summaries provide a useful starting point, allowing us to focus our analysis on the most promising areas. However, AI insights are not the final word; we carefully review and refine them, combining them with our own expertise to ensure a balanced and nuanced analysis. This approach allows us to work more efficiently without sacrificing quality.

8. Synthesise Findings 📚

After collecting and analysing both quantitative and qualitative data, we synthesise the findings into a comprehensive report. This report brings together the survey results, interview insights, AI-generated perspectives, and our own analysis. The goal is to provide our clients with a clear and actionable summary of our findings. We often create both a visual summary and a more detailed report. The visual summary is designed for quick reference, while the detailed report offers a deeper dive into the data, providing clients with the insights they need to make informed decisions. By presenting our findings in this way, we ensure that our research is not only informative but also practical and easy to use.

Conclusion

At We Are Open Co-op, user research is all about helping people make better decisions with insights they can trust. Our experience with the Job Readiness Credential project shows how dedicated we are to this work.

We’re here to support you in your own research efforts, whether you’re just starting out or looking to refine your approach. As our work progresses, we’ll continue to share insights that can help you make the most of your user research journey.

* Note that user research and user testing can get very involved and scientific. Going into more detail and depth is definitely important if you are doing things at scale. This post is aimed at encouraging those who may not have in-house capacity to get started for the first time!