Translate Learning Into Insights With a Well-Planned Debrief

Wait, isn’t this series about qualitative research? Debriefing isn’t really part of that, right? Wrong! By the “debrief”, I mean the time the team spends after the research discussing the learning together. This ranges from non-existent (on the client side) to a rushed 30 minutes of discussion before flights to (rarely) full day analysis sessions.

You’ve just invested hours, if not days, observing and listening to consumers speak about your industry, category, products or services and, hopefully, how those things relate to their lives. Rushing out of your research without the due diligence of an in-depth debrief session with your team is like spending hours preparing a multi-course gourmet meal, taking a couple bites and then throwing the rest of it in the trash. It’s a huge waste!

Qualitative research and the debrief session that follows should be planned in tandem—think about the outcome needed and work backwards to purposefully plan the exercises, discussions and experiences that will give you the raw input you need to put it all together afterwards. For example, ‘day in the life’ ethnography sessions might be translated into a single story or sample schedule for your target consumer. Or, metaphor elicitation done during a session can provide the input for creating archetypes in the analysis session after. It takes more time and discipline to plan this kind of research, but it’s difficult (if not impossible) to get the level of deep, actionable insights you need without it. It also has the very important benefit of deeper team engagement, which can bring diverse perspectives and help buy-in for the resulting conclusions and recommendations.

How do you get maximum value from your research debrief? Using listening guides during the research and/or creating analysis templates in advance can be a big help. Consider learning processing activities like picture analysis (for consumer collages), mind-mapping, creating consumer hierarchies of need or other mental models.  Listening is the easy part—putting it all together to actually extract the insights is hard work. An experienced moderator with strong facilitation skills can be invaluable for this.

If it’s a multi-day project, build in time at the end of each day as well as at the conclusion of the project. As a rule of thumb, the more concrete and specific the research objective, the shorter the debrief and vice versa. For example, if you’re doing a handful of interviews to optimize a specific piece of creative with specifically defined learning objectives, you can probably complete the debrief in an hour or two. However, if you’re doing fundamental consumer or market segment understanding, gathering inspiration for future innovation, or doing in-depth ethnography, consider spending at least 50% (or maybe even up to 75%) of the time spent with consumers doing debriefing/analysis afterwards .

Bottom line: to maximize value, qualitative research shouldn’t be considered over until all possible value has been extracted from the findings and applied to business challenges and opportunities via actionable conclusions and recommendations.

Maximizing the Value of Every Respondent

What does “wasting” respondents mean? It could mean dumbing down recruitment screeners to hit a certain incidence level[1] for budget reasons and the resulting opportunity cost of an imprecise recruit. It could also mean investing the time to find on-target respondents, but not spending sufficient time with them to go beyond surface responses. These two are inextricably linked because to extract the full value from each respondent and make it worth spending significant time with each one, it requires the investment of time up front to ensure rigorous recruiting criteria, and if necessary, additional live pre-screening before the research.

Many people tend to think of recruiting for qualitative research like this: I write a screener that identifies people who qualify as my brand target, get those people in a focus group room, and ask questions for an hour. However, there are some important considerations for maximizing the value of each and every respondent that are somewhat counter to these traditional assumptions.

First, let’s look at the importance of the screener for qualitative research. In some cases, recruiting your brand target may be perfectly appropriate. Adding further designations for non-users, lapsed users, and brand or category rejecters or loyalists may also be helpful. Here’s the important point: qualitative recruiting, by its very nature, is not meant to be nationally representative (that’s what quant is for). Qualitative research is your opportunity to find the leading edge, the trendsetters, the extreme users, the inspirational target, people who absolutely love your product, and people who absolutely hate your product. Don’t be afraid to use the ends of the scale as criteria! Will it decrease your incidence and drive up costs? Probably so. Will you maximize every consumer interaction because you have something unique to learn from each person (and therefore can recruit fewer respondents overall)? Absolutely!

Next let’s look at the value of time spent with each respondent. If you’ve taken the time to do the perfect recruit and find the exact few consumers you want to talk to, why limit your time to an hour or two in a facility room? Furthermore, why limit it to a single interaction or a single location? To extract the full learning value from each and every respondent and gain the opportunity to deeply understand their life context, consider multiple interactions over time. Depending on the objective, this could range from a day-in-the-life ethnography session or an “expert panel” where you engage with a select group of respondents once a week for several months to iterate or co-create. If you’re doing a longitudinal interaction, think about how you can keep people engaged between discussions—invite them to participate in an on-line community, write in a journal or blog, or contribute to a Pinterest page on the topic, etc.

To sum up, not “wasting respondents” means doing more precise and high quality recruiting to find the best possible respondents for your objective as well as designing in the right interactions and time frames to reveal deeper levels of learning from each person.



[1] Incidence Level: percentage of the population who meet your pre-defined set of characteristics (for example, the incidence of English speakers in the US is 82.1%, according to the CIA World Factbook).

Qualitative Objectives for Qualitative Research

There’s a common misperception that qualitative research is faster, cheaper and easier than quantitative research.  There’s a vast range of methodologies within each type, so in some cases that is true, but when each is done correctly, quantitative can actually be significantly faster and cheaper than qualitative for many types of research objectives. To know which is appropriate, check your objectives against the “3 Cs”: confirmatory, checkbox, and counts.

• Confirmatory: if you absolutely must do confirmatory research (meaning, research done just to       validate something you already are planning to do), a quick quantitative screener will give you an answer much more efficiently than qual.
  

• Checkbox: if your discussion guide looks more like a survey, it probably should be.  Don’t use qualitative research just to ask respondents which one they prefer or pick the best option. At the risk of over-simplification, if you want to know “what” or “which one”, use quant. If you want to more deeply understand the “why” (and sometimes “how”) use qual.
  

• Counts: if you need percentages, tallies, or any numerical data at all, you need quantitative research. Don’t let anyone cheat by talking about or reporting any statistics from qualitative data—it’s just incorrect, misleading and dangerous for decision making.
  

Discussion or interview guides should be just that—guides. A discussion guide should be an outline of topics to be covered in an approximate order. Professional moderators should be expert at guiding discussions organically so you get a more authentic view of your consumer while still getting all the research questions answered.  If you’re doing self or team-moderation, first ensure that everyone is trained on interviewing basics. This is not a survey checkbox exercise, this is a conversation. 

Try putting yourself in the respondents’ shoes. How interesting would it be to answer a series of closed-ended questions for an hour? Now back to our researchers’ shoes: how much more can we learn by asking for stories, examples and analogies? Not to mention, designing creative exercises that access the unconscious mind and observing actual consumer experiences in real life context.

By choosing the appropriate methodology based on the business questions and learning objectives, you can save both time and money.  When qualitative research is the right approach, structure the discussion guide and activities to take full advantage of the depth and richness of learning available from the interpersonal and interactive communication inherent in this technique.