Faulkner Insights
  • Home
  • About
  • Capabilities
  • Testimonials
  • Blog
  • Resources
  • Contact

​Think Like a Respondent: Consumer Research’s Golden Rule

1/24/2017

0 Comments

 
Picture
We’ve probably all seen examples of awful questionnaires or discussion guides. If you want more, there are plenty online—check out @MRXshame on Twitter for some hilarious ones. The crux of the issue is that we, as market researchers—client or supply side—have all been guilty of designing surveys that we would never want to complete ourselves. We conveniently forget or ignore how tedious those grid questions are, how annoying it is to answer the same question worded slightly differently multiple times, how impossible it is to remember something you bought six months ago, and how your attention span starts to wane after 10 or 15 minutes.

There are significant consequences of bad questionnaire writing, in the form of bad data and unreliable results—from straight-lining or random responses from the people who do finish your survey to chronically under-representing certain groups from people who drop out. For example, Quirk’s published a compelling study in February 2016 (“The impact of survey duration on completion rates among Millennial respondents”) which found that there’s a major drop-out inflection point among Millennial respondents after 15 minutes.
Education and training are critical to master good research design and quality, but even if we have a good foundation, we can still lose touch with the people who respond to and participate in our research. To that end, I want to offer a few simple suggestions that we can all start applying today to help us create research we would actually want to participate in ourselves.

1. Be someone else’s respondent.
Sign up for some online quantitative research panels or apps (e.g. e-Rewards, Field Agent, SurveyMini, The Pryz Manor, etc.) as a respondent. Always be honest—if they’re screening out market researchers, you can’t participate. But for the ones you can complete, you’ll get great ideas for what works, what to avoid, and how to make survey research more engaging. Nothing builds respondent empathy faster than taking a poorly-designed survey!

I would not recommend signing up for qualitative panels though. They should all have industry screen-outs and even if they don’t, the chance of messing up someone’s research in a qualitative setting with small base sizes is just too high.

2. Eat your own cooking.
Take your own survey. No, you might not be the target consumer, but you are a human. If filling out that complex matrix question drives you nuts—and you wrote it!—imagine how someone who doesn’t care nearly as much about your category/business will feel.

Look with alien eyes at that creative exercise you planned. Are the instructions clear? If you didn’t know what you know about your product/brand/category, would it make sense to you? How long will it really take to find all those images or complete that storytelling exercise?

3. Phone a friend.
Request peer-reviews of your questionnaires and discussion guides. If you’re on the client side, exchange surveys with colleagues for feedback, especially those outside your business unit/category if possible. On the supply side, you can also get feedback from co-workers, but just be mindful of confidentiality if you go outside the client team (i.e. use an in-market ad or package instead of the test one as stimuli, remove any proprietary client questions, etc.)

If you’re an independent consultant or don’t have ready access to colleagues for any other reason, strike a deal with a few trusted professional contacts to do a “feedback exchange” for questionnaires, guides, etc. where you review each other’s’ materials on a regular basis. The confidentiality caution applies here too.

4. Straight from the horse’s mouth.
There’s probably no better way to understand the real survey-taking experience than pre-testing it with consumers (i.e. not professional researchers.) There’s a range of ways to approach this—from very quick and informal all the way to an additional phase of research, depending on existing knowledge, business risk, and budget.

The most informal way to do this is to find people who fit the most basic criteria (e.g. pet-owners, restaurant-goers, detergent-buyers, vacation-planners, etc.) in your workplace or among friends and family and go through your screener, questionnaire or discussion guide with them. In this context, I think it’s most effective to administer it like a face to face interview where you read the questions out loud and mark their answers. You’ll get some instant feedback as you go (e.g. facial expressions, questions about the questions, etc.) and you can also ask for direct feedback too.

On the other end of the spectrum, if you’re planning a large-scale research project (multiple legs or geographies, a very high investment, or large potential business impact), doing a small qualitative phase up front to develop the questionnaire or guide can pay huge dividends. This also applies if you’re going to be researching a category/industry that’s relatively new to you and you don’t necessarily know all the right consumer language, response options, category attributes, etc.

--

Whether you’re learning to think like a respondent by actually being one, or getting feedback from a professional researcher or lay-person, we can all use these insights to make our research better—a little clearer, less complex, more engaging, and ultimately, higher quality.

0 Comments



Leave a Reply.

    Author

    Sarah Faulkner, Owner Faulkner Insights

    Archives

    July 2021
    April 2019
    July 2018
    April 2018
    January 2018
    October 2017
    July 2017
    May 2017
    March 2017
    January 2017
    November 2016
    September 2016
    July 2016
    May 2016
    March 2016
    February 2016
    January 2016
    November 2015
    October 2015
    September 2015
    July 2015
    June 2015
    May 2015
    April 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014
    August 2014

    Categories

    All
    Agile Research
    Business Tips
    Career
    Consumer Insights
    Consumer Research
    Design Thinking
    Digital
    Entrepreneurship
    Innovation
    Insight Audits
    Leadership
    Lean Startup
    Learning Plans
    Market Landscape
    Millennials
    Networking
    Qualitative Research
    Quantitative Research
    Storytelling

    RSS Feed

    Enter your email address:

    Delivered by FeedBurner

Picture
Copyright 2014-2023
All Rights Reserved

  • Home
  • About
  • Capabilities
  • Testimonials
  • Blog
  • Resources
  • Contact