Winning by design

Issued by Kantar

2 Aug 2018

Despite broad research supporting the negative implications of long survey design in market research - order bias, respondent fatigue, interviewer bias and poor survey design is still very common in South Africa. Caitlin Noonan and Marguerite Steyn from Kantar discussed this in their research paper, "Win by design", at the recent Southern African Market Research Association Conference.

If the trend in long surveys continues, say Noonan and Steyn, surveys will eventually become obsolete, so a drastic change in design needs to happen to ensure that surveys remain relevant and provide guidance to answer marketers’ core questions and objectives.

The market research industry is in a state of technological transformation where new tech tools are causing disruptions, and with the data collection pool broadening to mobile and online research, the need to become device and data agnostic is becoming more prevalent. Even more crucial is the need to design properly for mobile and online research.

To really reveal key insights, they say surveys should be more concise and smartly designed, regardless of the data collection tool used. Noonan and Steyn evaluated mobile and face-to-face data to identify their top 10 tips to help reduce order bias, respondent fatigue and interviewer bias, while making the surveys engaging.

The top 10 tips:

Short is best - Keep your surveys to 15 minutes or less to avoid respondent fatigue and poor data quality.
Rotate – Remember to randomly or reverse rotate option lists.
Think - Think carefully when to include “Don’t know” or “Neutral” answer options.
Engage – Keep surveys engaging for respondents and use gamification techniques to make questions visually impactful and engaging to prevent respondent fatigue and low response rates.
Less is more - Use fewer words to describe attributes, descriptions or market factors. Be concise with your question wording.
Simple - Use clear and simple language to avoid confusing respondents, especially as English might not be the respondent’s home language.
Honesty - If sensitive type questions or honest brand ratings are required, then mobile should be the preferred data collection tool, as this leads to more honest answers with the elimination of an interviewer.
Limit - Limit screening questions to a minimum, as numerous screeners on a mobile survey could lead to fatigue, resulting in respondents dropping off before they even start with the main survey.
Think small - Design surveys with a smallest screen device in mind, so the survey renders appropriately across all types of digital devices.
Be considerate - Consider the type of person you want to complete the survey and show respect for their time, as we are fighting for a small piece of a person’s attention.

By using these 10 guidelines when designing a survey, the results from their study prove that you will get better quality data, as it eliminates some of the negative human behavioural traits evident in respondents during interviewing. And improved data leads to valuable and actionable insights to guide business decisions and ultimately ensures the relevancy of surveys.

About the authors:

Caitlin Noonan is a Consultant for the Mobile Practice for Kantar’s Insights Division in Sub-Saharan Africa.

Marguerite Steyn is a Senior Research Executive at Kantar TNS.

Kantar is the world's leading evidence-based insights and consulting company. We have a complete, unique and rounded understanding of how people think, feel and act; globally and locally in over 90 markets. By combining the deep expertise of our people, our data resources and benchmarks, our innovative analytics and technology we help our clients understand people and inspire growth.