Fly-fishing for results - how targeted sampling generates better research results
Research methodology is always the hotly debated topic at the outset of a new project when, in fact, the quality of the sample is crucial to a meaningful research outcome.
As a professional researcher I am not ashamed to say that it's imperative for companies to conduct market research on an ongoing basis. There is a host of questioning methodologies out there, all with their specific benefits. But what is of the utmost importance, and often overlooked, is the quality of the sample from which the answers are derived. In plain English this means the sample you use will determine the results you get, so it is vital for accurate insights that the research sample correctly represents the client's intended target market.
Sadly though, in my experience this is not always the case, particularly when the more old school research methods are used. This is due predominantly to the inherent limitations in sourcing accurate samples.
For instance, the questioning time constraints that hamper many projects mean that the ever important profiling questions are often not asked; frequently at the client's request because they want as many answers as possible. And even when they are asked, the information needed is of a sensitive nature (e.g. household income) and respondents are hesitant to provide these answers leading to much umm-ing and ahh-ing as truth versus ego is contemplated. It is these uncomfortable uum and aah situations that lead to the inaccuracy of research results.
To be fair, it's not always the respondents who negatively affect the accuracy of results; fieldworkers are also to blame. Often the interviewer is more concerned about the number of responses achieved rather than the quality. This could lead to a situation where the interviewer fabricates results just to meet their targets. I'm not saying this is a given but it does have the potential to happen and when it does, the impact for the client is severe.
Focus groups can and do provide some really great qualitative insights into the consumer mindset. However they are often criticised because the conversation can be controlled by one or two participants, which does not allow for accurate representation. Also, the emergence of the focus groupie who has established a strong relationship with recruiters and almost built a career from attending focus groups is having a negative impact on the true value that this methodology can provide. However much this phenomenon is denied, the presence of a groupie becomes painfully obvious when participants present a song and dance telling us what we want to hear - clearly a spoon feeding exercise on the part of the recruiter.
So how do we keep research relevant and meaningful for the well informed client with limited budget? We must look to newer and maybe less personal research methodologies that allow us to access better quality samples.
The best alternative to our traditional ways is the online panel. Sure, it has its limitations. By definition, this methodology only provides access to a particular segment of the market which, in South Africa excludes a significant proportion of the population. But the online population in this country has to be considered as the early adopter that provides insight into, and leads key trends.
In addition to providing a quicker turn-around, an online panel also far exceeds other methodologies in delivering more accurate samples. Don't forget that the online panel is comprised of opt-in participants who have chosen to share their thoughts, attitudes and opinions so it must be assumed that the quality of their responses outweigh those of the 'coerced' sample.
But the root of the sample solution lies in the panel software that allows researchers to better manage and control who gets to respond. Simply put, researchers can specify the selection criteria but thereafter requests to participate are random. Respondents' answers can also be monitored to ensure that no irregularities creep in to the findings. Panel software will also ensure the correct frequency of participation as over-sampling can impact on the integrity of responses.
In a nut shell, the online panel tends to produce more accurate results from within a correctly targeted audience. The result being really meaningful answers to a client's very important questions.
Whilst online panels may have some limitations currently within our environment, the application of this technology must surely be the way forward for research companies that want to ensure they're future-proofed, and for one, I look forward to seeing how panel thinking will create innovation of the more traditional research methods!
About the author
Gary Greenfield, Managing director, Interact RDT