Over the past three decades I have observed countless focus groups and for the life of me I cannot remember a single one from which I emerged thinking that it had delivered trustworthy data.
Come to think of it, for years now whenever the subject of whether or not to hold a focus group has come up I have spent pretty much all of my time trying to talk my clients out of them.
I am not alone in my disdain of this marketing practice - there are many others who feel that the format is not conducive to getting human beings to speak their minds honestly or divulge their inner feelings.
There is a wealth of evidence to suggest that focus group participants not only lie, but often allow their egos to develop responses that make them look good in front of their peers.
In his book How Customers Think, Prof Gerald Zaltman of the Harvard Business School, said: "The correlation between stated intent and actual behaviour is usually low and negative."
He added that 80% of new products or services fail within six months when they've been vetted through focus groups. Hollywood films and TV pilots - virtually all of which are screened by focus groups - routinely fail in the marketplace.
"Unconscious thoughts are the most accurate predictors of what people will actually do," Zaltman said.
"In the space of five or 10 minutes in a focus group, which is the average airtime per person, you can't possibly get at one person's unconscious thinking."
There are many, like Prof Zaltman, who believe that one bunch of complete strangers being led in conversation by another complete stranger is completely contradictory to the laws of nature in terms of seeking the truth. It pretty much ends up with participants trying to justify themselves and ending up contradicting themselves or just plain saying what they think the moderator wants to hear. After all, that's the guy providing the tea and cakes as well as, in some case, a nice little fee.
All of this is, however, completely academic. Because these days, one only has to go onto the myriad independent online forums that have sprung up for most well-known brands to get a far more accurate picture of what consumers think.
Admittedly you will initially just get the criticism, but if one goes beyond the people who are just plain unhappy with a product or service and look at those who are responding to them, you get a very clear picture of brand loyalty and the reason for that brand loyalty.
Four by forums
For example, just have a look at the numerous off-road and 4x4 forums and you will get a very precise picture of what it is that makes owners of brands such as Land Rover and Toyota so fiercely loyal. The information you get there you would never ever get out of a focus group.
It is beyond me why so few companies are not putting human resources and technology into monitoring online forums. There is a wealth of honest information if you know how to look for it and I daresay it is a lot cheaper than setting up a focus group.
Apart from being a corporate marketing analyst, advisor and media commentator, Chris Moerdyk is a former chairman of Bizcommunity. He was head of strategic planning and public affairs for BMW South Africa and spent 16 years in the creative and client service departments of ad agencies, ending up as resident director of Lindsay Smithers-FCB in KwaZulu-Natal. Email Chris on and follow him on Twitter at @chrismoerdyk.
LEGAL DISCLAIMER: This Message Board accepts no liability of legal consequences that arise from the Message Boards (e.g. defamation, slander, or other such crimes). All posted messages are the sole property of their respective authors. The maintainer does retain the right to remove any message posts for whatever reasons. People that post messages to this forum are not to libel/slander nor in any other way depict a company, entity, individual(s), or service in a false light; should they do so, the legal consequences are theirs alone. Bizcommunity.com will disclose authors' IP addresses to authorities if compelled to do so by a court of law.
I am with you completely, Jean (and Roy Langmaid). I appreciate that Chris has probably over the years seen some examples of uninspired moderating, poor interpretation, and that other approaches often do have more merit, depending on the objectives. But often views like this come from having been exposed to 'qual researchers' who are not really experienced or insightful. Unfortunately clients are often also at fault in the dumbing down of qual research, as they themselves do not really know what they are buying - i.e. what makes for great qual and what is less great....
I have seen first hand how people claiming to be qual researchers are conducting their groups like foreigners on a trip to a game reserve - ticking off the questions (big five) as they hear them, and moving on to the next one as soon as this topic has been covered and boxed off. The fluidity, creativity and intuition are lost when the process is so tightly structured. There is a real problem in SA, and elsewhere I believe, of people thinking they can get up and run with a qual project, after having watched a few groups - how difficult can it be? But the skill of a good qual researcher does not come from being able to pull off the process neatly, nor from the ticking off of questions in a perfectly written discussion guide - it comes from hours of churning through notes and transcripts, uncovering insights in the oddest of situations, knowing how to tease the information/thoughts out of people without them realising it, being in constant conversation with the category.
Let's not damn the focus group - when managed properly by real qual researchers with passion and insight and flexibility, it can be a truly great research tool!
I have had a good deal of admiration for Chris Moerdyk over many years. What a disappointment to read his comments on focus groups. He appears to be completely out of touch with what focus groups achieve, how they should be run and used and how much information and insight can be gained from them I give him this, however, that clients these days - those who commission focus group studies - appear to have amongst them a large number who display a complete lack of understanding of what groups can achieve, how projects using them should be briefed and that 'focus groups' per se is not the only methodology available to qualitative researchers. Such clients persist in interfering in the planning and executing qualitative studies to such an extent that the researcher ends up having to stick to a "questionnaire" that sometimes runs to 8, 10 or even more pages complete with notes on how long each section should be allowed in the discussion. It sounds to me that these are the types of groups that Chris has been watching! In a paper written for the Association of Qualitatve Researchers in the UK, Roy Langmaid, world renowned and highly successful qualitative researcher, and a good friend of mine, expressed exactly how I feel. "I have growing concerns about what we have allowed qualitative research to become – and indeed I fear that we may be collaborating in sowing the seeds of destruction of our own industry. "I fear we are moving away from people. We are going online or we are using topic guides in which the product, service or advertising gets much more attention. Research today is more about products than people. "There are several contributors to this growing distance between researchers and respondents but let me mention just the following five: 1. The bureaucratization of research. There are standard procedures, venues, processes, and templates for analysis, presentation and reports. All in all it adds up to a kind of PowerPoint World; slick and quick yes, but authentic – I don't think so. 2. The DIY Factor. As research became better known and more widely accepted, it acquired a kind of "anyone can have a go" character. After all, we all have conversations don't we – why shouldn't your clients" questions be just as useful as yours? 3. The unspoken benefit we get from not having to do groups face-to-face in out of the way places. If you can moderate the group from your desk, in front of your PC, you don't have to go out of your way to be with people – and neither do they. Surely everyone wins? Even the esteemed client can tune in from his desk and welcome the fact that his bill is smaller. However, as a climate for deep inquiry, it feels remarkably like the pretending in Mad Men that we saw way back there in the early 60s. 4. The continued shortage of advanced training and expertise for qual researchers. To my knowledge there is only one advanced training and there should be more. Do researchers appreciate the limitations to disclosure created by professional relationships – the type you initially get when you pay people for their attendance and participation? Do they know how to move from professional to personal, even private? What are the tools and processes you need? How much do researchers know about the psychodynamics of groups – aspects of group process that are always present and, unless raised into consciousness, will drive the group from the background? 5. If your qualitative work is about simple cause and effect relationships, these things may not concern you: but most is not. Most decisions and choices are derived from an interplay of observable and private reactions and understanding the dynamics of this relationship is often at the centre of a research brief, as it is at the centre of a decision to buy or not. Herein is the nub of the matter, to truly understand how choices and decisions are made it's necessary to experience both surface and beneath to figure out how they stand in relation to each other. It is the deeper stuff that is far more likely to determine behaviour, so without it our answers are bound to be superficial and insubstantial – a growing criticism of qualitative work that most of us are not doing nearly enough to rebut. If we don't do something about it our proud nail will be flattened and the world will walk over it without noticing we are there. Herein lie the seeds of our destruction. Roy Langmaid 2012
Roy Langmaid Roy Langmaid is one of Europe's leading consumer psychologists with a career spanning forty years. The founder of Co-creation in the UK in 1991, Roy has pursued a passion for innovation and his contribution has been recognised by awards and a Fellowship of the Market Research Society.
I agree with Michele and Magnum Octopus - poorly planned research, irrespective of cost or speed will result in poor data, insights and conclusions. MROCs are different as Michele said and can offer great insight with direction - but simply scraping info from a forum is problematic as mentioned. Also, I do believe that focus groups still have their place and offer valuable insights, given that the research is of course done systematic and in line with good research principles.
In the interests of full disclosure, I am a researcher, and one who has spend over fifteen years using both physical and Internet based qualitative research methods.
I agree with Magnum Opus(Octopus).
The quality of results of the focus group depends on how the group is run, who is running them, and how the observations are analysed.
MROCs have been all the rage for the last few years - it is naive to think that people will be more truthful or more engaged because they are online. They are just as likely to be human (conflicted, unreliable, likely to change their minds) as people in the real world.
Be careful, there is a big difference between a community of interest (a user forum) and a MROC.
When scraping comments from a forum, you open an ethical barrel of worms. Do you have permission to observe and use comments said in an informal setting for commercial purposes?
Never mind all that nasty stuff about respondent bias - but then again, if you are after cheap and fast, why should ethical, accurate and insightful matter?
Companies commissioning research generally have very defined research questions and objectives that they want answered. Is there not a risk that relying on online forums means that companies will be forced to revise, restructure or scrap entirely their research questions in favour of content that brand enthusiasts happen to want to discuss at the time? And where does one hear what the non-enthusiasts think, or those brand enthusiasts who don’t have the time or inclination to go online? An online forum is by definition not a random sample, so the results obtained are very likely to be skewed. Qualitative research does, indeed, seek to uncover the unconscious thoughts underlying consumer behaviour, usually by supplementing direct questioning with projective techniques - that's the entire purpose of this field of research. The point of a focus group discussion is not merely to find out if people like your brand – surveys can accomplish this – but why they like it, or perhaps why they do not. How is an online forum more adept at uncovering unconscious thoughts than a skilled researcher who knows how to ask probing questions? How can an online forum prevent participants from faking good, or bad for that matter, any better than a researcher? It’s obvious that monitoring online forums can give marketers very valuable insights into a brand’s consumers, but they are a far cry from what formal research is intended to achieve, and can never be an adequate substitute for properly conducted qualitative research.
I agree with this sentiment and I think a lot of advances have been made since. MROCs (http://bit.ly/QlH4Gt) are one way in which the focus group mentality can be overcome - especially in long-term communities where it becomes very similar to actual communities such as the 4x4 forums etc. Clients need to be more critical of the methodologies that are the status quo.