Ads should be tested Quantitatively
Good advertising pre-testing involves the practice of measuring (testing) elements, yielding scores for the various components of the advertising's workings. This means that the best way to pre-test a commercial is by using a one-on-one quantitative interview.
Focus groups are the most popular method for pre-testing advertising, perhaps because they are perceived to be inexpensive and fast. Whilst focus groups are an excellent method for problem solving (many minds make light work), and creative idea generation (divergent views stimulate lateral thinking), they are a very poor tool for testing anything.
This is because they don't provide scores, and worse still the well documented (Janis, 1972) "Groupthink" phenomenon causes groups to shift their opinion from left to right, and vice versa as dominant respondents change the views of weaker ones. This means that respondents' feedback runs the risk of not being a true account of their beliefs, but rather a modified opinion based on the reactions of other people to the subject under test. Focus groups therefore run the risk of giving wrong answers.
From an economic point of view, focus groups in fact yield poor value on the basis of volume of output per respondent. A typical focus group consists of 8 respondents, interviewed for 2 hours, at a cost of about R14 000 each. This means that each respondent gets to speak for an average of only 15 minutes, at a cost of R1750 per respondent! Compare this to a 40 minute face-to-face interview at an average cost of R300.
What to Measure
In 1981, the PACT agencies delivered their world-renowned recommendations on advertising testing. Amongst their nine key principles of advertising research is the notion that advertising works in many ways, and therefore advertising testing needs to embrace the principle of multiple measures.
In particular, the multiple measures recommended are: -
1) Impact: the ability of the advertising to be noticed and remembered.
2) Communication: the ability of the advertising to impart a message, which is clearly, and uniformly understood by the target market.
3) Relevancy: the ability of the advertising to persuade consumers that their needs will be met by the product.
4) Affinity Building: the ability of the advertising to generate consumer affinity (liking) for both the advert and the brand being advertised.
5) Call to Action: the ability of the advertising to motivate consumers to try or re-buy the brand being advertised.
6) Brand Building Ability: the ability of the advertising to create, change or reinforce certain key predetermined brand attributes (features, benefits, feelings) as encompassed in the brand's positioning objectives and strategy.
7) Involvement: the ability of the advertising to involve the consumer, or keep him/her interested.
8) Brand Fit: the ability of the advertising to demonstrate brand fit, as opposed to generic category fit or, worse still, competitor brand fit.
9) Creative Diagnostics: the pre-test should elicit a host of creative diagnostics to help answer the " whys?" that always emerge from behind the above measures.
In summary: In our view, face-to-face interviews, using a highly structured questionnaire that has been tailor-made for the category, brand and advertising, is the ideal way to pre-test advertising. The questionnaire ought to embrace the principle of multiple measures.
This approach is difficult however when the advertising is a multi-commercial campaign, and when the commercials have to be "explained" to the respondents as in the case of storyboards, animatics or scamps.
For "rough-cut", multi-commercial campaigns it is necessary to convene respondents into a central location to be able to present and explain the campaign in a proper manner.
Nominal Groups
To this end, the Nominal Group provides the ideal solution. The nominal group was developed by Janis in the 1960's specifically to overcome the problem of incorrect conclusions resulting from Groupthink.
In essence, nominal group delegates approach each topic from an individual perspective in written format, prior to any discussion. Each delegate's views are privately recorded without the influence of the other delegates' opinions. Where required each individuals' opinions can be tabled and discussed to generate a group consensus view.
The individual workbook approach allows the creation of quasi-quantitative (multiple) measures against which to score the advertising.
From an economic perspective, nominal groups provide much more output per respondent, because each respondent works together in parallel with the others, yielding almost 2 hours of output per delegate.
In Summary:
Depending on the state of advancement of the advertising stimulus material, we recommend conducting face-to-face interviews (finished material that needs no explanation), or nominal groups (unfinished material that needs to be carefully presented and explained).
Always the goal is a quantified sample of respondents who closely match the profile of the target market.
