Advertise on Bizcommunity

Subscribe to industry newsletters

Advertising pretesting done right (Part 1)

Contact for more insights or a discussion on advertising pretesting.
We would love to hear your views on advertising pretesting. You can leave your comments here. Let us know what you think about this field of research.

Part One: Avoid the “check box” approach to advertising pretesting

123 years since Harlow Gale’s attempt to measure the relative values of illustrations versus text in print advertising in 1896, advertising pretesting research has been shrouded in controversy and conflict.

“Copy testing is one of the most controversial areas in the field of advertising. Its use gives rise to strong feelings. Both for and against. And endless disputes about how. It has repeatedly provoked passionate arguments amongst all those who have a stake in the results.”
The PACT Agencies, 1982, 2

This controversy and conflict finds its roots way back in the 1920s when Daniel Starch, and others, began the commercialisation of copy research. The primary driver of this commercialisation was to try to establish “universal truths” about how advertising works, and which commercials work better than others, and then to make money from judging ads against these claimed universal truths.

From here onwards history is littered with countless proprietary, model-based, advertising research systems. Most have not survived. Yet new models continue to emerge and become fashionable for a decade or so.

The 1950s saw the emergence of the focus group as a method for pretesting advertisements. In fact, it was all the rage then. Still today, focus groups are one of the most common, and least desirable, method for evaluating advertisements prior to flighting.
Focus groups are all too often perceived to be inexpensive and fast. Whilst focus groups are an excellent method for problem solving (many minds make light work), and creative idea generation (divergent views stimulate lateral thinking), they are a very poor tool for testing anything.

This is because, in the first place, they don’t provide reliable scores as their sample sizes are too small, and worse, the well documented (Janis, 1972, 1) “Groupthink” phenomenon results in dominant respondents dramatically shifting the opinion of other group members either in the direction of approval or disapproval.

This means that respondents’ feedback runs an extremely high risk of not being a true account of their beliefs, but rather a modified opinion based on the reactions of other people to the subject under test. Focus groups therefore run a high risk of yielding wrong answers.

But potentially a more insidious problem with advertising pretesting is the plethora of methodologies that set out and seek to “score” the candidate advertisement on a set of preconceived hurdles deemed to be constituents of “good advertising”.

This checkbox approach to advertising research has the definite effect of “failing” advertising with endearing qualities such as uniqueness, creativity, entertainment appeal, originality, memorability and, most importantly, the quality of imbuing the brand with that elusive characteristic called “love”.

One thing that has been proven in advertising research, many times over, is that consumers tend to love and be loyal to brands who run advertising that they in turn love. Advertising is not merely a functional business, it is largely emotive.

“Creativity is art and research is science - an oversimplification, certainly but one which has traditional meaning. The distinction emphasises differences between intuition and subjectivity on the one side and intellectuality and objectivity on the other. Of course, nothing in this life is totally one or the other, but cultural history will show that our heritage is full of the tidal pool between feeling and thinking, deduction and induction, accepting the unprovable and trying to prove the unacceptable.”
Vaughan 1983, 3

In this modern age of extreme advertising clutter and short consumer attention spans, it becomes vital that advertising is able to attract and hold attention, as well as reward the consumer with a relevant emotional tug at the heartstrings over and above the imparting of functional brand attributes.

The checkbox approach to advertising research cannot possibly account for all the possibilities involved in emotive response. Accordingly, the checkbox approach tends to pare advertising down to the level of the average, the medium, normal and even the mediocre.

The checkbox approach is popular because it doesn’t require any mental effort. It is the cookie-cutter approach to creativity.

Against this backdrop, here are a set of well-founded philosophies and tips to think about when next pretesting your advertisement. At Bateleur, we have applied these principles relentlessly for more than 30 years with proven experience that this practice goes a long way to properly evaluating advertising without killing the goose that lays the golden egg!
  1. Tailor-make the research design to embrace the creative rationale. All good advertising has a clear and documented creative rationale. Moreover, this rationale is usually unique to each advertisement or campaign. The researcher must have a clear, sensitised and deep understanding of the creative rationale and expected production values. Thereafter, the research needs to be tailor-made to specifically dovetail this rationale. Our industry has been plagued for decades by one-size-fits-all approaches to advertising research. It’s no wonder that great ideas die, and standards migrate towards merely average. The implications of this first principle means that the researcher needs to be briefed properly in respect of the creative rationale, and then develop a methodology that specifically evaluates its efficacy.

  2. Tailor-make the research design to embrace the advertising strategy. Advertising research only begins after the groundwork has been laid by fundamental strategic planning. Advertising research is hard pressed to separate out the expression of strategy versus that of execution. They are inextricably intertwined. This means that the researcher needs to be well versed with the strategic intent behind the campaign, and be sensitised to tailor making the research design (especially the questionnaire) to incorporate and evaluate this strategic intent. Once again, we see that checkbox and cookie-cutter approaches to advertising research frustrate this important principle.

  3. Avoid the “pass-fail” mentality in designing advertising research surveys. Advertising research is not there to “pass or fail” a campaign idea as measured against some normative benchmark. The purpose of advertising research is to provide insights about consumers’ reactions to a campaign, to fuel the decision-making process. It should be viewed as an aid to judgement, not a bar to surpass. The implications of this are that the researcher needs to avoid a pass - fail mindset and actively adopt a learning mentality. It is true to say, that the same applies to advertisers.

  4. Adopt the principle of multiple measures. Professional advertising researchers understand that it requires more than one measure to understand any particular aspect of a proposed campaign. The principle of “multiple measures” means that from a design point of view we need to ensure that we approach each aspect of the communication evaluation from several angles. This triangulation principle helps us hone in on the correct set of insights and decisions. The implications are that the researcher needs to design a survey that adopts the principle of multiple measures. Specifically, the measures need to conform to the objectives inherent in the communication strategy and creative rationale.

  5. Consider the likely effects of production values on the research results. Good advertising research needs to recognise that the more finished a commercial is, the more soundly it can be evaluated. The effect of production values is enormous. One needs to apply judiciously sensitive interpretation to the results of advertising research applied to non-finished work and account for, or estimate, the effect of these diminished production values.

  6. Consider the implications of exposure context. Good advertising research gives consideration to the intended exposure context for this campaign. This means that in an ideal world, we need to expose the advertisement to respondents in a way that is similar to what’s expected in real life during campaigning. Ideally, the methodology should present the stimulus material in a way that mirrors intended life reality.

  7. Evaluate advertising with members of the target market. Lastly, good advertising research ensures that respondents conform to the definition of the campaign’s target market. This seems fairly obvious, yet it is amazing how many ads are tested on a “general population panel”, rather than a sample that is screened to make sure that the target market has in fact been reached with the research. This means that the research needs to be designed in a way that screens in respondents who conform to both demographic, psychographic and behavioural characteristics as defined in the target market specification as ought to be laid out in the communication strategy document.
In a nutshell, all the above advertising research principles say that the researcher needs to be properly briefed, and then tailor-make methodology that specifically focuses on developing sets of measures relevant to the objectives, strategy and tactics inherent in the intended advertising campaign. This is not a formula-based activity.

Avoid the checkbox approach to pretesting advertising!

We would love to hear your views on advertising pretesting. You can leave your comments here. Let us know what you think about this field of research.

Contact for more insights or a discussion on your advertising pretesting needs.

Look out for part two in the series, coming soon...

Part Two: Tools to tailor-make advertising pretesting that invigorates creativity and results in lovable ads

1. Janis, I. L. (November 1971). "Groupthink". Psychology Today.
2. PACT Agencies (1982). “Positioning Advertising Copy Testing”.
3. Vaughan, R. (1980). “How Advertising Works: A planning Model”, Journal of Advertising Research, Volume 20, Number 5, October.

Bateleur Brand Planning's press office

Bateleur Brand Planning
Bateleur is a unique market research provider in that our focus is fixed on our clients' future success.