Measuring the consumers of advertising

Bringing precision and elaboration into ad testing

mike broom  16 Jun 2017

This is a very complex field that has troubled the minds of many researchers long before marketing was ever dreamed of. A good place to go looking seemed to be medical science. When you go to the doctor a few measurements are taken, you tell your story and you’re asked a few questions. After about 10 minutes, you’ll be told what you are doing correctly or wrongly in your physical life. The big question is…

How can the doctor do this so efficiently, quickly and accurately?
After all, the human body is a very complex entity. The follow up question is…

Why can’t we do this with brands – or even advertising?
Well, it has to have something to do with the body of knowledge, the robust research discipline and the measurements made.

Take a look at this picture…

Dolgachov © – 123RF.com

A market researcher may say:
On a scale of 0 to 10, where 0 means no difference and 10 means a huge difference, please give a score for how much taller Sally is than Jack?

A medical person would probably say:
Why on earth do you want to ask that question? A far more useful way of going about it is to establish the ages of Sally and Jack and ask whether either is taller or shorter than the population median at that age.

The point here is that market research can often be blindsided by the question rather than how useful the answer would be. Having norms like the median age connects the question to a number of subsets of the knowledge base that would answer questions like: ‘what reasons could there be that Jack is so small if he is 17-years-old?’

Norms are useful in that way, but there are also other ways of asking questions.

A new measurement paradigm that is ideal for ad testing

mike broom  23 Jun 2017

Think about the measurements an optician does when you go to have your eyes tested. The only questions he asks is a series of binary choices. “Is this better? Or this?,” as different lenses are slipped onto the eye piece. Now, this idea of deriving very precise measures from simple binary choice questions is very interesting because it is a simple, non-intrusive method that is very reliable in producing valid measures. We have adapted this procedure in developing the MSc measurement that we have pioneered in Ad-Audit.

The measurement methodology involves taking all the attributes, say, and comparing them in pairs, and asking; “Which is more important to you in buying X product: Attribute A or Attribute B?” So, looking at the picture, the questions gets rephrased to “Which is taller, Jack or Sally?”

We still need to apply a metric (in the case of the optician’s test’s the different lenses have been manufactured to specific measures), and we do this by following the reasoning of Daniel Kahneman’s System 1 and System 2 – where System 2 involves cognition, while System 1 is instinctual thinking. In our MSc measurement system, we ask “How easy was it to decide?” We have been testing this scale for several years and the range from ‘Obvious’ through to ‘too difficult to decide’ is giving us excellent results on many characteristics, including replicability, ease of use, speed of response, internal and external validity.

It therefore consists of decomposing the predictive models of executional exceptionalism (shown in the table below as advertising basics) and strategic achievement into statements that are subjected to the MSc measurement process of full factorial paired comparison. The results that emerge are in simple tables like the following:

There, of the four possible executions tested, Advert A is the best. Because it performs the best on ‘truthful and believable’, which in this category accounts for 31.8% of executional effect. It also scored highest on ‘Puts you in a better mood’, which is the third most important factor in this category. The fact that Advert D scores pretty well implies that as an option, it may be better to tweak Advert D and give it a lighter touch to enhance mood better.

It thus turns ad testing into a collaborative adult discussion about effectiveness rather than the opinions about creative execution.