Marketing research - Fact or fiction?

© yuriz via 123RF.com.

Starting with knowledge…

We ‘know’:
’Half the money I spend on advertising is wasted; the trouble is, I don’t know which half (John Wannamaker, 1838-1922).

Fact:
Nobody knows which half. It could quite easily be 90%. Outside of the retail environment, there’s little evidence of a strong causal relationship between advertising and sales.

We ‘know’:
’Everyone knows the new product failure rate is over 80%’.

Fact:
It’s about half that.

We ‘know’:
’Marketing is strategic‘

Fact:
Marketing is about driving sales. All strategic marketing definitions make it clear that the goal is ‘… achieving a sustainable competitive advantage’ (Wikipedia).

Consider the proposition that ‘marketing research is always wrong’. The proof being that, it can’t measure how wrong it is.
So, how can we accept or reject this proposition?

To start with, remember that a fact is ‘something that actually exists; reality; truth known by actual experience or observation’.

Interesting to learn that antonyms of the word fact include theory, fabrication, lie and, not as is commonly believed, the word ‘fiction’. It is comforting to know that if marketing research is not fact, then it’s not necessarily ‘fiction’. Why would we be so relieved that marketing research it is also not ‘fiction’? Because ‘fiction’ is something feigned, invented, or imagined; a made-up story.

What does that mean for practitioners and marketing decision makers to be dependent on a field that is factless and fictionless, when it generates so much marketing decision support information?

It means that it lacks credibility and/or integrity. It means that it can be ignored with impunity. There is no sanction for not conducting marketing research, nor any reward. Compare this with, for example, research in the fields of engineering, medicine, finance, law, among many others.

There are some ways that marketing research can improve to command more respect in decision support science and sit more comfortably in the boardroom.

Evidence

Become more factual in the true sense. It’s not good enough to measure something once, nor to have doubt about that measurement. There needs to be replicability and validity. This means adopting the methodology and protocols of evidence. The chief value of evidence is that it enables reliable prediction.

If you have evidence that something happens when certain conditions are met, then it’s reasonable to expect the same outcome, should the same condition re-occur. Importantly, if the same conditions recur and the prediction doesn’t come to pass, we now have a mandate to interrogate the evidence. Typically, this never happens in marketing research.

For marketing research data to approach the stature of evidence It needs to pay attention to the Measurement and methodological issues associated with replicability and validity.

How this can be done

Ask the same question of the same person at a different time. If you know enough about the person and the topic of the question, then you can expect the second measure to be the same, higher or lower. Now one can test this hypothesis with the outcome of the second measure. If the hypothesis is accepted, it is a candidate to be treated as evidence.

In July 2018 Absa began a roll-out to relaunch the Absa brand and communication. BizCommunity published an article of PSA’s findings in a survey preceding the launch. PSA sent the online survey to 200 people selected to be nationally representative of the South African online population. For results see below.

What's the use of marketing research? A cynic's answer: Ask Absa!

mike broom  13 Jul 2018

To demonstrate, even PROVE, the arguments outlined above, PSA conducted a repeat survey in the last fortnight of December 2018 – six months after Absa spent millions (or more) announcing their new logo and rebrand. 67% of the people surveyed in the December wave had completed the July survey. The remainder were new recruits to the TellUsAboutIt panel.

According to brand attribute importance theory (aka Fishbein theory), there are two ways that a brand can gain competitive advantage:

1. Increase the importance of attributes that are most strongly associated with one’s own brand. e.g. Apple with the attributes ‘innovation’ and ‘quality’
2. Be perceived to have more association than competitors with attributes that are important. e.g. SAB with the emphasis on supporting sport.

The findings presented in the comparative charts below show highlights of the findings within this context.

This chart above shows that the massive extra spend did not alter the importance of the attributes driving the market. The minor changes (in no case more than 0.1%) are easily attributable to the 33% fresh sample. Therefore, the hypothesis that there was no change in the perceptual structure in the market can be accepted. This is in line with the observation that none of the logos makes any claim to any of the attributes. This argues for the power of the methodology to be reliable.

To see the effect of the perception of brands across the attributes driving the market, we calculate a weighted ‘image score’, each brand weighted by the importance of each attribute and extent to which that brand is associated with that attribute. This should show what was achieved in terms of competitive advantage.

The chart below argues for the validity of the methodology in that the effect of the transfer of association from the old to the new logo is clearly demonstrated (presumably the company’s intention) and the control brand (Capitec) changed by an insignificant 2% against 41% up for the new logo and 30% down for the old logo.

Our conclusion six months ago, that the Absa rebrand may prove to be a blunder, is now retracted.

Absa marketing are to be congratulated on the success of a difficult task because the evidence, being reliable and valid (because of the sampling and measurement methodology), shows that nothing of serious consequence happened in the market over the past six months, except the transfer of image equity from the old logo to the new logo.

We wait to see what the point of it was in the months ahead.

Notes

• The detailed methodology can be found in the July article. 
• We are happy to make the data and calculations available to interested parties.
• Similar fieldwork and methodology are incorporated in new product sales forecasting.