Time to stop misleading SMS polls

Last week for example, halfway through e.tv's 7pm bulletin, the following question was asked: “Do you think South Africa should be helping Zimbabwe refugees?”

Viewers had less than 10 minutes to send in their SMS replies and the results at the end of the bulletin were those of the 7782 respondents: 35% said yes and 65% said no. Given the speed at which viewers had to respond, it is logical to assume that those who felt the most strongly about the Zimbabwe refugee situation – the xenophobes – would be those most likely to bother to grab their cellphones and spend R2.00 on an SMS.

Muckraking

Viewers who felt Zimbabwe refugees should be helped were equally logically not nearly as motivated to rush for their cellphones. It is called human nature. And what e.tv was doing is called moneymaking muckraking.

Particularly as professional research conducted last year found that the overwhelming majority of South Africans of all races felt that people from neighbouring countries should be welcomed.

Barbara Cooke, one of South Africa's most respected market researchers and founder of Target Group Index (TGI) in this country, said that SMS polls “are no more than a self-selected sample of interested persons, either pro or con. As such, they are representative of no real universe except perhaps a universe of persons at the extreme ends of a spectrum of opinion.

Misleading

“Nobody else would bother to respond. So yes, these polls are misleading. They are also statistically rubbish and could actually be very dangerous in forming the opinions of the fence sitters.”

Cooke said that in attempting to research this [Zimbabwean refugee] issue, “you have to ask three questions: whose opinions do I want? [The universe]; how do I properly canvass their opinions, especially since I can't contact everyone in my defined universe? [The sampling methodology] and once I have the results, how do I correctly interpret them? [The analysis tools]

In the case of the e.tv opinion poll, Cooke added “I would want to know who was actually watching the programme being broadcast at that time [the universe] and then what percentage responded and who those people were?

Mean nothing

“Were they representative of the total viewership of e.tv during that time band? Unlikely. These kinds of polls are just grist to the TV content mill, and make for sensational tit-bits on the programme. They mean nothing at all. “

Heidi Brauer, marketing director of one of South Africa's most respected research companies, Markinor, responded to my question about whether SMS polls have any value by saying, “Well, yes, in that they generate revenue for the media owner (R2.00 each) and they provide biased editorial and maybe discussion in people's homes.”

Lack of sampling

Is it research?

“Not really, says Brauer, “there are two main problems – one is the way a question is usually phrased and the limited range of responses (usually yes or no ) that are available. But, even more of a problem is the sampling process, or total lack of it.

“This process relies on a sample to self-selection. So, people choose whether or not to participate rather than being selected out of a drawn or constructed sample. So, there is no definition of the universe firstly and then no selecting of a representative sample from within that universe, which is what the margin of error calculation is based upon. In these cases one can't even calculate a margin of error – it's just skewed stuff.”

Brauer added, “Typically in a self-selecting sample (think of the little forms you may or may not chose to fill in a hotel, or on the customer service agent's desk, or in your rental car, or even the electronic devices on counters in CNA etc) you elicit responses from two ends of the spectrum – happies versus grumpies or lefties versus righties, supporters versus dissenters. Not a representative sample of the average Joe.

Discredited

“They love to do these things – vote for this and that and then report the results as credible. Remember when SABC did the ‘Vote for your favourite South African” via SMS a couple of years back? It was eventually discredited totally – who could afford to SMS, how many votes did each person make, did I get my family and friends to vote for me?”

From a public opinion research point of view, concluded Brauer, “it really can be dangerous stuff”

Not xenophobic

Hopefully those gung-ho people in mass media newsrooms will give some thought to the consequences of such ill-advised and amateurish research. As e.tv has done by tending to suggest with some considerable sincerity that 65% of the populating of this country is xenophobic when, in actual fact, far more credible research shows the opposite.

There is no doubt that SABC news has lost a lot of credibility in this country and if e.tv keeps on this sort of sensationalist track, it will also suffer and, like the old days of apartheid, we will all have to go back to relying only on 702 and Cape Talk for broadcast news we can believe.

Which would be a crying shame because e.tv has done an outstanding job in news presentation but need now to be very careful of staying clear a tendency I am picking up of crossing the boundaries of credibility into the realms of sensationalism, presumably to boost its viewership.