Positive report on South Africa's radio currency by UK expert

Gane is a media research veteran whose experience includes being research director at RSMB, MD of the media research division of Ipsos UK, and MD of RAJAR, the UK's radio audience measurement body.

Working with Nielsen both on site and in the field in November 2013, Gane sought to determine whether or not RAMS was providing valid and stable results, and was generally 'fit for purpose' in the context of international standards and methodologies.

Overall findings

His overall conclusions were that "the RAMS service is designed and being operated as specified and will deliver audience estimates which are representative of the radio broadcasting situation in South Africa."

In addition, he felt that the overall design and methodology of RAMS should "produce period-to-period results which are as stable as may be expected, given the limitations of sample survey research, and the complex demographic and geographic nature of the country."

He concluded that, in the short-to-medium term, the current RAMS methodology could be retained "without major concerns over its operation or quality."

Addressing specific issues

The NAB and SAARF had a number of specific queries that were included in his brief.

• On using amps as a vehicle: This is cost-beneficial to RAMS, and ensures there is no pro-radio bias in the placement process. The AMPS/RAMS sample design also delivers a "good quality sample that takes into account the complexity of the market. The current sample frame provides as robust an approach as is reasonable to expect given the current environment."

He warned that the length of the AMPS interview might pose some risk to the quality of RAMS data. He also emphasised the positive point that "the wide-ranging coverage of the AMPS interview means that there is no obvious focus on the radio medium and hence little or no possibility of skewing respondent selection or listening claims as a result."

• On the disproportionate sample design: Currently, AMPS and RAMS over-sample some areas and under-sample others, to accommodate the requirements of SAARF users' business needs. While he acknowledged that the disproportional sampling of these areas was in principle "appropriate," he nonetheless suggested that, in time, it might be beneficial to revisit the present area sampling structure, particularly as internal migration continues and neighbourhoods in the large-urban sector become more ethnically diverse.

He suggested that SAARF undertake a fundamental review of the disproportionate design, with discussions between both RAMS and AMPS users.

• On flooding: With flooding, all adults in the household of the AMPS respondent are given radio diaries to complete. He felt that this methodology does not lead to any substantial loss in the quality of RAMS data. He did find evidence that there is some loss in audience caused by poorer diary completion by secondary respondents, but this is at a relatively low level.

All told, he felt there were no sampling or statistical grounds for rejecting household flooding, and that given its financial attractions, "every effort should be made to retain it."

• On audience-level fluctuations: SAARF and its users expressed concern about the extent to which audiences fluctuate from period to period. He reported that he did not see any unexpected audience volatility in RAMS releases, and that the degree of audience "bounce" seen was not exceptional.

• On updating population estimates: His view is that the current arrangements for the provision and updating of population estimates are of a high quality.

It is his view that there were no viable alternatives available to either SAARF or Nielsen to deal with the two major trend disruptions, which RAMS suffered because of the change of demographer in 2012 and the updates reflecting the new census results in 2013.

• On the practice of rolling data: Rolling the RAMS data does tend to reduce the opportunity for broadcasters to evaluate the effects of new programming, promotions and will also tend to obscure seasonality.

He believes that the key issue is the main purpose of the research - which is as a tool for the trading of airtime. He felt that rolling data should be retained, but ideally with some possibility for broadcasters to be able to look at discrete survey periods.

• On frequency: From his limited enquiries, he did not see any demand for more frequent releases, and indeed agencies might not object to fewer releases. He pointed out that media agencies in the UK have consistently indicated no appetite for RAJAR, the UK's radio audience survey, to be more frequent than quarterly, reflecting the way radio airtime is planned and traded.

• On the practice of combining six-month data for small urban/rural areas with each of the three reporting periods in the following six calendar months, he mentioned two concerns. One, that the published data does not all relate to the specified period and two, that the use of rural data in three successive reports flattens trends within the six-month period concerned, while likely causing audience level fluctuations when the new rural set is introduced.

Nonetheless, he felt that rolling data should be retained, though ideally "with some possibility for broadcasters to be able to look at discrete survey periods." He added that SAARF is not unique in having "to choose between precision in representing specific time periods and publishing results on samples which are sufficient to remove excessive movements in audiences which are unlikely to be real."

• On whether stations' broadcast areas are correctly represented: The issue of station areas is also not one that can be simply resolved, according to him. If precise representation of each station's 'survey area' is the prime objective, the only solution is to adopt a completely new approach, similar to that used by RAJAR, where each commercial station would have to define its own broadcast area.

The country would be split into segments, each of which would have a unique combination of commercial stations. However, this could not be applied to community radio stations. This approach, while feasible, would pose significant sampling, fieldwork, and weighting challenges, and would represent a major change to the survey. Careful thought and discussion with media agencies would be required on the extent of the benefit the change would bring.

Gane's recommendations come at exactly the right time for possible implementation in future surveys. In the interim, however, the industry can rest easy in the knowledge that RAMS is delivering a credible and representative view of radio listening in South Africa.