Latest AMPS offers new technology, weighting

The results of AMPS 2009A (July 2008-June 2009) will be released at the upcoming SAARF AMPS industry presentations, in Johannesburg (29 September, 9.30am), Durban (30 September, 9.30am) and Cape Town (1 October, 9.30am).

Research recommendations implemented

Following the successful audit of SAARF's Radio Audience Measurement Survey (RAMS) in 2006, the SAARF Print Council requested a similar audit of AMPS to investigate whether the survey's results were comparable to globally accepted standards, and whether any changes were needed to fine-tune the survey.

International media research consultant Erhard Meier conducted the audit in mid-2007. His overall finding was that, in comparison to similar surveys conducted internationally, AMPS was a well-designed and executed survey.

Yet he suggested a number of changes, some minor, others more impactful in nature to ensure that AMPS could benefit from the latest methodologies, which were beginning to deliver improved accuracy to media audience research around the world.

SAARF presented Meier's recommendations to relevant stakeholders, including the Print Council, to determine which of his recommendations should be implemented.

The impact of two of the more important recommendations accepted by stakeholders is evident in AMPS 2009A. It is these changes, DS-CAPI and the household weighting, which have ushered in a new starting point for trending across AMPS.

DS-CAPI

The decision to move to DS-CAPI took extensive consultation with the print media, which was convinced this was the right way forward for print media audience measurement. The SAARF Print Council consequently made it a condition of the recent tender that the contractor would use DS-CAPI.

Changing to DS-CAPI ensures that the survey is as accurate as technologically possible and gives the survey the ability to keep pace with future industry requirements.

In the short term, the SAARF Print Council wanted DS-CAPI since title confusion has always been a concern for print and this new methodology significantly reduces the chance of confusion. It can now also measure the growing number of title off-shoots, which previously would not have been included precisely because of the potential for confusion.

In the medium- to long-term, the move to DS-CAPI is necessary as a platform for possible future developments in the print arena. DS-CAPI will allow the survey to handle increasingly complicated readership questions better.

Methodology

SAARF has moved from the CAPI methodology, used since the early 2000s, to DS-CAPI. With CAPI, the interviewer used a laptop to conduct the interview and gave the respondent paper prompt material and black-and-white media show-cards to jog his or her memory.

With DS-CAPI, both interviewer and respondent have a laptop, which offers several major improvements over CAPI:

• The paper show-cards are replaced with screens showing nine, full-colour mastheads at a time. These mastheads are automatically displayed on the respondent's screen at the appropriate time in the interview
  
• Daily and weekly newspapers are no longer measured separately. Daily titles and their related weekend editions are now displayed together. This has substantially reduced the potential for title confusion
  
• Newspapers have also been de-regionalised. All respondents now see the same set of newspapers, no matter where they are in South Africa
  
• For newspapers and magazines, the choice of which nine mastheads to place together on each screen was done in collaboration with the SAARF Print Council and the print media owners. Where titles were similar, with high potential for confusion - such as the house and home or motoring sectors - these would be grouped to help respondents identify which they had actually read.
  
• DS-CAPI is an improvement over CAPI when it comes to shuffling the mastheads: with CAPI, the interviewer did this manually. DS-CAPI has automated this with three interlinking rotations for each interview.

  • First, it rotates between displaying newspapers and magazines first. Second, the order of showing the masthead screens is rotated in either forward order or reverse order. Finally, on each screen, the mastheads rotate on a fixed basis, clockwise around the screen.
    
  • This thorough rotation of mastheads means that each title has an equal chance of being seen in every possible position, thus removing the impact of position on respondents' answers.

• Once the respondent has identified the titles he or she has read in the past six months, the interview moves into a series of readership questions, such as how many issues were read and whose copy they read. (The readership questions remain the same as previous surveys.)
  
• From a fieldwork perspective, interviewers cannot skip any mastheads, and it is easier for them to conduct the interview. Interviewers have also reported that respondents are engaging more with the survey since the introduction of the new methodology, and remaining more focused. A colour masthead is now shown to the respondent on the second screen during these questions - another improvement over CAPI.

Effects on AMPS

DS-CAPI, reports SAARF, has had a significant effect on the data for the print media, resulting in many readership changes. However, what might appear to be significant up or down shifts in readership should not be claimed as such. While there may well have been underlying factors, which influenced these changes, the impact of the methodological change to DS-CAPI outweighs these influences.

The readership figures from AMPS 2009A should not be compared to previous AMPS results.

The DS-CAPI methodology has affected the way media such as cinema, the Internet or outdoor are measured but to a lesser extent. Data for these media are therefore trendable over past AMPS releases. The same is true for TV and radio AMPS, although both these media have their own currency in SAARF TAMS and SAARF RAMS.

AMPS 2009B (January-December 2009) will be the first, full DS-CAPI release, and will therefore mark the true start of the new currency for print.

Changes to the household weighting

AMPS 2009A has also had a household weighting change, as it became clear that the household sampling design, with one adult being selected per household, led to a systematic over-sampling of respondents in small households, and an under-sampling of respondents in large households, relative to the average adult household size.

Smaller households have now been given a lighter weighting, and larger households a heavier one, thus rectifying this problem. The household population has changed from 11.139-million to 12.483-million, better reflecting how many SA households there are.

On a national level, the changed weighting does not affect population variables and therefore has a negligible impact on media. For household variables however, such as electricity, water on the plot and appliances in the home, the weighting creates a new benchmark.

AMPS 2009A should not be compared to previous AMPS releases when it comes to any data based on household population.

AMPS users should also be aware that while the number of households may have gone up in thousands, it could potentially reflect as a lower incidence. For example, in AMPS 2008B (January-December 2008), 8,591-million households, or 77.1% of households, had water in the home or on the plot. With the new household weighting applied, AMPS 2009A shows that while more households - 9,581 million - have water, the incidence has dropped to 76.7% because it has been percentaged on a larger base.

Unlike DS-CAPI, which will only show its full impact from AMPS 2009B, the new household weighting has been applied to the entire survey and will thus have no further impact on surveys.

To register for presentations, go to www.saarf.co.za.