Developments in media monitoring

Automatic media monitoring by computer is also viable, says Lourens, who gave examples of the different types of technologies that exist for media monitoring.

Media monitoring, he said, although not completely in place for all facets of media, is quite viable.

Media monitoring is done primarily for proof of performance purposes, as there are also some potential spin-off benefits. It should preferably be done, he said, automatically by a machine like a personal computer, because machines do not tire and work consistently whilst they are operational.

In the technological world concepts and algorithms like encryption, watermarking and pattern recognition (fingerprinting) are being studied and utilised intensively.

"We have the ever-increasing power of the modern-day personal computer that makes implementation of these algorithms possible by means of their numerical processing speed and efficiency. We also have cellphones which can detect broadcast audio on their microphones, they can take photographs, they can even run applets and determine position by means of built in Global Positioning System (GPS) units."

On yet another side there is the target audience who do not want to be bothered by artefacts in either visual or audio media, and also do not want to be bothered by market research (audience ratings).

Solutions

Certain technologies do exist that offer solutions for the media monitoring needs and market research without upsetting the audience: pattern matching, encryption, watermarking and topic spotting.

Audio Auditing started off as audio media monitoring service business using an audio pattern recognition (fingerprinting) technique. The image or audio track itself is the pattern being looked for. The monitoring system has to be trained beforehand by supplying it with the pattern, and from then on the monitoring system looks for these and only these, and reports any detections.

Although this is viable, said Prof Lourens, problems with this approach are that original image/track may not have the properties to be a good pattern, which may result in false alarms, confusion with other patterns, and possibly non-detections if little impairments make the pattern unrecognisable. Added to that, an image/track can only be monitored if the signature exists already.

Encryption is what Multichoice does with its old analogue and newer digital DSTV systems. The idea with encryption is to hide the content of the message, and only allow paid up subscribers to view the content. This field of general encryption keys, or public and private keys, is irrelevant to advertising, because with advertising the intention is to distribute the message to as many as possible listeners/viewers/readers. Watermarking is the field of adding something to an image/track which is invisible/inaudible for the viewers/audience.

There are two fields of watermarks, said Lourens. One field is that of vulnerable watermarks which would disappear if an image/track is tampered with. This is being used for authentication purposes, which is not the goal of media monitoring. The field of robust watermarks is the one applicable to media monitoring; the watermark must stay detectable while the image/track is a fair reproduction of the original. The watermark is like a bar code which can only be detected by knowing the protocol by which the watermark is recognised.

It is possible to add watermarks to images/tracks in such a way that they are not detectable to the human being, said Lourens.

Because the human eye cannot resolve certain spatial frequencies it seems not to be there, when in fact it is. This creates an opportunity to insert watermarks that computers can see, but human beings cannot see and will not be bothered by.

Watermarks

This means that images (printed media, billboards, and television to an extent) can be watermarked without annoying the viewer, and such watermarks can be detected by image processing systems. Similarly as with the human vision system, the human auditory system (ear) also has exploitable characteristics.

For example, because of the physical nature of the basilar membrane in the ear, a psycho-acoustic masking effect exists whereby small signals close by in frequency to large signals, cannot be perceived. The compression obtained by MP3 compression is the result of utilising this effect to ignore redundant parts of sounds that will anyway not be perceived by listeners. Such frequency components, definitely present but inaudible to the human being, can be used to make an audio watermark.

Also, added Lourens, echoes coming in soon after the original (like in a small room) is being integrated by the human auditory system as being part of the original. Only echoes that come in more than about 20 milliseconds after the original will be perceived as echoes.

This means that watermarks in the form of echoes can be inserted in audio, and will not be perceived by humans, while computers can detect such echoes. Different watermarks can even be layered, for example a watermark in the background music in an advertisement, and another layer to watermark the commercial as a whole.

So audio clips can also be watermarked, and such watermarks can be detected by computers.

Research

For market research it is necessary to know what people listen to or watch. One could put a layered watermark on audio channels, and use a person's cellphone to pick up the audio, decipher the watermark and report back from when to when the listener was exposed to which channels. Again an innovative idea like paying a listener with freecell phone airtime can serve as motivation for the individual to perform the task truthfully but passively and without the need to remember which channels were listened to at what times during the day or night, he suggested.

To be accurate, a television monitoring system will have to use both image watermarks and audio watermarks. The image watermark technology may not be mature enough to handle the varying size, position and aspect ratio of commercials squeezed in somewhere on the television image, but the audio watermark technology is mature enough to stand up to the task of monitoring commercials with audio.

One active field of research in speech processing is that of topic spotting. This is a technique for detecting phrases independent of the speaker, or even to detect a topic from the context of the speech around the phrase without an exact sequence of words being spoken. Although this is still a young technology, prone to false alarms, and demands extensive processing power, it is maturing by the day. So one can safely say that broad topics can be detected by means of modern day speech processing, said Lourens.

Topic spotting technology can be used to detect such live reads, and even to monitor advertisements of competitors without knowing exactly what is being said in such possible advertisements. More research is necessary to mature the technology for this specific application, but results at this stage are encouraging enough to persist with the developments, he said.