Set meters measure whether a television set is on or off, and if on, to which channel it is tuned. They were common when television was analogue. They do not collect person (individual viewer) data. In some markets, they were supplemented by diaries for that purpose. The diary data often disagreed with those of the set meters, causing controversial reconciliation practices. Set meters were expensive because they were custom-made and installed in a labour-intensive way.
Set meters equipped with buttons by means of which sample household members and guests were intended to check in and out of viewing and to confirm viewing periodically. People meters are the main means of collecting data by the providers of ratings currency. A problem since people meters were introduced has been partial non-cooperation by sample household members, especially children, which cannot be easily detected, and its effects, which in most but not all instances amount to an undercount. A more recent and fatal flaw of this technology is that measures only conventional fixed TV sets and only at home, while television is increasingly delivered by new platforms to mobile devices, neither of which are visible to people meters. People meter samples are expensive, limiting their size typically below that required to measure much of today’s fragmented television market.
The means of data collection in many audience measurement services until the 1990s, and still present in some countries for television and more so for radio. The only means of collecting persons data until the introduction of the people meter in that decade. Typically requests the sample member to fill in which channel he or she watched in each 15-minute calendar interval for one or two weeks. Diaries have numerous methodological problems. Because they require the sample member to note his or her viewing or listening contemporaneously, and many sample members are unable or unwilling to do so, they are prone to noncompliance (large percentages of diaries are usually filled in ahead or well after of viewing or listening). Unable to capture the relatively short viewing events of minutes or seconds of which modern real-time television viewing consists. In practice, require sample members to recall their viewing after it occurs, thereby favouring the major channels that have higher name recognition. Impossible to use to measure timeshifted viewing. Diaries are still used in some radio measurement services because they are portable and inexpensive.
Phone surveys, including CATI (Continuous Automated Telephone Interview)
Phone services were used in some television audience measurement services and CATI remains one of the dominant methods of data collection (alongside diaries) in radio. Have a very poor level of recall even when performed at the end of the day, and even in comparison to diaries.
A very rarely used data collection method. No data is publicly available on its accuracy. However, the expectation that a sample member will respond, especially over an extended period of sample membership, would seem to cause a particularly high degree of self-selection, which, although present in any data collection method, is regarded as contrary to sample representativeness and subject to reduction as much as possible.
RPD (return path data)
Data from pay-TV operators’ set-top boxes (STBs). Many such systems are cumbersome (the software is integrated into the STB and may require retesting with every STB firmware release; it is also usually heavily dependent on other systems) and expensive (some providers abuse their monopolistic control of the STB firmware to charge high prices). At many pay-TV operators, only an unrepresentative cohort of STBs is equipped with a return path, or none at all. At many operators, only some of the STBs in a household can transmit measurement data. The processing of RPD data typically ignores these problems and such systems are rarely audited. Furthermore, even in the best RPD installations, individual viewers cannot be identified but only inanimate screens. Of all demographics, usually only addresses and IP addresses are available, which can be insufficient to determine even the household’s socioeconomic status. Finally, only viewing of operator-provided content on STBs is visible (and requires effort to measure DVRs and operator-provided VOD), not viewing on mobile devices or of standalone VOD or OTT services.
Connected (smart) TVs
Measurement of content that is viewed on a &ldquot;smart&rdquot; or connected TV by means of ACR fingerprinting. Can measure integrated OTT services. Enables very large samples (more than 15 million such devices are available in the United States). However, substantial amounts of viewing, notably on mobile devices, are invisible to such measurement, as is any viewing on other TV sets (which can be most of the TV sets in the household) or outside the home. As in RPD systems, no information is collected on the persons using such devices, and even the approximate address of the residence used to provide socioeconomic context to RPD data isn’t available (the IP address visible to the manufacturer can denote a diverse geographic area). These limitations make such measurement useful only for addressable advertising displayed on the measured screens.
Mobile device-based measurement using ACR fingerprinting and/or watermarking
There are several implementations, almost all of which have one or more of the following disadvantages:
A custom device is used instead of a smartphone. Because the measurement device is not otherwise necessary for everyday life, the sample member often does not carry it, resulting in either low cooperation or high expenses on panelist compensation. Custom devices manufactured in low volumes also have high unit costs.
Watermarking is used either instead of or in preference to ACR fingerprinting. Watermarking typically requires encoding at content output, to which broadcasters, channel operators, pay-TV operators and OTT services are resistant as it complicates their plants, introduces points of failure, and is usually expensive. It is almost impossible for a noncurrency ratings service to compel broad compliance with a watermarking requirement.
The ACR implementation is of low quality—rather deaf. Many of the best-known ACR services do not have very high recognition rates.
The mobile app has poor stability or survivability (the ability to continue measurement reliably during phone sleep). Both problems cause low cooperation.
The processing of data received from the app does not handle such data correctly or otherwise violates industry standards. Data from a mobile app must be processed extensively to be meaningful, much more so than data from people-meters or RPD.
Most or all such services are quite expensive, with high base and sample-size-dependent costs.
ACR fingerprinting is the only one that can measure media use on any playback device, at home and out of home, mobile or fixed. When based on a smartphone with a high-reliability app and competently designed processing system, and offered at an affordable price, it approaches the ideal of audiovisual audience measurement. As far as we know, the only such system available today is Alldience