Mediapulse: Criticism of expert reports

Members of the Swiss regional television association Telesuisse are questioning the foundation's mission. At an extraordinary general meeting on Friday morning, Telesuisse members took note of the results of the audits of the new TV measurement system Kantar commissioned by Mediapulse. Contrary to the announcement of the board of directors of Mediapulse on the correct functioning of the measurement system Kantar, the regional television stations have had to note a further inexplicable, massive decline in viewer numbers, Telesuisse said in a communiqué on Friday. The decline mainly affects the private stations that broadcast in Switzerland, it said. In addition, the system provides only very strongly fluctuating values at the regional level, which are completely insufficient for an actual program analysis.

For Telesuisse members, it was therefore clear that the very expensive new measurement system was not suitable for broadcasters working in the small but complex concession areas, the statement continued. "The general meeting had to conclude that the audience figures published by this inadequate measurement system threaten the economic survival of the regional broadcasters and consequently decided to recommend to its members to terminate their contracts with Mediapulse." In addition, it had instructed the board to evaluate alternatives to Mediapulse's measurement system. Finally, the general meeting had asked its board to intervene with the federal authorities to eliminate the discrepancy between the contracts awarded by Bakom to the licensed stations and the contracts awarded to Mediapulse.

Mediapulse: Release of new television usage data

3 Plus also criticizes expert reports

In the conflict over TV audience research, 3 Plus station boss Dominik Kaiser criticizes the expert reports published by Mediapulse. The review mandate and the review process are non-transparent, the experts are not independent and the two expert reports are not suitable for confirming the correctness of the Kantar measurement data, Kaiser says. Instead of getting to the bottom of the errors and testing the system in ongoing operation and under real conditions, Mediapulse had tried in a non-transparent process to gloss over the deficiencies with the help of superficial, theoretical expert opinions and to whitewash criticism of the system. Mediapulse also blatantly disregarded its own organizational regulations prescribed by the Uvek, which have been in effect since the introduction of the new system, and did not involve the Media Science Commission, the Board of Trustees or the User Commission.

"The Mediapulse experts have compared figures from a Mediapulse study (NES New Establishment Survey) with the figures in the Mediapulse panel. The Mediapulse experts did not check whether the figures from the Mediapulse study were collected correctly and whether these correspond to reality. Unfortunately, figures from the Mediapulse studies are not correct," Kaiser explains. In other words, Mediapulse, the former SRG research service, had it checked whether the content from Mediapulse studies had been correctly transferred to the new Mediapulse panel. Kaiser: "In other words, the experts found - to put it bluntly - that Mediapulse had mostly copied correctly from itself. A poor result." During the review, the experts had not checked the audience figures of the new system, nor had they tested the system in operation. The whole review had been designed to come up with nothing, he said.

"Adherence to international standards in no way guarantees that the TV usage data is actually correct as a result," Kaiser said. Mediapulse also continuously violated international standards ("Ten Principles of GGTAM," Global Guidelines for Television Audience Measurement). The top priority here is absolute transparency and the involvement of the industry in all methodological and technical details of data collection. 3 Plus complains that crucial information about the panel, allocation rules and weightings is still being withheld.

Kaiser also criticizes the review mandate and the review process: "The mandate for the review was not openly defined, and we also received neither access to the review results nor answers to the TV broadcasters' questions until the publication of the expert reports." In addition, an audit of such a complex system would not be possible completely in such a short time, the Mediapulse experts would also confirm this.

Although a representative could be sent to the hearings on the expert reports, he was expressly not allowed to report to anyone internally under the threat of a high penalty (25,000 francs per piece of information). "3 Plus was previously excluded from the expert reports and the hearings.

3 Plus has also received the expert reports only today at noon, at the same time as the media. One had now, after an initial brief analysis of the expert reports determined that:

- the experts have not reviewed any usage data from the current system at all,
- the system has not been tested in running operation under real Swiss circumstances with large TV usage and the very large variety of channels.
- the question of whether the specifications of the Mediapulse study NES (New Establishment Survey), which were to be implemented in the panel, really correspond to Swiss reality, was not clarified (For example: How many IPTV households are there really in Switzerland? Where 3 Plus has access to the data, as with the distribution types via the annual reports of Swisscom and Sunrise, there is an exclusive difference (170,000 households) between the figures in the NES and the devices sold by the companies in reality).

Also explicitly had not been checked:

- Whether all TVs and computers in each metering household are connected and metered with functioning meters,
- Whether the evaluated raw data, which are measured, are correctly assigned in the very complex software system
- The wrong assignment in case of identical programs on several channels,
- The missing or incorrect allocation of the advertising blocks,
- The effects of not taking into account the central criterion of "location size" (urban/rural distribution of measurement households the distribution of the various TV distribution types (IPTV, cable analog, etc.) in the measurement households
- Whether the Kantar measurement data are confirmed by a parallel survey of other households using a different method (for example, interviewing, logging),

Both expert reports themselves emphasize the very limited scope of the review and point out everything that was not reviewed, according to 3 Plus. They also emphasize how complex the Swiss TV market is (many stations, different stations with the same content, etc.) and that it is essential to define additional editing rules and that the use of additional technologies such as watermarking is recommended. According to the 3Plus release, Kaiser also points out that many of the questions posed by the TV stations weeks ago before the hearings remained unanswered by the Mediapulse experts and Mediapulse.

Superprovisional injunction against Mediapulse will not be withdrawn

The super provisional injunction obtained by 3 Plus against the publication of incorrect TV ratings by the Mediapulse Group (and third parties) is still in force. 3 Plus station boss Dominik Kaiser wants to continue to fight for correct TV usage data and for an error-free system, including by legal means, the statement continues. "After the gross errors that have emerged so far, we at Mediapulse are demanding a truly independent review of all open questions about the new system, and one that is well-founded by recognized experts who are familiar with the system and with Switzerland," Kaiser said. In addition, he said, it was now time for all the shortcomings that had already been uncovered to be rectified immediately. For this purpose, 3 Plus will send Mediapulse a comprehensive list of requirements after a thorough analysis of the experts' reports at the beginning of next week.