The PR Measurement trap

Updated: Apr 2

In the late 1960s, two Chapel Hill, N.C., professors took the budding field of public relations research to another level. A strong correlation (r = 0.967 for those of us keeping score) was found between what 100 residents of this small college town thought were the most important issues in the 1968 presidential campaign and the issues covered in the press.

If the mass media set the agenda for the public, so the thinking went, then published news was a proxy for public opinion.

Agenda-setting theory was born, and by extension a framework for measuring the importance of public relations and the success of PR campaigns -- a foundation that has been in place since the days of three television networks and two metro newspapers per city.

Setting aside for the moment any concerns about correlation as causality, the assumptions behind agenda-setting theory set a seductive measurement trap for PR. In the years since, clip books have given way to sizzle reels and automated online dashboards, but the underlying assumptions remain the same.

Based simply on the volume of news published about a company, PR could claim success in driving awareness with consumers. Taken further, the tonality of the news coverage became a proxy for influence of the media on people's perceptions and behaviors.

What's the rub?

Well for one thing, news volume has always overstated actual exposure to news. As we've discussed, content analysis assumes that every story in every newspaper is read by every subscriber, and every segment of the nightly newscast is seen by every viewer. Not true.

Tonality or sentiment scoring generates statistically unreliable data, with the "best-in-class" natural language processing claiming 80% reliability. That's a problem for businesses with expectations of 95% confidence intervals.

Finally, if we simply multiply volume and average tonality of news every day, agenda-setting theory would posit that we are tracking the reach and resonance of news with people. Not true. We have now run correlations between news streams and survey data for 40 companies in class. The highest correlation? It was typically below 0.20. Weak.

Why? Even if volume and tonality were proxies for reach and resonance of news -- and they aren't -- content data cannot measure consumer recall. In some cases, news has long-tail recall that can extend for weeks, months, and in extreme cases, years. I tracked one story with a half-life of more than a decade among consumers

Furthermore, content analysis by definition is rearward facing. Taking recall out of the equation, news data cannot be trended forward. Today's news cycle does not predict tomorrow's, and cannot be forecast based on yesterday's. But in business, forward-facing time-series data is critical, especially to guide pricing and product placement decisions in a crisis.

As a result, to the extent PR relied on news volume and tonality to gauge success, business communications was boxed in. PR can issue press releases, but cannot control the press.

Years ago, I talked through these media measurement challenges with one of the leading PR researchers, a top executive at a global PR agency. At the end of the conversation, I asked him why continue to hold out content measurement as a proxy for the reach and influence of news. Why set the trap?

His answer was as candid as it was unexpected. "Because PR people buy it. And until they stop buying it, I'm going to keep selling it to them."

That was the end of that.

In the first blog in this series, I mentioned that our journey together will take us off the beaten path. If you are fully invested in existing PR measurement and management approaches, and see no reason to challenge conventional wisdoms, we have reached the end of our journey together. Thanks for traveling this far. Respect.

Still with us? Over the coming semester, we will separate signals from the media noise, and examine some new rules of engagement with the press. We'll explore third rails, long tails and some epic fails. We'll build risk frameworks that provide early warning indicators, gauge collateral damage, and forecast the depth and duration of crisis events. We'll find solace in a miraculous exception that proves the rule.

Ultimately, we'll identify ways to adapt emerging technologies to generate forward-facing, actionable insights, and consider a paradigm shift that redefines communication research and elevates business communication.

Next step: Forging the News Stream.

Back to the Whiteboard.

If you are working for a large public company

and open to working with one of my students

to see how all this works,

feel free to email me at

Copyright Headline Risk LLC, 2021. All rights reserved

24 views0 comments