Wednesday, November 7, 2007

Junk Research

"A survey reveals ... 22% of women know they're marrying the wrong man on their wedding day ... eight out of 10 children say their parents' choices of Christmas presents are a waste of money ... almost half of people living with home-assembled flat-pack furniture are in danger of falling out of bed."

This is from the opening script to a recent radio programme lamenting the dubious relationship between market research surveys and journalism. Now it might be a bit rich for someone like me working within the market research industry to fret about 'junk research', but this is an issue I've got to take a stand on. I genuinely worry about the long term consequences for both market research as a profession and journalism as a practice because of the growing abuse of surveys for PR purposes.

John Kay has written about this in the Financial Times, and his comments reflect my own views and concerns:

The difference between the bogus survey and real research is that real research has the objective of yielding new information, while bogus surveys are designed to generate publicity. ...

Public relations professionals understand these triggers, to such an extent that commissioning a bogus survey is now a standard element in the pitch they present to potential clients and conducting these surveys is an increasingly large part of the activity of market research organisations. ...

It is called “thought leadership”. That term illustrates the problem. It probably does not matter much that the bogus survey is used to generate spurious news. The danger is that opinion polls designed to produce eye-catching answers displace serious thought and analysis. ...

Newspapers, broadcasters and consultants will start to distinguish bogus surveys from substantive knowledge only when their audience demonstrates that it knows the difference. Academics and think-tanks need to be reminded that generating publicity is not a legitimate research objective.

It is perfectly legitimate for a business, working with its PR agency, to commission market research to explore a relevant issue or trend, and to publish the findings in the public domain as a report. Such a report should be clear about the methodology used, the size and validity of the sample, the statistical signficance of the main findings and only present conclusions that are reasonable given these constraints. This should not deny the PR agency an attention grabbing headline for their press release (that's why they want the research after all!) - but it should mean that any serious probing of the findings by a professional journalist will show that the research (and the headline) are legitimate.

For the record, my own company Amárach has worked on a number of such public domain studies in recent years, and we have found that a rigorous approach to research methodology actually enhances the PR value to the client.

What worries me most these days is the growing number of 'pseudo-surveys' ("a text poll of our listeners reveals ...") that usually amount to little more than a paragraph in a press release, with no substance behind them. Even more alarming is the manner in which many journalists and broadcasters religiously publish the same pseudo-research without any challenge or commentary.

Ultimately it is incumbent on people in the market research industry to do more to promote proper standards in market research, especially research that is published in the public domain. To that end, I would recommend any business or PR agency that wants to avoid the 'junk research' trap to only use legitimate market research agencies that are publically committed to objective quality standards. And in Ireland that means members of AIMRO (the Association of Irish Market Research Organisations). Take your pick (even one of my competitors!) but please lets stop the rot before it does irreparable damage to both market research and journalism.


  1. Blogs are good for every one where we get lots of information for any topics nice job keep it up and Pretty good post, this is one of the best articles that I have ever seen! This is a great site and I have to congratulate you on the content.
    I appreciate it!

  2. This Post gives me lot of information Great to read


Related Posts Plugin for WordPress, Blogger...