Daily Express, Better You, The Leadership Factor and Laura Clout: the Yes Minister approach to market research

We were shocked to see a Sunday Express story by Laura Clout, stating that:

Research by natural health firm BetterYou [which sells nutritional supplements] found that more than eight in ten of us do not eat fruit and vegetables in our daily diet.

These is a really striking figure, so we asked BetterYou for the research behind it. They, and their PR agency Lucre Communications, were very helpful with our questions: the research was carried out online by The Leadership Factor, with a total sample of 1,000 adults. However, we found a number of issues with this research. As you’ll see in the above Yes Minister clip, there is a noble British tradition of surveys which reliably give certain answers and – while this may be entirely accidental – The Leadership Factor and BetterYou appear to have followed this great tradition.

The 8 in 10 figure came from responses to the question

Out of the following, what do you mainly rely on to keep you going?

Responses included

– “Lots of water” (24.6%)
– “Endless cups of tea. coffee [sic]” (33.5%)
– “Plenty of fruit and vegetables” (19.7%)

19.7% of respondents picked the latter, and the Express article and BetterYou press release therefore reported that 8/10 people did not include fruit and veg in their daily diet.  This is clearly unjustified: drinking lots of coffee clearly does not prevent one from eating lots of fruit and veg as well.

The BetterYou press release used the same question to report that

Worryingly, only a quarter drink water regularly

However, once again, lifestyle choices such as eating lots of fruit and veg or drinking lots of coffee do not prevent one from drinking ample water alongside this.

Subjects were also asked

When it comes to following the ‘Five a Day’ rule, which of these statements most applies to you?

The only options available to them were:

– “What is ‘Five a Day’?” (4.7%)
– “I know what ‘Five a Day’ is but don’t follow it” (35.2%)
– “It’s too expensive for me to stick to every day” (21.5%)
– “I try to follow it but I/my family don’t really like fruit and veg” (20.4%)
– “I forget to follow it because I’m too busy” (18.2%)

Subjects apparently had no other options: all 1,000 picked one of these options, and no other questions are listed in the spreadsheet reporting the research results. There was apparently no option to say ‘I know what 5-a-day is, and eat at least that much fruit and veg’. In order to generate valid results, it would also have been good to see the survey seeking further information – asking, for example, about subjects’ understanding of what constitutes a portion. It would be most surprising if not one of the 1,000 people surveyed both knew what 5-a-day is and ate that much fruit and veg.

However, despite these serious issues, the press release and Express article used this research to claim that (to quote from the press release, rather than Express)

an astounding 60 per cent don’t follow the Government’s Five A Day guidelines because they either can’t afford to, don’t like fruit and vegetables, or because they are simply too busy

A final point to note is an interesting comment from a market research professional on Gimpy’s blog – claiming that

“in the halls of academia” researchers do not need to be governed by the rules of the Market Research Society? So research from a reputable commercial agency is more likely to have the checks and balances in place to ensure fair reporting

The Leadership Factor – responsible for carrying out this research – is a member of the Market Research Society. The flaws in this survey design are so significant that any competent researcher should have spotted them. I wonder where the Market Research Society stands on such dismal research from its members – and if this Society, along with other reputable market research companies, will speak out against this embarrassingly poor research?

One can also note that academics who have been through the rigorous ethics review process that is generally required for research on human subjects might be surprised to hear about how unregulated they are. I wonder how the Market Research Society’s approach to regulation compares, and what the consequences are for members who fail to ensure fair reporting?

BPSDB

Advertisements

3 Comments

Filed under Current events

3 responses to “Daily Express, Better You, The Leadership Factor and Laura Clout: the Yes Minister approach to market research

  1. That ‘research’ is so staggeringly poor it is difficult to believe that neither the Leadership Factor nor those disseminating their findings spotted the flaws. Bizarre.

  2. weight watcher

    i may be wrong but i read somewhere that the daily express no longer has staff journalists as such for most of the sections of the paper but they just rewrite press releases for the health and living sections.

    it looks like what happened here.

    they wouldn’t care if the story made sense as long as they had a filler and it either upset or scared their readers.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s