Patrick Holford, YorkTest and a Migraine Study

Former Visiting Professor Patrick Holford but still Head of Science and Education at Biocare has an enthusiasm for interventions that we wish we might share but, too frequently, when we examine the studies on which he relies, the results do not support his claims or the scope of his optimism.

In a recent newsletter, Patrick Holford was rather excited about a recent study: Randomised controlled trial of food elimination diet based on IgG antibodies for the prevention of migraine like headaches. For reasons that may become clear, he neglected to mention the full title of the paper.

Last month a study published in the Nutrition Journal showed that 85% of people affected by debilitating migraines had their symptoms reduced, and quality of life improved, when their food triggers were discovered and avoided. Having a hidden food allergy is one of five common causes of migraines.

This recent study tested migraine sufferers for food allergy using YorkTest’s FoodScan test. Eighty four of the volunteers were put on their food allergy free diet, while 83 were given a sham ‘allergy’ diet based on fake test results. At the end of four weeks those on the real allergy free diets had had 23% less migraines than those on the sham diet.

You might be a little surprised that Holford was quite so excited by this result after 4 weeks of assessment that the newsletter was subject-lined, “Relieve your migraine without painkillers” and headlined, “What’s causing migraines?”. Holfordwatch consulted the original study and can not agree that Holford provides a useful interpretation of its outcome.[1] According to the reported results, it seems as if even the study’s authors might quibble with Holford’s optimism:

The results indicated a small decrease in the number of migraine like headaches over 12 weeks, although this difference was not statistically significant (IRR 1.15 95% CI 0.94 to 1.41, p = 0.18). At the 4 week assessment, use of the ELISA test with subsequent diet elimination advice significantly reduced the number of migraine like headaches (IRR 1.23 95%CI 1.01 to 1.50, p = 0.04). The disability and impact on daily life of migraines were not significantly different between the true and sham diet groups. [Emphasis added.]

There are many other problems with this paper (eg, the participants are effectively self-selected from a group that is pre-disposed to believe that food intolerance influences migraine; there is no clinical verification of the migraine-like headaches description; respected experts in allergy and immunology caution against the notion that Yorktest’s IgG Food Intolerance test is diagnostic of food intolerance or clinically relevant; the number of study drop-outs compromises the power of the study effect). The study lacks scientific rigour to the extent that the only surprise is that a journal reviewer changed his opinion between 1st and 2nd review:

This paper has strong deficiencies in respect to the study design, recruitment, compliance, and no medical control and assessment of the subjects, not meeting the criteria for a scientific paper. Due to the huge amount of uncertainties, also acknowledged by the authors, this paper has no new information to offer and is of limited interest.
Level of interest: Reject as not of sufficient priority to merit publishing in this journal. [1st review]

Such flaws in the design and other areas can not be corrected merely by re-writing yet the reviewer accepts the revisions (eg, the revised title now refers to “migraine-like headaches” rather than “migraine”) and changes the review comment to:

Level of interest: An article whose findings are important to those with closely related research interests. [2nd review]

Recall that the research finding is not, as Holford leads, “[You can relieve] your migraine without painkillers” but that the study revealed that at 12 weeks (the study’s stated primary outcome): “this difference was not statistically significant“. Holford’s account is partial and inaccurate. This is lamentable when one considers that he claims to be a valuable intermediary between the public and the practical reporting of health research. It is unsettling when one considers that there is considerable apprehension that some patients and healthcare providers might be persuaded by such claims to lobby for such ineffective tests and diets to be funded by the NHS despite their lack of clinical relevance or efficacy.

Notes

[1] Holfordwatch notes that Yorktest provided a similarly partial account of the study findings in their September 2011 news items and that the media that ran the Yorktest release on this study did not investigate the findings but reproduced this and other unduly favourable interpretations (the rollcall of shame includes: Metro, Red Online, Top Sante, Women’s Fitness, Women’s Weekly, OK Magazine, Woman’s Weekly, Female First.

Leave a comment

Filed under food intolerance, IgG tests, intolerance, patrick holford, yorktest

Oxford University, Food for the Brain, Alzheimer’s Disease and a Curious Test

The People’s Medical Journal (aka Daily Mail) has a touching faith in the value of early diagnosis and screening tests. It would be rather charming to note that their history of being wrong has not as yet reduced them to cynicism if it were not for the errors and false hope that they present as verified fact to their readers.

Food for the Brain (CEO Patrick Holford) claims to offer an online test to detect the early symptoms of Alzheimer’s Disease. Alzheimer’s Disease is a subject that excites great concern and, by definition, those who research in this area are aware that they are typically dealing with some vulnerable people.

Dr Margaret McCartney recently examined the claims for Food for the Brain and self administered cognitive tests after some enthusiastic media reports and found them unwarranted, and not in line with available evidence. She evaluated the claims again in the BMJ, An early warning for Alzheimer’s disease, and questioned FFTB about the claims made for the test in the diagnosis of Alzheimer’s Disease or its prodromes and the evidence base for its recommendations.

Patrick Holford, who describes himself as a “nutritionist” and chief executive officer of Food for the Brain, told me, “We, the charity, deemed the evidence to have become substantial enough to warrant the launch of our Alzheimer’s prevention project . . . the primary aim of which is to encourage early screening of cognitive function from age 50, followed by homocysteine testing.” Food for the Brain’s adviser, the pharmacologist David Smith, told me that the online test is “not a diagnostic test, and there is no definitive outcome; it simply tells the user about their cognitive status.

So, media coverage (eg, 15-minute online test for dementia: DIY memory quiz detects early signs of Alzheimer’s in people as young as 50; Online test for early signs of Alzheimer’s) persuaded people to take an online Cognitive Function Test (CFT) developed by an “Oxford research team” as a way of detecting the early symptoms of Alzheimer’s Disease and as a way of persuading them to take/pay for non-evidence-based tests and supplements. However, Professor David Smith, Chairman of FFTB’s Scientific Advisory Board, admits that the test has no diagnostic value. (Holfordwatch readers with a good memory will recall that David Smith has previously admitted that FFTB has not done a “proper job” of research it attempted.)

As is too frequently the case when discussing Patrick Holford and Food for the Brain, it is difficult to outline all of the misunderstandings and errors that accompany their claims. We can’t begin to cover all of the issues which, inevitably, also involve: the inappropriate promotion of the crystal-ball of homocysteine testing as a biomarker; the advice that test takers should ask their GPs for a test that is not available for that purpose on the NHS or have recourse to Yorktest private testing; the promotion of supplements. This is not the time to explore the ethical concerns that must accompany the availability of a direct to consumer test that purports to diagnose such a widely-feared condition and has already caused some distress while also illustrating a worrying (and perhaps unwarranted) confidence in the significance/value of the test.[1]

The following is not a complete account as the story is still unfolding, however, even these items highlight the contested nature of this test and why the marketing/media coverage of it is inappropriate. The inconsistencies and recent redactions must also question whether the test ought still be available until such time as various issues are clarified. It isn’t clear why Oxford is failing to protect its reputation as it can not be to its advantage to be associated with such a questionable test and set of recommendations.

i) Mid-May various media outlets covered FFTB’s Cognitive Function Test (CFT): they reported that it had been developed by an “Oxford research team” and was made available online, direct to consumers, and promoted as diagnostic of Alzheimer’s Disease and its prodromes.

Food for the Brain (FFTB) emphasised the involvement of Oxford University in news stories and the then current version of its own website: Patrick Holford has likewise stressed the involvement of Oxford Uni. in his marketing materials for his own website and for FFTB for which he is the CEO. Oxford researchers are said to have played the role of lead developer.[2]

However, approaches to several people, including Virginia’s Professor Timothy Salthouse and Oxford’s Dr Celeste de Jager, subsequently revealed that neither of them played such a substantial role in the development of the test.

Professor Salthouse reports that he granted permission to the authors to use an adapted version of his perceptual comparison tests. However, he emphasises that that is the extent of his involvement and makes no claims with respect to the role of his tests in assessing the risk of memory decline or the development of Alzheimer’s Disease.

Dr de Jager’s involvement was recently clarified by an amendment to the FFTB website:

The CFT composes three elements:
A Episodic memory, using cued recall and paired associate learning test constructs, developed by Catharine Trustram Eve for FFB, with the advice of Dr Celeste de Jager.

So, it seems as if the test, developed by an “Oxford research team”, was substantially the work of Catharine Trustram Eve who is listed as an “Independent Market Research Professional” albeit that is not made clear in the Letter to GPs that test-takers are advised to give their doctors. Catharine Trustram Eve’s profile does not list any qualifications in neurology, psychology, cognitive science or similar relevant disciplines.

How many test-takers took the test because they were reassured by the much publicised Oxford provenance of the CFT? What, if anything, is Oxford doing to dissociate itself from the CFT? Even today, the Daily Mail is linking Oxford and this test: Test to detect early onset of Alzheimer’s for all over 65s to be introduced within two years.

One of the items in the box insert states:

“Oxford University has devised a memory test that can be taken at home in 15 minutes and can spot the signs of Alzheimer’s in people as young as 50.”

Both Patrick Holford and Food for the Brain promoted the CFT to their mailing lists. Why have neither Patrick Holford nor Food for the Brain issued corrections to their mailing lists to clarify the provenance of the test?

ii) The CFT is said to be validated for a specific age range and it is promoted as diagnostic in news coverage and on the FFTB website.[2], [3] However, as Dr McCartney explains, at the time of the news items, the validation for this test was not available and there were no data relating to sensitivity or specificity (false positives and false negatives).

The Daily Mail story (inter alia) refers to pilot studies:

The researchers cannot put a figure on the test’s accuracy, but in pilot studies it worked as well as tests already used in GP surgeries and specialist memory clinics.

In the absence of detail that would allow for appropriate scrutiny, the claims for validation were premature at best. Irritatingly, in late May the FFTB site was amended to state:

“based on the pilot, it appears that the CFT is sensitive to MCI. A full description of the pilot and analysis will be available from this page by 1st July 2011.”

http://www.foodforthebrain.org/content.asp?id_Content=1825

However, following the latest updates to the FFTB website, we now learn that we are not to be permitted to see the detail that ‘validates’ these tests for some time:

“Previously it was stated that ‘A full description of the pilot and analysis will be available from this page by 1st July 2011.’ However, in light of an expert academic critique of the CFT validation, we have decided to submit the work for publication prior to publicising the results on the website.”

http://www.foodforthebrain.org/content.asp?id_Content=1825

Prof. Salthouse’s contribution to the test is well validated within its usual sphere of use. Prof. Salthouse makes no claims for the integration of his test within the CFT and states that his comparison tests should not be treated as valid predictors of the risk of memory decline or the development of Alzheimer’s Disease. Under the circumstances, it is arguably imperative that the pilot studies and materials that underpin the “validation” of the CFT should be made available. However, the absence of accessible validation is not made clear to the public, nor, perhaps, to some NHS Commissioners who are given funding proposals for which there has not been adequate due diligence.

iii) Media coverage, Patrick Holford and the FFTB promoted the CFT as diagnostic of Alzheimer’s Disease or its prodrome when the CFT has not yet been publicly validated as an appropriate instrument to identify or quantify mild cognitive impairment.

David Smith modified those claims when challenged by Margaret McCartney. FFTB updated its website July 1 and, in line with Smith, has considerably modified its claims for the scope of the CFT:

“Does the CFT diagnose dementia, Alzheimer’s disease or Mild Cognitive Impairment?
No. The CFT is not a diagnostic test, but a test designed to inform/educate the user about their cognitive function. If the result is below a threshold we suggest that they visit their GP who can perform whatever diagnostic tests are required at their discretion.

However, both Patrick Holford and Food for the Brain promoted the CFT to their mailing lists: the former was headed, “15 minute free test to prevent dementia” and the latter “A 15 minute free test could stop you ever getting Alzheimer’s”. Why have neither Patrick Holford nor Food for the Brain issued corrections to their mailing lists to update them as to the reduced scope of claims for the test?

Allegedly, many people have taken this test. Perhaps the test-takers were reassured that it was developed at Oxford University, and that it is a validated test for Alzheimer’s and its prodromes. Patrick Holford claimed (in a later deleted blog post) that 55,000 people took the CFT in 10 days. Since then, FFTB has claimed that more than 70,000 people have taken this test.

A test that plainly was not developed by an “Oxford research team”. A test that is now said not to be a test for what people were told that it was but now “simply tells the user about their cognitive status” albeit the validation for that is not available.

Why hasn’t Oxford contacted media outlets such as Daily Mail and Telegraph to instruct them to correct their stories if Oxford’s involvement is as limited as the current version of the FFTB website implies? Were the pilot studies on which the ‘validation’ rests conducted at Oxford, and, if so, did Oxford oversee their clinical governance?

Why haven’t Patrick Holford or Food for the Brain alerted media outlets that they have modified their claims concerning: the provenance of the test; the scope of the claims for the CFT’s diagnostic purpose; and that the claims for validation ought to be held in abeyance until such time as the details are published?

Why haven’t Patrick Holford or Food for the Brain alerted their mailing lists as to these substantial revisions concerning the CFT? Has there been any attempt to contact GPs who’ve received one of these letters from a patient to inform them of the modified status of these claims? (The GP letter is still available on the FFTB site and still contains claims that are out of date.) If not, why not?

The CFT has created anxiety amongst some users and has given false confidence to others. Rather curiously, FFTB brags of the thank you letters it has received although it now seems as if they were being thanked for reassurances that can no longer carry any weight given the modifications and reduced scope of the claims for that test.

A curious test and a curious business. There will be more to come when more information is available about the ethical approval for this test and other pertinent matters.

Notes

[1] Sample quoted from

http://www.womanandhome.com/forums/showflat.php/Cat/0/Number/726038/Main/725925/

[Northwindrider] Mine said I was at low risk of developing Altzeimers which is quite comforting as I have a Grandmother with it and know that my Mum was in the early stages when she died.

[snowy47] I have completed the test and i have a very low chance of developing Altzeimers, my D passed last year and he put my M through hell with it.

[susieblue] I have just done the test and told I was low risk. But I am pretty sure my mother would have been told the same had she done it at my age too. Read an article about it on Yahoo. Complete rubbish! For starters my mother, aunt and uncle all had/have it.
[aec13cat] Was so curious in the end I took it but it shows I’m at risk -totally depressed now and wish I hadn’t taken it

Following from

http://forum.alzheimers.org.uk/showthread.php?33976-Someone-look-at-this-for-me-please

[Danny] I took the online test myself,I scored 37/110. It was a bit worrying to be told I could be at serious risk of developing Alzheimers. It has worried me to bits.It will teach me to stop researching so much.

[Tony] I just took that test and scored 29 yesterday
had my 6 monthly test for memory clinic scored 30/30 now I’m confused with the results was the memory clinic test 2 easy

[Gill66] I am suitably happy with a score of 88. Use a mouse, it’s a lot easier. With a history of dementia in the family i feel quite content this evening

Quoted from Patrick Holford’s blog:

“The positive response to my test results came as a great relief, as my father, uncles and paternal grandfather all developed symptoms of senile dementia of one form or another, when they were precisely the age I am now. I feel a great weight has been lifted from my shoulders, at least for the foreseeable future.” Yours, Hugh G. “A really informative website – and the opportunity to put my mind at rest by doing the cognitive test was priceless. I found the test itself very well introduced and explained. The examples are particularly helpful. I’m sure I’m not the only one who approached the test with some anxiety but I found I was far less panicked than I expected.” said Marion. Ivor, age 75, said “The nightmare of Alzheimers has been put to sleep by the results of this test. Thank you.”

http://www.patrickholford.com/index.php/blog/blogarticle/951/ – if no longer available, please see

http://www.freezepage.com/1307103965TWDDWXGIGE

It is worth noting that Ivor is outside the age range for the test yet Patrick Holford nonetheless includes this testimonial.

[2] “The Cognitive Function Test assesses three critical areas of cognitive processing associated with cognitive decline leading to Alzheimer’s disease…This test has been developed in collaboration with Dr Celeste de Jager from the University of Oxford, Professor Timothy Salthouse from the University of Virginia and Catharine Trustram Eve.”

Original link for text: http://www.foodforthebrain.org/content.asp?id_Content=1824

Freezepage for the page May 20 2011: http://www.freezepage.com/1305892729EKYIERRJCR

A similar claim is made in the results letter than test-takers are advised to give their GPs:

“Your patient has completed the Cognitive Function Test at http://www.foodforthebrain.org, an educational trust whose mission is to promote the link between mental health and nutrition. This is a validated screening test for those aged 50 and above, designed to detect early cognitive impairment. This test has been developed with Professor Timothy Salthouse and Dr Celeste de Jager, specialists in assessment of cognitive function.”

Original link for text: http://cft.foodforthebrain.org/doctors-letter-r.aspx?name=Patrick%20Holford&dob=4/5/1953

Freezepage for the page May 20 2011:

http://www.freezepage.com/1305892790UAAUYMCITZ

Daily Mail and other accounts credit “Dr Celeste de Jager, [as] the main developer of the Cognitive Function Test”.

http://www.dailymail.co.uk/news/article-1386912/15-minute-online-test-dementia-DIY-memory-quiz-detects-early-signs-Alzheimers-people-young-50.html

Patrick Holford blog post 15 Minute Online Test for Dementia:

“The test, available from http://www.foodforthebrain.org, also tells you how to delay memory decline and possibly reduce Alzheimer’s risk, based on research of people with mild cognitive impairment, the stage before Alzheimer’s, by Oxford University experts Professor David Smith and Dr Celeste de Jager.”

Freezepage for the page 20 May 2011: http://www.freezepage.com/1305892842FQLAHXZRQC

[3] “The Cognitive Function Test assesses three critical areas of cognitive processing associated with cognitive decline leading to Alzheimer’s disease…The test has been validated for the age range of 50 plus. ”

Original link for text: http://www.foodforthebrain.org/content.asp?id_Content=1824

Freezepage for the page May 20 2011: http://www.freezepage.com/1305892729EKYIERRJCR

The Daily Mail account of the CFT (in common with the Telegraph and other news outlets) explicitly claims that the test can detect early signs of Alzheimer’s which implies that the test is diagnostic.

“An early warning test for Alzheimer’s that can be taken online in 15 minutes has been developed by British scientists.
It can spot signs of the debilitating brain disease in people as young as 50.
The computer-based interactive quiz provides an instant result and could help delay or prevent the condition by advising simple diet and lifestyle changes.”…
“But most are still in the early stages of development and none, other than the new Cognitive Function Test, which has been devised by Oxford University scientists, can be taken online in the comfort of a person’s own home.
This is likely to make it popular with those who fear their memory is failing but are too embarrassed to discuss their worries with their doctor.

[B]ecause apparently healthy people have no way of telling if they are among those who could benefit from the vitamin B memory boost, the Oxford research team created the test.
It measures mild cognitive impairment – or the slight memory lapses that can be a precursor to Alzheimer’s – which affects one in six aged 70-plus, or 1.5million Britons. Half will develop dementia within five years of diagnosis.”

http://www.dailymail.co.uk/news/article-1386912/15-minute-online-test-dementia-DIY-memory-quiz-detects-early-signs-Alzheimers-people-young-50.html

4 Comments

Filed under Alzheimer's, Food for the brain, patrick holford

Homocysteine, B Vitamins, Brain Atrophy and Mild Cognitive Impairment

A new PLoS article has been published, arguing that

The accelerated rate of brain atrophy in elderly with mild cognitive impairment can be slowed by treatment with homocysteine-lowering B vitamins

I don’t have time to deal with this in as much detail as I would like (lots of things getting in the way of blogging, which is why it has been quiet here lately) but I think this is worth some quick notes. This post is short and a bit messy: for a summary of what the article does and doesn’t know, see Behind the Headlines; for a summary of some concerns with it, see Evidence Matters. Keep reading if you’re interested in the Holford angle. Continue reading

1 Comment

Filed under hometesting, homocysteine, yorktest

The Independent confuses algal DHA with fish oil

In an Indepedendent article arguing that science journalism “standards are pretty high”, Jeremy Laurance discusses Goldacre’s critique of Denis Campbell’s recent Observer piece on DHA and children’s concentration. Laurance describes a trial which

showed that the fish oil “enhanced the function of those brain regions that are involved in paying attention”, as revealed by a brain scanner.

However, as Goldacre noted

It wasn’t a study of fish oil…but of omega-3 fatty acids derived from algae

Nom, algae. Continue reading

2 Comments

Filed under patrick holford

UPDATED: Observer plugs fish oil for concentration, ADHD and depression

I was surprised to see the usually excellent Guardian Science tweeting that “Fish oil helps schoolchildren to concentrate”. This linked to Denis Campbell’s Observer article, reporting that

Fish oil helps schoolchildren to concentrate
US academics discover high doses of omega-3 fish oil combat hyperactivity and attention deficit disorder
Children can learn better at school by taking omega-3 fish oil supplements which boost their concentration, scientists say.

Boys aged eight to 11 who were given doses once or twice a day of docosahexaenoic acid, an essential fatty acid known as DHA, showed big improvements in their performance during tasks involving attention.

Dr Robert McNamara, of the University of Cincinnati, who led the team of American researchers, said their findings could help pupils to study more effectively and potentially help to tackle both attention deficit hyperactivity disorder (ADHD) and depression.

Unfortunately, the Observer’s claims about fish oil are not evidence-based. Continue reading

7 Comments

Filed under ADHD, depression, fish

Nature Publishing promote bad autism science

UPDATE: Nature Publishing Group now appear to have removed the links criticised in this post and the page looks to be much improved (although see the additional criticism at LBRB)

Nature Publishing’s Publishing Group’s (NPG) Scitable has previously been a fairly good example of accessible online science information. However, as Kev has noted, Nature’s NPG’s autism Scitable is well below their usual standard:

who thought it necessary to link to no less than three anti-vaccine links on the home page of this….blog? Wiki? Two links to Autism Speaks whose controllers recently attended a DAN! conference and one link to ARI itself.

Nature also links to Thoughtful House (the US autism clinic where Wakefield was based) when “Explaining the rise of reported cases”. Classy.

Particularly interesting – ‘interesting’ in the banging-head-on-desk-sense of the world – is the Autism Research Institute document they link re the Nutritional Treatment of autism Continue reading

9 Comments

Filed under autism, autistic spectrum disorders

BBC Grow Your Own Drugs: implausible and risky advice

I have been disappointed with some of the content of the BBC’s Grow Your Own Drugs: in particular, its discussion of the use of turmeric and willow bark. I have therefore submitted two complaints to the BBC, and will explain my concerns in this post.

The programme suggests a daily dose of turmeric tea for arthritis treatment: arguing the curcumin in this may be beneficial. Sadly, though, this treatment is simply implausible: curcumin’s bioavailability is poor and (even if black pepper can improve bioavailability) it seems impossible that a daily cuppa would give enough of a dose to do anything particularly useful. I pointed this out to the BBC, and they responded to me:

While we appreciate your concerns, it’s always been the case that James Wong doesn’t believe that natural remedies are a replacement for conventional medicine, and he reminds viewers of this during the series. The programme’s website also explains this

While I am delighted that the BBC make clear that implausible treatments shouldn’t be used to replace actual medicine, it is nonetheless unhelpful for them to suggest implausible treatments in the first place. I have therefore asked them to consider my complaint again. Continue reading

8 Comments

Filed under curcumin, turmeric