Former Visiting Professor Patrick Holford is Head of Science and Education at Biocare so, presumably, they believe that he enhances their reputation and scientific credibility. Holford regularly presents himself as a fearless advocate for scientific accuracy and rigour and the bringer of unpleasant truths. Despite an abundance of evidence to the contrary, he is so convinced of his own competence that Holford regularly accuses others of inaccuracy and questions the integrity of leading researchers such as Professor Carolyn Summerbell (we should point out that Holford was in error, not Summerbell). Holford usually does all this under the guise of a plucky underdog and gadfly to medicine so it’s not surpising that he has recently claimed that “conventional medicine, doesn’t have a very good track record”.
Patrick Holford makes this extraordinary claim because he recommends that readers should purchase some of his subscription services and his Health and Action Plan. He recommends that you should:
- purchase the upcoming new YorkTest product, the GL Check[a] (price unknown at present)
- take the 100% Health Check for a free snapshot of your health profile
- purchase the 100% Health Profile (normally £24.99 but £19.99 to subscribers) to obtain a comprehensive set of results (from the questionnaire), “a detailed report and a precise action plan to follow”
- purchase the monthly subscription to 100% Health and Action Plan for “ongoing support to help keep you motivated and inspired” with a recipe book to meet individual needs, access to specific special reports[b] and email reminders. Available for £14.99 per month or £9.99 for members.
Holford not only wants you to purchase these products because it is money in his pocket, with an additional revenue stream from any supplements that you purchase from him but because that is his mission. Holford has created his own arbitrary health assessment criteria and scoring system.[c] His explanations might provide an excellent example of what Dr Ben Goldacre complains of as unnecessary complication that intimidates people into believing that they need the advice of experts in order to know what to eat. The same experts, we should remember, who in the same newsletter, (wrongly) inform the reader that ‘chicken drumsticks and thighs are leaner and have a lower glycaemic load than chicken breast’.
My aim is to help you achieve a total health score of over 85%…You’ll…receive ongoing personalised information to help you put all your health advice into practice. For example, if you are vegetarian, have poor glycation and oxidation scores, you’ll receive recipes appropriate just for you…[100%health newsletter, Jan 2009, No. 49, pg. 1.]
HolfordWatch has to confess to being a little stumped by this. We can understand how it might be possible to estimate glycation by interpreting a haemoglobin A1c blood test (possibly the YorkTest product that isn’t yet available) but from a questionnaire? It’s not too clear what is meant by oxidation unless this involves a comprehensive cholesterol analysis with a detailed lipid profile. It’s entirely possible that a thorough health profile might involve some costly blood tests although it is plausible that some helpful health entrepreneur might manage to recommend his favourite laboratory for such tests.
Why does Patrick Holford feel that you need to spend money on some as yet unspecified tests and a health questionnaire that has never been validated? What might happen if you ignore his advice?
The alternative, that’s conventional medicine, doesn’t have a very good track record. According to research by the British Medical Journal into some 2,500 medical treatments, only 13% are rated as beneficial. Other research suggests there are an estimated 30,000 deaths associated with prescribed drugs every year in the UK, which is 1 in 20 of all deaths (annual deaths overall is about 600,000). What’s more, our risk of dying prematurely from an heart attack or stroke, or developing diabetes, breast or prostate cancer is about one in two.
It’s not rocket science to see that you are much better off investing in staying informed about the real causes of health and disease, and doing what you need to stay fighting fit rather than drifting towards preventable diseases. [100%health newsletter, Jan 2009, No. 49, pp 1-2.]
Frightening stuff. Hands up if you are surprised by the absence of references for these startling statistics or any framework for interpreting them. When you make an informed guess as to the references, you may well consider that, “It’s a bit more complicated than that, Patrick Holford, however, it is fair to say that you are somewhat distorting the facts”.
HolfordWatch is going to take a punt that these figures for medical treatments are from BMJ’s Clinical Evidence. What does Clinical Evidence actually state?
Of around 2500 treatments covered 13% are rated as beneficial, 23% likely to be beneficial, 8% as trade off between benefits and harms, 6% unlikely to be beneficial, 4% likely to be ineffective or harmful, and 46%, the largest proportion, as unknown effectiveness (see figure 1).
So, yes, “only 13% are rated as beneficial” but that doesn’t actually mean that the remainder are actively harmful as one might reasonably conclude from yoking that figure together with the dire figures from an unattributed anecdote[d] that Holford has used on many occasions. It is possible that Holford has repeated the anecdote so often that it has taken on a degree of authority for him but it still doesn’t make it either accurate or relevant. Where is this fabulous track record that Holford is implying for the nutritional interventions that he is recommending? Let us not forget that Patrick Holford attempted to dismiss the Cochrane Review of antioxidants which covered approx. 230,000 participants in favour of his own anecdote with n=2.
Dr Ben Goldacre provides a context for interpreting such figures in Bad Science .
Another way of measuring is to look at how much medical activity is evidence-based, taking consecutive patients, in a hospital outpatients clinic for example, looking at their diagnosis, what treatment they were given, and then looking at whether that treatment decision was based on evidence. These real-world studies give a more meaningful figure: lots were done in the 1990s, and it turns out, depending on speciality, that between 50 and 80 per cent of all clinical activity is ‘evidence-based’. [Bad Science , pg 182.]
We’ll repeat those last figures: “between 50 and 80 per cent of all clinical activity is ‘evidence-based'”. That sounds rather different to Holford’s summary of the evidence. Andrew Booth has compiled a classic review and resource guide of these “real-world studies” for people who would like to look through the detail. Drs Harriet Hall, Steve Novella and Peter Lipson have all commented on the distortion of figures around the evidence-base for medical treatments and the estimation of iatrogenic harm. Holford v. Goldacre, Booth, Hall, Novella and Lipton. In whom do you repose your trust?
We have no idea where Holford obtained the numbers relating to premature death, diabetes and cancer, but it is sufficient to say that he has been spectacularly wrong about similar numbers on previous occasions. Plus, it is rather a truism, but more men die with prostate cancer than from prostate cancer, so, the relevance of the incidence of prostate cancer rather depends on whether it is progressive or not. Dr Dave Gorski points out that The early detection of cancer and improved survival: [Is more] complicated than most people think.
Sadly, as HolfordWatch has documented on many, many occasions, Holford is not an adequate participant in discussions about research. He is not even an reliable intermediary for disseminating research findings or understanding them appropriately. He should certainly not be accepted at his own valuation as an expert or even competent commentator.
People send us extracts from Holford’s 100%health subscription newsletters and Special Reports because they believe that they prove that Holford is a gifted researcher and that we fail to give him due credit for his scientific rigour. They expect us to read these snippets and collapse sobbing as we are overcome with remorse at our unfounded criticisms of one of the UK’s leading thinkers and scientists.
As for “staying informed” and Holford helping readers…Time and time again, Holford’s scholarship is demonstrated to be shoddy and inaccurate. As Professor Tom Sanders remarked, nutritional therapists tend to “feed on the detritus that comes out from the scientific community” – but it doesn’t mean that they assimilate it correctly.[e]
[a] GL Check, without further information, this test may be as unevidenced and arbitrary as some of the other offerings in the YorkTest portfolio.
[b] We have previously commented on just how out-of-date and inaccurate some of these Special Reports are and the fact that they seem to be very poor value for money.
[c] Holford has a track record for establishing his own levels when current ones don’t dovetail into his supplement recommendations. Holford’s recommendations for homocysteine levels are substantially lower than those of people who have some actual research or clinical background in the topic: he does not give a satisfactory explanation for this although it is serendipitous that by establishing a very low baseline, it may well happen that more people need to take supplements to achieve his recommended levels.
[d] This anecdote turns up on many occasions and is worthy of its own place in that fine journal, Things I heard in the pub.
I met a man who processes drugs statistics for many NHS hospitals and he estimates the number of deaths associated with adverse drug reactions in the UK to be around 30,000 per year. That’s one in twenty deaths! [100%health newsletter, Nov. 2007, No. 42, pg. 1.]
[e] Yet again, we refer interested readers to an overview of the Justin Kruger and David Dunning paper that discusses how difficulties in understanding one’s own incompetence can lead to inflated self-assessments.
What’s even more amazing is that when they then shared the performance of other participants with the people who performed poorly (hoping that they would then adjust their self-perception downward) people who scored poorly failed to adjust their self-perception of their performance. In other words, they are completely unaware of their own [in]competence, and can’t detect competence in others.
It really it a very helpful paper that explains many otherwise inexplicable actions. (Holford Myths offers an excellent discussion as to why pundit brand equity ensures that Holford still receives so much positive attention in the mainstream media.)
A Photon in the Darkness likewise offers a helpful discussion of this paper: The Arrogance of Ignorance.
We do, however, acknowledge that there are some examinations of the variance of ability as a function of ability. It may be true that we all misjudge our ability and task difficulty but some of us seem to be more rigorous in scrutinising the evidence to guard against this and are open to correcting any errors.