Jerome Burne and Bio-Identical Hormone Replacement Therapy: Part 2

Jerome Burne is co-author of Food Is Better Medicine Than Drugs (FIBMTD) with Former Visiting Professor Patrick Holford. FIBMTD has a chapter on Balancing Hormones in the Menopause -The HRT scandal vs natural control: there is a brief discussion of “Natural progesterone – a safer way with hormones”.

Progesterone is given in amounts equivalent to that normally produced by a woman who is ovulating (between 20 and 40 mg a day) and, unlike oestrogen or synthetic progestins, it has no known cancer risk – in fact…quite the opposite. [pg. 167, the reference for this bold assertion is a self-help book, not a journal paper or similar, if you were curious. And, no, no specific page reference or indication that this is a study/trial, in vitro, in vivo or animal.]

Mid-May we noticed that Burne had left a long comment, recommending his own research, on a post about The Alternative that Isn’t: Bioidentical Hormones at Science-Based Pharmacy. Gazing into our crystal ball, we anticipated that a Burne special on the topic must be in progress and so were not surprised to read today’s Should middle-aged women be taking natural HRT? in the Daily Mail. The shorthand version of the remainder of this post is:

No. Not if you are relying upon the Holftorf review to provide a comprehensive overview of the relevant evidence on efficacy and safety.

In Part 1 of these observations on Burne’s article we mentioned that Burne relies upon Dr Kent Holtorf’s “major review” (Are Bioidentical Hormones (Estradiol, Estriol, and Progesterone) Safer or More Efficacious than Commonly Used Synthetic Versions in Hormone Replacement Therapy?) for which Holtorf declares there he has no conflict of interest although he is part of a practice that promotes bio-identical hormones and he offers seminars and physicians’ training in the intervention.[a]

We decided to take a closer look at the paper. We repeat the disclaimer and statement of interest from part 1. This post isn’t about bio-identical hormones or menopause. We are interested in the scholarship of the Holtorf paper and Burne’s use of it and its influence on the balance of the reporting in Burne’s article. We note that the more recent version of the article are better than the earlier version.

Burne relies quite heavily upon Dr Kent Holtorf’s Are Bioidentical Hormones (Estradiol, Estriol, and Progesterone) Safer or More Efficacious than Commonly Used Synthetic Versions in Hormone Replacement Therapy?

Burne writes:

But do bio-identicals actually work? Last February, a major review of more than 200 studies on bioidenticals in the Postgraduate Medical Journal concluded that they were more effective and had greater health benefits than regular HRT.

‘Many physicians state there is no evidence they are safer,’ says the author Dr Ken Holtorf, ‘but the medical literature clearly shows they are highly effective and have some distinctly different, often opposite, physiological effects to regular HRT.’

Scott of Science-Based Pharmacy accurately described this article and others like it:

A close analysis reveals that the evidentiary base relied upon is weak. A consistent theme is the cherry-picking of supportive studies, and the extrapolation of weak data into broad elaborations of the safety and effectiveness of BHT.

Anybody who has taken even a cursory glance at a Cochrane Review or a good-quality systematic review or meta-analysis is aware of just how laborious the process of such evaluations can be. So, it was a little unusual to see that there is a sole author for something described by Burne as “a major review of more than 200 studies on bioidenticals “. Albeit, with reference to Patrick Holford and his oft-cited-as-a-mark-of-quality ‘hundreds of references’, citing references is no guarantee that they are i) accurate; ii) reported appropriately; iii) correct citations; iv) actually exist; v) relevant.

The abstract looks promising.

Published papers were identified from PubMed/MEDLINE, Google Scholar, and Cochrane databases, which included keywords associated with bioidentical hormones, synthetic hormones, and HRT. Papers that compared the effects of bioidentical and synthetic hormones, including clinical outcomes and in vitro results, were selected.

However, it all became reminiscent of a cargo cult exercise[b] when we realised that:

  • Holtorf either failed to provide an adequate description of his search terms and strategy in his methodology or failed to run some obvious variations and therefore may have missed some relevant papers
    • Eg, Holtorf states in the methodology that he searched on the term bioidentical hormones. 7 July 2009 at 15:30, on PubMed the term bioidentical hormones returned 21 hits and bio-identical hormones returned 10 hits: none of these papers appeared in both lists
    • Interestingly, one of the papers that Holtorf does not include in his review (it was in the bio-identical hormones hits) is the critical discussion, FDA warns claims for pharmacy-made “bio-identical” hormones are misleading

      Among the false or misleading claims the agency takes issue with are the description of these products as “bio-identical” implying that these products are natural or identical to hormones made by the body. Additionally, the agency has warned the pharmacies to stop making unsupported claims. Such claims assert that these BHRT products are safer or more effective than FDA-approved hormone therapies and that they can treat or prevent conditions such as Alzheimer disease, stroke, heart disease, or cancers.

      Some of these deprecated claims are ones that Burne discusses in his article, albeit, he seems to be impressed by them.

  • Holtorf neglected to detail his inclusion and exclusion criteria
  • Holtorf did not report how many hits he had collected from his search, how many were duplicates, how many were trials, peer-reviewed, published in full, abstracts, brief communications etc.
  • Holtorf seems to have been the sole arbiter for selecting or excluding papers. Usually, this is done by a minimum of 2 researchers who initially work independently and
    • agree the relevance of a paper from its abstract
    • obtain the full text (where available)
    • re-assess the relevance and then decide whether to include or exclude the paper from a review, based on clearly-defined criteria
    • where the 2 researchers do not agree, the disputed papers may be submitted to a 3rd party who will make the inclusion/exclusion decision.
  • Holtorf does not list any excluded papers or explain their exclusion
  • Holtorf does not consistently assess the quality of the studies (eg Jadad score), whether interventions were randomised, any blinding, the use of a control group or placebo or even the nature of the studies (in vitro, in vivo, animal, human)
  • Holtorf does not sufficiently distinguish the papers that he discusses so it is difficult to tie up his interpretation with the paper in question and establish whether the interesting results are from humans or human epithelial cells in a petri dish or rats, whether the therapeutic interventions were short or long-term. Eg, Holtorf writes:

    in contrast to the demonstrated increased risk for breast cancer with synthetic progestins,[7, 8, 58, 71-98] studies have consistently shown a decreased risk for breast cancer with progesterone.[22, 23, 25, 60, 61, 66-70, 99-101]

    Ref 22 is a randomised, placebo-control, double-blind trial of women about to undergo a lumpectomy and the intervention lasts 10-13 days; 23 similar to 22; 25 is a study of “estrogen-induced mammary carcinogenesis in ovariectomized W/Fu rats”; 60 is a nested case-control study of steroid hormone levels in pregnancy and later breast cancer; 61 is about endogenous sex hormones and subsequent breast cancer in premenopausal women; the remainder do involve women rather than rats but their nature is mixed.

Holtorf has not supplied sufficient information about his literature search strategy nor his inclusion-exclusion criteria: this may have resulted in an incomplete and partial view of the research literature.

Prof Tricia Greenhalgh’s How to read a paper is a fine commentary of what is expected in a review. Dr Ben Goldacre provides a good overview of what is involved in compiling and reporting comprehensive or systematic reviews (including Jadad scores and publication bias) in Bad Science. Dr Petra Boynton has produced an excellent list of resources for those who are interested in learning how to review and appraise research literature.

Although Burne characterises Holtorf’s paper as a “major review” it more closely resembles a selective interpretation of literature to support a particular viewpoint: according to Google Scholar, it has not, as yet, been cited in an academic piece. As Scott wrote, the evidentiary base is selective and insubstantial and Holfort seems to extrapolate “weak data into broad elaborations of the safety and effectiveness of BHT”.

The contentious status of HRT does not automatically mean that bio-identical hormone therapy must be as/or more efficacious or safer. Burne writes:

‘Many physicians state there is no evidence they are safer,’ says the author Dr Ken Holtorf, ‘but the medical literature clearly shows they are highly effective and have some distinctly different, often opposite, physiological effects to regular HRT.’

We would disagree that Holtorf has made his case that bio-identical hormone therapy is safer or more efficacious than standard HRT.

The current version of the article carries fuller caveats from Dr Nick Panay than in the earlier edition:

‘We just don’t know yet whether damaging effects will show up in the long term with bio-identicals. Trial results look encouraging at the moment, but we lack the data to say so definitely.’

However, the dedicated section about Heart disease, cancer and the tantalising evidence acts to undercut the impact of Panay’s appropriate words of caution.

However, that section is interesting enough to deserve its own post, and that shall be Part 3.

In conclusion, this post is an elaborated discussion of the fact that in the absence of explicit information in the Holtorf review, the reader can not fill in the gaps so must assume that this was an imperfectly-conducted literature search that may have resulted in an incomplete and partial view of the research literature. Under those circumstances, it might be unwise to accept the assertion that the ‘medical literature shows that bio-identical hormones are more effective and have greater health benefits than regular HRT’.

Notes

[a] Burne is appropriately punctilious in identifying other relevant posts that may flag a conflict of interest when using quotations from Drs Schwartz and Wright who are both doctors who support the use of bio-identical hormones.
[b] Cargo Cult Science by Richard Feynman. Adapted from the Caltech commencement address given in 1974.

In the South Seas there is a cargo cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to imitate things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas–he’s the controller–and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land…

[T]here is one feature I notice that is generally missing in cargo cult science. That is the idea that we all hope you have learned in studying science in school–we never explicitly say what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty–a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid–not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked–to make sure the other fellow can tell they have been eliminated.

Advertisements

4 Comments

Filed under Food Is Better Medicine Than Drugs, Jerome Burne, patrick holford

4 responses to “Jerome Burne and Bio-Identical Hormone Replacement Therapy: Part 2

  1. Pingback: Jerome Burne and Bio-Identical Hormone Replacement Therapy: Part 3 « Holford Watch: Patrick Holford, nutritionism and bad science

  2. Pingback: Jerome Burne and Bio-Identical Hormone Replacement Therapy: Part 1 « Holford Watch: Patrick Holford, nutritionism and bad science

  3. jerome burne

    Reluctant as I am to spend more time on your blog, having already replied to your misleading criticisms in Part I, I can’t let your claims, which may remain on the web for years, go unchallenged. I’m not precious, but if you are going to criticise my work it should be on justifiable grounds

    Although you claim to be objective the tone is set by the opening sneer. You say you came across a long post by Burne (on a blog about bio-identical hormones) “recommending his own research”. Actually what you came across was a detailed critique of bioidentical hormones with a comment by me pointing out some favourable evidence. It’s odd to describe an informed contribution as “recommending my research” Isn’t that what sites like this are about? Already it’s clear this is not a disinterested analysis.

    Interestingly, the original post contained one of the frequently repeated canards about bioidenticals that you also used in your Part 1 attack: “There are no bioidentical hormones that do no undergo some sort of laboratory manipulation. Consequently there are no truly “natural” bioidentical hormones.”

    As I pointed out in my response to Part 1, this is nonsense and deliberately misrepresents the whole reason why bioidenticals may well be safer – not because they haven’t been manipulated, but simply because they are identical to the ones in your body.

    Then we come to the nub of your critique. You say that the shorthand version of the whole critique it is that in response to question: Should middle aged women be taking HRT ? That the answer is: “Not if you are relying upon the Holftorf review to provide a comprehensive overview of the relevant evidence on efficacy and safety.”

    And then you plunge into a remarkably detailed assault on the Holtorf paper, saying that you are not arguing about bioidentical hormones but only about the “scholarship of the Holtorf paper and Burne’s use of it and its influence on the balance of the reporting in Burne’s article.” And you further state: “Burne relies quite heavily upon Dr Kent Holtorf’s paper.”

    But this is a total misrepresentation. I don’t know whether to assume it was wilful or whether you don’t know what “relies heavily” means. Either way the Holtorf paper is a straw man. I can’t imagine why anyone would believe that I was only relying on the Holtorf paper, not least because it takes less than 150 words in a 2000 word article. That is the extent of its influence on the balance of my reporting.

    In fact there are four other strands of evidence in the article on which I rely, as you must realise. First and foremost that bioidentical hormones are currently available on the NHS, a fact which you totally ignore, You can get them from your doctor like any other drug. They have gone through the whole lengthy evidence based licensing process.

    For various reasons most doctors and their patients are not aware of this fact and so the main function of the article was not to set out a peer reviewed journal-style assessment of the evidence of these strange things but essentially to let women know that they were there and had already been approved.

    This fact alone demolishes the whole slant of your critique which is that the article was pushing some wacky treatment of doubtful value without any real understanding of what a sound evidence base means.

    The second strand of evidence was a comment from another American doctor – also s supporter of bioidentical hormones – but one heading a society that has 250 members who are also doctors. So maybe they know something about evidence.

    Thirdly and also pretty relevant is the fact that a UK gynaecologist who is also a member of the UK menopause society is quoted at some length as saying that bioidenticals should be thought of as a reasonable alternative to the regular non-bioidentical version, although all the evidence for greater safety wasn’t in yet.

    Fourthly there was the case of France. There a bio-identical version of progesterone has been used for decades and a large observational study recently found that it was safer than the non-bioidentical version

    So what on earth are you doing directing all that firepower on Holtoff? Let alone attempting to suggest that the very fact that I mentioned it raises serious doubts about the thrust of the whole article. Could it be that, far from being a piece of evidence that I relied heavily on, it was in fact the only one you could find to criticise?

    But there is a broader problem with your exclusive concentration on the Holtorf article which goes to the heart of your whole project which appears to be a ruthless attempt to discredit anyone who writes about anything to do with medicine that doesn’t adhere to medical orthodoxy.

    The central issue is the extent to which it is reasonable to expect working journalists, many of whom myself included do go to considerable lengths to be fair and accurate, to write their articles as if they were tenured academics critiquing something that had appeared in a peer-reviewed journal.

    You make 12 specific critiques of Holtofs research methods, referring to criteria set out for Cochrane reviews and concluding: “Holtorf has not supplied sufficient information about his literature search strategy nor his inclusion-exclusion criteria: this may have resulted in an incomplete and partial view of the research literature.”

    Now I am not about to go through all your points and evaluate them; it is possible to critique any paper and I have no doubt that Holtorf would have a number of points to make in response. But are you seriously suggesting that any journalist writing on science topics should devote that much attention to every piece of research that is used in an article to? Certainly no doctor would do that. Instead both journalists and doctors rely on peer reviewed journals to do some of that heavy lifting.

    As it happened Holtorf’s paper was far from the only one that I looked at when preparing the article but I didn’t include them for the simple reason that I was writing for a consumer pages of a newspaper whose pages are not designed to be written to the standards of a Cochrane review. As I pointed out in my comment to Part 1 of your critique, my other background reading included: “Could transdermal estradiol + Progesterone be a safer postmenopausal HRT? A review by Marc L’Hermite et all in Maturitas – July/August 2008. The conclusion was that it “might offer significant benefits and added safety.”

    Or how about a case control study from France involving 271 consecutive cases with “a first documented episode of idiopathic VTE (venous thromboembolism). Conclusion: “oral but not transdermal estrogen is associated with increased VTE risk. In addition our data suggest that norpregnane derivates may be thrombogenic whereas the micronized progesterone and pregnane derivatives appear to be safe with respect to thrombotic risk.” Circulation Feb 20 2007

    It was on the basis of these and other papers that I considered the quote I used from Holtorf perfectly justified. This was: “Many physicians state there is no evidence they are safer,’ says the author Dr Ken Holtorf, ‘but the medical literature clearly shows they are highly effective and have some distinctly different, often opposite, physiological effects to regular HRT.’

    And there are flaws in your critique of Holtorf. You complain that he didn’t include an FDA review that referred to “false or misleading claims…(that). these products are “bio-identical” implying that these products are natural or identical to hormones made by the body.” Actually it is very clear that is precisely what these products are. Coming on top of the whole “natural” confusion it is obvious that there is a lot of misrepresentation about bioidenticals. My article does clear them up, it’s a pity you didn’t read it carefully.

    At the end you claim that Holtorf’s article is a good example of “Cargo cult science” – something that (and I heavily précis) has all the trappings of science but actually lacks the essential ingredient, And what is it that is missing in cargo cult science? It is, you solemnly suggest, “a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty–a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid–not only what you think is right about it.”

    Now the assumption behind this site and its relentless attacks on anyone who strays from the solid evidence base that supports mainstream biomedicine (although there is actually nothing about bioidenticals that is at odds with biomedicine) are practising a version of “cargo cult science”.

    In light of this it is interesting that a study published this week in PLoS Medicine seems to have identified a clear cut case of “cargo cult science”. Researchers from the University of California, San Francisco compared the information that drug companies had supplied to the American FDA when applying for a license, which legally has to be comprehensive, with the versions of that data eventually published in medical journals to inform doctors.

    The study found a yawning gap between the two. Only three-quarters of the original trials were ever published, and it turned out that those with positive outcomes were nearly five times as likely to be published as those that were negative. Not quite a “principle of scientific thought that corresponds to a kind of utter honesty” then.

    Which raises the question of what really are you up to? You lurk behind a pseudonym, drum up detailed and half-baked attacks on responsible journalists doing their job while ignoring ongoing distortions of scientific evidence being done to promote the sale of pharmaceuticals. It is precisely because of this kind of dishonesty that so many women are looking for something else to help with the menopause, your deliberate muddying of the water does no one any favours. Except, let me think, who might benefit if lots of women were warned off bioidenticals?

    • Thanks for the comment. Don’t have time for a full reply at the moment, but a couple of quick points.

      the original post contained one of the frequently repeated canards about bioidenticals that you also used in your Part 1 attack: “There are no bioidentical hormones that do no undergo some sort of laboratory manipulation. Consequently there are no truly “natural” bioidentical hormones.”

      As I pointed out in my response to Part 1, this is nonsense and deliberately misrepresents the whole reason why bioidenticals may well be safer – not because they haven’t been manipulated, but simply because they are identical to the ones in your body.

      Glad that you agree that its unfortunate to refer to bio-identical hormones as ‘natural’. The article’s headline was “Should middle-aged women be taking natural HRT?”. Assuming you didn’t choose this headline yourself, perhaps you might have a word with the Mail about it?

      let me think, who might benefit if lots of women were warned off bioidenticals?

      We don’t benefit if this happens.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s