It is a good thing that clinical trials have to be registered, but is not good that there is no obligation to reveal the outcome. Many are never published. Nobody knows quite why they are not published but clearly it is a source of ‘publication bias’ if results that somebody doesn’t like, whether for financial or ideological reasons, simply vanish.
That has been a problem with the pharmaceutical industry, as discussed by Ben Goldacre in This is a very broken system. For example, It has turned out that the SSRI antidepressants are essentially ineffective in mild/moderate depression, but that fact was concealed because negative trials were hidden by the drug companies. Likewise, it must be very tempting for homeopaths and other advocates of magic medicine, to quietly forget about trials that don’t come out as they wish. Nobody knows how often that happens, and Homeopaths certainly don’t always bury negative results. Peter Fisher has published trials with negative results. So has George Lewith. Both, needless to say, continue to prescribe it.
It seemed until this month that burial had been the fate of a trial of homeopathy at the Bristol Homeopathic Hospital, run by Dr Elizabeth Thompson.
Dr Thompson was also an author of the infamous paper, Spence et al (2005) paper [download pdf]. This paper was no more than a customer satisfaction survey. Half the patients felt better or much better after a visiting the Homeopathic Hospital, but there was no control group and so not the slightest reason to think that they felt better because of the homeopathic treatment. This pathetic apology for a trial is, needless to say, widely cited by homeopaths. Children with asthma were the group who most often said they felt better, and that might have been taken as a hint ro do a proper trial.
That isn’t what happened though. A small unblinded trial was proposed and it was run between January 1st 2005 and September
30th 2007. In March 2006 the University Hospitals Bristol NHS Trust announced a trial that was being run by Dr Elizabeth Thompson at the Bristol Homeopathic Hospital. The press release has now vanished, but it was reported by the BBC in May 2008.
But no results appeared. Soon people stared asking where the results were. It was discussed on the UKSKeptics forum in October 2006, and on the James Randi Educational Forum. I wrote to Dr Thompson in December 2007 to ask if their results of the asthma study were available yet, and got a rather rude reply (some details below). I was also told by Dr Thompson that
"I have just submitted the funders report today and we have set ourselves the deadline to publish two inter-related papers by March 1st 2007.".
But it still didn’t appear. I and others wrote to the Hospital Trust but got nowhere. Two years later, In 2009, a Freedom of Information Act request was sent to the Hospital Trust (not by me) to try to discover what the still unpublished results were. In February 2010, the Hospital’s lawyers declined to provide the information on the grounds that publication was "imminent".
Then, at last, in July 2011,four years after it was promised, the paper appeared, in the journal Homeopathy. UCL does not, thank heavens, subscribe to that journal, but a request on Twitter produced three copies in no time: twitter is great for crowd-sourcing. The paper is The feasibility of a pragmatic randomised controlled trial to compare usual care with usual care plus individualised homeopathy, in children requiring secondary care for asthma.
It’s not surprising that publication was delayed. The results are completely negative. In fact it shows that the homeopathic treatment didn’t even produce a placebo effect, never mind an effect of its own.
Some details of the paper
The paper compared ‘usual treatment’ with ‘usual treatment plus homeopathic treatment’ for children with asthma. Children were allocated randomly to one treatment or the other (good) but of course they were aware of what treatment they were getting (not good). Ernst has pointed out that this sort of trial can never give a negative result unless the homeopathic treatment is actually harmful. The usual care plus homeopathy group can only benefit from any placebo effect produced by the homeopathic consultation.
The pills used were all 12C dilutions or even weaker, so none of them contained anything whatsoever.
The remarkable thing about this trial was that there was no detectable difference between the ‘usual treatment’ and ‘usual treatment + homeopathy’ groups.
The homeopathic treatment was not just ineffective in itself, but it didn’t even have any detectable placebo effect.
In that respect, the result resembles those in a recent paper in the British Journal of General Practice that showed acupuncture didn’t even produce any useful placebo effect.
The trial was quite small, 39 children aged 7 – 14 years, with moderate or severe asthma were divided into the two groups, and 35 finished the trial. The follow-up periods was 16 weeks which should be enough to show any substantial effect on asthma. Twelve different outcomes were measured and none showed any difference between the two groups (despite the fact that no allowance was made for multiple comparisons, and no primary outcome was specified in advance).
A cost-benefit analysis was done. There was no benefit but there was certainly a cost. On average, each of the children in the usual care group cost the NHS £323, but when homeopathy was added, the cost was £937. That’s an extra cost of £615 for no benefit.
The authors’ conclusions are simple
Conclusions: A future study using this design is not feasible,
That’s pretty feeble. They don’t state the conclusion as "homeopathy doesn’t work", far less that "homeopathy doesn’t even have a placebo effect". Just the eternal cry after every failed trial of magic medicine: the trial design was wrong and more research is needed. An excuse was offered in the form
"A further limitation was the length of the study period which may have needed to be longer in order for homeopathic treatment to make an impact in a complex disease with high variabilitythrough the year."
This is a paraphrase of the typical homeopathic modus operandi. Keep trying a different pill until the patient gets better anyway, then claim the credit.
Some details of the attempts to discover the results
Some of this was recounted in 2007, but it’s so bizarre I’ll repeat it here.
On 11 December 2007 I wrote to Dr Thompson, thus
In March 2006, a press release http://www.ubht.nhs.uk/press/view.asp?257 announced a randomised trial for homeopathic treatment of asthma in children.
This was reported also on the BBC http://news.bbc.co.uk/1/hi/england/bristol/4971050.stm .
I’d be very grateful if you could let me know when results from this trial will become available.
The reply, dated 11 December 2007, was unsympathetic
I have just submitted the funders report today and we have set ourselves the deadline to publish two inter-related papers by March 1st 2007.
Can I ask why you are asking and what authority you have to gain this information. I shall expect a reply to my questions,
I answered this question politely on the same day, 11th December.
I know we disagree about the strength of the existing evidence, but nevertheless I was surprised by the strength of your reaction, and the rather abusive stance about my scientific credentials. I’m rather interested in evidence (my first academic work was a text book on statistics), and I’m always eager to see new data. From little I could learn it seemed that your data might be rather better quality than usual. If the evidence is good enough, I’m quite happy to change my mind. That’s how science works isn’t it?
With best regards
Nevertheless my innocent enquiry drew forth a rather vitriolic complaint from Dr Thompson to the Provost of UCL (dated 14 December 2007). Despite the fact that I’d replied on December 11th, she said to the provost (with a lot more invective)
"As yet I have not received a reply from Professor Colquhoun as to the authority he is using to make direct enquiries to me about my research. I would be grateful if you could reassure me th~t UCL have really thought through the implications of having someone, with such strong opinions that seem to extend beyond reason, promoting their opinions at such a high profile".
Dr. E. A. Thompson, BA Oxon, MBBS, MRCP, FFHom,
Lead Clinician/Consultant Homeopathic Physician &
Honorary Senior Lecturer in Palliative Medicine
In this case, the Provost came up trumps. On 14 January 2008 he replied to Thompson:
“I have looked at the email that you copied to me, and I must say that it seems an entirely proper and reasonable request. It is not clear to me why Professor Colquhoun should require some special authority to make such direct enquiries”.
Dr Thompson seems to be very sensitive. Now we have seen the results of her trial, perhaps it’s not surprising
“We’d better not do that again.”
The paper says:
I wonder when that’ll appear? Also, because it’s qualitative rather than quantitative, are you taking any bets on the conclusion?
I was rather impressed/shocked/surprised that they came right out and said:
“further investigation of a potential role for homeopathy in asthma management might be better conducted in primary care with children with less severe asthma.”
: – or to paraphrase – “we might try again with patients who aren’t all that ill”.
Dr Thompson’s lack of self-awareness when writing this about someone else is quite remarkable.
“It is a good thing that clinical trials have to be registered, but is not good that there is no obligation to reveal the outcome.”
Actually, whether there is an obligation to reveal the outcome depends on where you are, and sadly, it’s an area where the US are ahead of us. In the US, the FDA Amendments Act 2007 makes it compulsory for the main results of regulated clinical trials to be made publicly available.
I guess this wouldn’t help with homeopathy trials, as they’re not regulated, but it does help massively to ensure that the results of drug trials are made available.
It is to be hoped that regulators on this side of the pond will catch up.
Thanks for that clarification. Despite some steps in the right direction, the system is a mess. According to Ben Goldacre, there isn’t just one clinical trials register, but 58
I wonder if clinical trials registers have suffered from the “how standards propagate” effect described recently by XKCD?
There are indeed many clinical trials registers, but FDAAA mandates posting results in a specific one, namely clinicaltrials.gov.
However, for trials that aren’t regulated by FDAAA, I don’t think anyone could argue with you when you say the system is a mess.
I was gobsmacked to read Dr Thompson’s unprofessional responses to you. But, on reflection, the only surprising thing about this story is that Dr Thompson continues to believe in homeopathy in spite of the evidence that she has found that it’s a crock.
It would seem to lend some weight to the view that homeopathy is a cult.
Thanks for the info – I’ll add this trial to my new helpful website:
It seems you can’t escape from homeopathy:
I wonder if this was filmed at the Bristol Homeopathic Hospital?
Now, here’s the thing. We on the sceptical side have been prepared to allow that SCAM therapy may offer genuine benefit, albeit only as a placebo and so we have debated how to deliver placebo therapies with honesty. We have struggled to work out how this might be done. Some sceptics (Ben G, I’m looking at you) have described the placebo effect in detail and how it is subject to various influences- more invasive, more effect; more ritual, more effect. None of this is untrue, but I think we have allowed the SCAMsters too much space in which to operate. Certainly, chronic disease has a large psychosomatic component, but I am increasingly of the view that we have been overly generous to the lying deceptive toads of SCAM. The evidence really is stacking up, now, that for real diseases the Placebo Effect amounts to Bugger All. If anything, we need to be even more dismissive of SCAM’s touted benefits. The SCAMsters are standing on a sandbank in a floodplain and the flood of evidence is eroding the sand closer and closer to their feet.
“…the only surprising thing about this story is that Dr Thompson continues to believe in homeopathy…”
You cannot possibly know that she believes in it.
What you can know is her public professions of confidence.
@Michael Kingsford Gray
You are right, of course. It is one of life’s mysteries to try to understand what goes on in the minds of people like Thompson, Fisher and Lewith. They have all found negative results but they all go on using it.
I often get the impression that people like these are rather like the middle-aged vicar who long since stopped believing in god, but isn’t willing to lose his country rectory because of that.
How do they know that it was unrelated to the homoeopathic treatment?
[…] Homeopathic Hospital’s study finds that homeopathic pills have no effect […]
[…] secondary care for asthma published in Homeopathy is deconstructed by David Colquhoun at his blog DC’s Improbable Science. I’m not going to copy his attempts to obtain the results, nor the details of his discussion. […]
We all know homeopathy is b******s so change the record.
What’s the alternative David? Salmeterol? Not according to Cochrane (Cates CJ 2008).
All-cause mortality was higher with regular salmeterol than placebo but the increase was not significant. Non-fatal adverse events were significantly increased when regular salmeterol was compared with placebo.
There were more adverse reactions recorded in a medicine containing an active ingredient than to one containing none? No sh1t! The relevant data is surely what percentage of adverse reactions occurred in relation to no adverse reaction, and the comparative successful outcomes of the drug compared to placebo.