National Institute for Clinical Excellence
Peter Dixon is a chiropractor. He is chair of the General Chiropractic Council (GCC). He was also a member of the hotly-disputed NICE low back pain guidance group that endorsed (you guessed it) the use of chiropractic, a decision that has led to enormous criticism of the standards of the National Institute of health and Clinical Excellence (NICE).
As a consequence largely of the decision of the British Chiropractic Association (BCA) to sue Simon Singh for defamation, there has been an unprecedented interest taken in the claims made by chiropractors in general.
Peter Dixon has a problem because something like 600 individual complaints about unjustified health claims have been sent to the GCC. Even when a web site does not claim to be able to benefit things like asthma and colic, a phone call may reveal that claims are made in private (one of the many complaints to the GCC concerns such behaviour by two practices belonging to, ahem, Peter Dixon Associates).
The crucial question is, as always, one of evidence. The BCA claim to have a plethora of evidence for their claims, but they have been strangely reluctant to produce it. In fact evidence is cited on the “Your first visit” page on Dixon’s site.
At the bottom we see “How effective is Chiropractic?”.
That sounds very impressive indeed: . ” . . . patients who received chiropractic treatment improved by 70% more than those given hospital out-patient.”
But hang on. If we look at the paper, Meade et al., 1990 [download reprint], we see that Figure 2 looks like this.
Several things jump out. First, the Oswestry disability index scale runs for 0 to 100, but scores are plotted only from 0 to 35, so the size of the effects are exaggerated. Second, there are no error bars on the points. Third there is essentially no advantage for chiropractic at all when all patients are taken together (top graph). Fourth, and most important, the patients who were followed up for two years (bottom graph) seem to show a slight advantage for chiropractic but on average, the effect is 7 percent (on the 100 point scale, NOT 70 percent as claimed on the web site of Peter Dixon Associates.
What sort of mistake was made?
The abstract of the paper itself says “A benefit of about 7% points on the Oswestry scale was seen at two years.” How did this become “improved by 70% more”?
It could have been simple a typographical error, but that seems unlikely, Who’d boast about a 7% improvement?
Perhaps it is a question of relative versus absolute change. The Figure does not show the actual scores on the 100 point scale, but rather the change in score, relative to a questionnaire given just before starting treatment. If we look at the lower part of the Figure, restricted to those patients who stayed with the trail for 2 years (by this time 28% of the patients had dropped out), we see that there is a reduction in score (improvement) of about 10 points on the 100 point scale with hospital treatment (not a very impressive response). The improvement with those sent to private chiropractic clinics was about seven points bigger. So a change from 10 to 17 is a 70 percent change. What’s wrong with that?
What’s wrong is that it is highly misleading, as relative changes often are. Imagine that the hospital number had been 7 points and the chiropractic number had been 14 (both out of 100). That would mean that both treatments had provided very modest benefits to the patients. Would it then be fair to describe the chiropractic patients as have improved by 100 percent more than the hospital patients, when in fact neither got much benefit? Of course it would not. To present the results in this way would be highly deceptive.
Put another way, a 70% increase in a trivial effect is still pretty trivial.
That isn’t all either. The paper has been analysed in some detail on the ebm-first site. The seven point difference on a 100 point scale, though it may be real, is too small to be ‘clinically significant’ In other words the patient would scarcely notice such a small change. Another problem lies in the nature of the comparison. Patients were, quite properly, allocated at random to chiropractic or to to hospital treatment. BUT the comparion was very from blind. one group was treated in hospital. The other group was sent to private chiropractic clinics. The trivial 7 point difference could easily be as much to do with the thickness of the carpets rather than any effect of spinal manipulation.
What this paper really tells you is that neither treatment is very effective and that there is little to choose between them.
It is really most unfortunate that the chairman of the GCC should show himself to be so careless about evidence at a time when the evidence for the claims of chiropractors is under inspection as never before. It does not add to their case for criticising Simon Singh and it does not add to one’s confidence in the judgement of the NICE guidance group.
The Pain Society revolt. A letter has been sent from several distinguished members of the British Pain Society to its President and Council.
“We, the undersigned, call upon the President of the British Pain Society to issue a statement to NICE and to the press condemning outright the conclusions of the recent UK National Institute for Health and Clinical Excellence (NICE) guidelines . . .”
The sigificance of this letter is that the present president of the British Pain Society is Professor Paul Watson who was a member of the NICE guidance group that produced the recommendations which have engendered such criticism. He was clinical advisor to the guidance group. There is a video of Paul Watson talking about back pain, that seems to me to illustrate very well the problem with the guidance. He says it is a huge problem (everyone knows that) and that something must be done, but he doesn’t say what. There is no admission that, in very many cases, nobody knows what to do. It is exactly this sort of hubris that that makes the NICE report so bad,
One caustic comment on the letter says
“We are led by a physiotherapist! A Professor who cannot even interpret straight forward evidence when it is presented to him on a plate.
Who’s going to be the next BPS President? A Hospital Porter?”
This is another short interruption in the epic self-destruction of chiropractors. In a sense it is more serious. One expects quacks to advocate quackery. What you don’t expect is that the National Institute of Clinical Excellence (NICE) will endorse it. Neither do you expect the Medicines and Healthcare products Regulatory Agency (MHRA) to betray its mandate to make sure that medicines work.
The saga of the NICE low back pain guidance has been the subject of a deluge of criticism, It seems doubtful that the guidance can survive, not least because of its absurd endorsement of chiropractic, at a time when chiropractic is undergoing self-immolation as a consequence of the persecution of Simon Singh by the British Chiropractic Association (see here, and here, and here, and here and thousands of other sites).
The other betrayal has come to the for after the MHRA approved highly misleading labelling of a homeopathic preparation. At the time,
in 2006, when the principle was approved by the MHRA, just about every scientific organisation, even the Royal Society, condemned the action. What was discouraging that the clinical organisations all stayed silent. It is still a mystery why the MHRA made this enormous mistake, Some said that European regulations required it, but that is quite untrue, as Les Rose has shown. It appears to have been the result of a pusillanimous MHRA bowing to pressure from a deeply unscientific Department of Health (a letter from Caroline Flint at the time borders on the surreal).
On 20 May 2009, the British Medical Journal printed an article Drugs agency grants its first licence to homoeopathic product by Deborah Cohen (available free). The comments were mostly highly critical of the MHRA. The BMJ asked, as it does from time to time, for my comment to be converted to a letter
for the print edition. That isn’t freely available, so here it is.
Published 9 June 2009, doi:10.1136/bmj.b2333
Homoeopathic product licence
MHRA label seems to be illegal
The strap line for the Medicines and Healthcare products Regulatory Agency (MHRA) is “We enhance and safeguard the health of the public by ensuring that medicines and medical devices work and are acceptably safe.”
Yet the MHRA has made mockery of its own aims by ignoring the bit about “ensuring that medicines work” and allowing Arnica 30C pills to be labelled: “a homoeopathic medicinal product used within the homoeopathic tradition for the symptomatic relief of sprains, muscular aches, and bruising or swelling after contusions.”1
This label should be illegal anyway because the pills contain no trace of the ingredient on the label, but this deceit has been allowed through a legal loophole for a long time now. If you sold strawberry jam that contained not a trace of strawberry you’d be in trouble.
But I can see no legal loophole that allows the manufacturers of Arnica 30C to evade the provisions of the Consumer Protection from Unfair Trading Regulations 2008. One of the 31 commercial practices which are in all circumstances considered unfair is “falsely claiming that a product is able to cure illnesses, dysfunction, or malformations.”
The consumer protection laws apply to the way that “the average consumer” will interpret the label. The average consumer is unlikely to know that “used within the homoeopathic tradition” is a form of weasel words that actually means “there isn’t a jot of evidence that the medicine works.”
Since there is not the slightest evidence that Arnica 30C pills provide symptomatic relief of sprains, etc, the labelling that the MHRA has approved seems to be illegal. The MHRA is not selling anything itself, so I presume that it won’t find itself in court, but anyone who follows its advice could well do so.
Cite this as: BMJ 2009;338:b2333
David Colquhoun, research professor1
University College London, London WC1E 6BT
Competing interests: None declared.
It is, I suppose, just a sign of the chaos that reigns in the multiple agencies and quangos responsible for ‘regulation’ that one arm of government proposes action that a different branch would consider illegal. That is an inevitable consequence of trying to regulate something without first deciding whether it is nonsense or not. The Department of Health appears to be quite incapable of grasping this simple and obvious fact.
Health: best treatments. The Guardian seems to the be picking up BMJ stories and ran this one.