Peter Dixon is a chiropractor. He is chair of the General Chiropractic Council (GCC). He was also a member of the hotly-disputed NICE low back pain guidance group that endorsed (you guessed it) the use of chiropractic, a decision that has led to enormous criticism of the standards of the National Institute of health and Clinical Excellence (NICE).
As a consequence largely of the decision of the British Chiropractic Association (BCA) to sue Simon Singh for defamation, there has been an unprecedented interest taken in the claims made by chiropractors in general.
Peter Dixon has a problem because something like 600 individual complaints about unjustified health claims have been sent to the GCC. Even when a web site does not claim to be able to benefit things like asthma and colic, a phone call may reveal that claims are made in private (one of the many complaints to the GCC concerns such behaviour by two practices belonging to, ahem, Peter Dixon Associates).
The crucial question is, as always, one of evidence. The BCA claim to have a plethora of evidence for their claims, but they have been strangely reluctant to produce it. In fact evidence is cited on the “Your first visit” page on Dixon’s site.
At the bottom we see “How effective is Chiropractic?”.
That sounds very impressive indeed: . ” . . . patients who received chiropractic treatment improved by 70% more than those given hospital out-patient.”
But hang on. If we look at the paper, Meade et al., 1990 [download reprint], we see that Figure 2 looks like this.
Several things jump out. First, the Oswestry disability index scale runs for 0 to 100, but scores are plotted only from 0 to 35, so the size of the effects are exaggerated. Second, there are no error bars on the points. Third there is essentially no advantage for chiropractic at all when all patients are taken together (top graph). Fourth, and most important, the patients who were followed up for two years (bottom graph) seem to show a slight advantage for chiropractic but on average, the effect is 7 percent (on the 100 point scale, NOT 70 percent as claimed on the web site of Peter Dixon Associates.
What sort of mistake was made?
The abstract of the paper itself says “A benefit of about 7% points on the Oswestry scale was seen at two years.” How did this become “improved by 70% more”?
It could have been simple a typographical error, but that seems unlikely, Who’d boast about a 7% improvement?
Perhaps it is a question of relative versus absolute change. The Figure does not show the actual scores on the 100 point scale, but rather the change in score, relative to a questionnaire given just before starting treatment. If we look at the lower part of the Figure, restricted to those patients who stayed with the trail for 2 years (by this time 28% of the patients had dropped out), we see that there is a reduction in score (improvement) of about 10 points on the 100 point scale with hospital treatment (not a very impressive response). The improvement with those sent to private chiropractic clinics was about seven points bigger. So a change from 10 to 17 is a 70 percent change. What’s wrong with that?
What’s wrong is that it is highly misleading, as relative changes often are. Imagine that the hospital number had been 7 points and the chiropractic number had been 14 (both out of 100). That would mean that both treatments had provided very modest benefits to the patients. Would it then be fair to describe the chiropractic patients as have improved by 100 percent more than the hospital patients, when in fact neither got much benefit? Of course it would not. To present the results in this way would be highly deceptive.
Put another way, a 70% increase in a trivial effect is still pretty trivial.
That isn’t all either. The paper has been analysed in some detail on the ebm-first site. The seven point difference on a 100 point scale, though it may be real, is too small to be ‘clinically significant’ In other words the patient would scarcely notice such a small change. Another problem lies in the nature of the comparison. Patients were, quite properly, allocated at random to chiropractic or to to hospital treatment. BUT the comparion was very from blind. one group was treated in hospital. The other group was sent to private chiropractic clinics. The trivial 7 point difference could easily be as much to do with the thickness of the carpets rather than any effect of spinal manipulation.
What this paper really tells you is that neither treatment is very effective and that there is little to choose between them.
It is really most unfortunate that the chairman of the GCC should show himself to be so careless about evidence at a time when the evidence for the claims of chiropractors is under inspection as never before. It does not add to their case for criticising Simon Singh and it does not add to one’s confidence in the judgement of the NICE guidance group.
The Pain Society revolt. A letter has been sent from several distinguished members of the British Pain Society to its President and Council.
“We, the undersigned, call upon the President of the British Pain Society to issue a statement to NICE and to the press condemning outright the conclusions of the recent UK National Institute for Health and Clinical Excellence (NICE) guidelines . . .”
The sigificance of this letter is that the present president of the British Pain Society is Professor Paul Watson who was a member of the NICE guidance group that produced the recommendations which have engendered such criticism. He was clinical advisor to the guidance group. There is a video of Paul Watson talking about back pain, that seems to me to illustrate very well the problem with the guidance. He says it is a huge problem (everyone knows that) and that something must be done, but he doesn’t say what. There is no admission that, in very many cases, nobody knows what to do. It is exactly this sort of hubris that that makes the NICE report so bad,
One caustic comment on the letter says
“We are led by a physiotherapist! A Professor who cannot even interpret straight forward evidence when it is presented to him on a plate.
Who’s going to be the next BPS President? A Hospital Porter?”
[…] Oh, and chiropractic’s research numbers don’t stack up. […]
you couldnt make it up
You might ask did the CHRE contact any of these former council members before compiling the CHRE report for 2008
Technorati Tags: CHRE,General Chiropractic Council
Private and Confidential
We, the undersigned, having all served as members of the General Chiropractic Council, have serious concerns about its ability to perform its statutory duties. A small ‘inner circle’ runs a Council that is anything but open and transparent. At one stage there were many foolish complaints by the chairman and council members against each other, none ever resolved. These complaints distracted Council, generated costs and were designed to quell opposition.
Bullying of Council members was endemic: some were sanctioned for questioning procedures, others simply for expressing an opinion; post was opened and retained. Agendas were engineered so that we could only rubber stamp. It was difficult to get anything onto them. One crucial item, which was passed by the full Council, was later reversed by a small minority with no attempt to contact Council members.
A number of members have resigned over these and other areas of dissatisfaction. The Secretary of State nominee, Brian Mouatt , a man of the highest integrity, resigned at the end of a meeting with no prior warning. His views had been ignored and his attempts at diplomacy marginalised.
Two years ago, there was extensive unacceptable behaviour by one of the employees, Greg Price, the Executive Officer for Regulation, which put Council at serious risk and led to his resignation. Despite attempts by several members, Council was never allowed to discuss his departure (the reason given was that it would violate the Data Protection Act!) Considering that all members of Council signed up to collective responsibility and were responsible for the welfare of staff, this is unacceptable.
We urge an independent audit or investigation (along the lines of the NMC) so that the GCC will be able to fulfil its duties properly in the future.
Enclosed is a very small portion of evidence relating to the above issues. Five other Council members agree with the contents of this letter but are reluctant to sign as they either fear reprisals or are thoroughly disheartened.
5th October 2008.
Cc: Privy Council, Mark Lancaster MP, plus enc.
The Meade study was done reasonably well, it just came up with a ludicrous conclusion (as you commented on with regards to the thickness of carpets).
Employ chiropractors in a busy NHS physiotherapy department – sans whale music – and you that find that the 7% difference vanishes over night.
If you are a hammer then everything you see will probably be a nail. Physiotherapy manipulation may be inferior to that of chiropractic, but PTs are more widely trained to deal with a long list of medical conditions, most of whom do not need manipulation.
[…] chiropractic trade has a big problem with evidence. Indeed, Peter Dixon has been criticised by Professor David Colquhoun of being ‘careless about evidence’. The reason is straightforward. […]
[…] le regole anche il presidente di quel Consiglio, […]