Truth, falsehood and evidence: investigations of dubious and dishonest science

LOB-vs
Download Lectures on Biostatistics (1971).
Corrected and searchable version of Google books edition

Download review of Lectures on Biostatistics (THES, 1973).

Recently read
Latest Tweets
Categories
Add to Technorati Favorites
Hits on this blog

Jump to follow-up

During the last year, there has been a very welcome flurry of good and informative books about alternative medicine. They are all written in a style that requires little scientific background, even the one that is intended for medical students.

CAM, Cumming | Trick or Treatment | Snake Oil Science |
Testing treatments | Suckers | Healing, Hype or Harm

I’ll start with the bad one, which has not been mentioned on this blog before.

Complementary and Alternative medicine. An illustrated text.

by Allan D. Cumming, Karen R. Simpson and David Brown (and 12 others). 94 pages, Churchill Livingstone; 1 edition (8 Dec 2006).

The authors of this book sound impressive

Allan Cumming, BSc(Hons), MBChB, MD, FRCP(E), Professor of Medical Education and Director of Undergraduate Learning and Teaching, and Honorary Consultant Physician, College of Medicine and Veterinary Medicine, University of Edinburgh, Edinburgh, UK;

Karen Simpson, BA(Hons), RN, RNT, Fellow in Medical Education, College of Medicine and Veterinary Medicine

David Brown, MBChB, DRCOG, General Practitioner, The Murrayfield Medical Centre, and Honorary Clinical Tutor, University of Edinburgh

Sadly, this is a book so utterly stifled by political correctness that it ends up saying nothing useful at all. The slim volume is, I have to say, quite remarkably devoid of useful information. Partly that is a result of out-of-date and selective references (specially in the chapters written by alternative practitioners),

But the lack of information goes beyond the usual distortions and wishful thinking. I get the strong impression is that it results not so much for a strong commitment to alternative medicine (at least by Cumming) as from the fact that the first two authors are involved with medical education. It seems that they belong to that singularly barmy fringe of educationalists who hold that the teacher must not give information to s student for fear of imparting bias. Rather the student must be told how to find out the information themselves. There is just one little problem with this view. It would take about 200 years to graduate in medicine.

There is something that worries me about medical education specialists. Just look at the welcome given by Yale’s Dean of Medical Education, Richard Belitsky, to Yale’s own division of “fluid concepts of evidence”, as described at Integrative baloney @ Yale, and as featured on YouTube. There are a lot of cryptic allusions to alternative forms of evidence in Cumming’s book too, but nothing in enough detail to be useful to the reader.

What should a book about Alternative medicine tell you? My list would look something like this.

  • Why people are so keen to deceive themselves about the efficacy of a treatment
  • Why it is that are so often deceived into thinking that something works when it doesn’t
  • How to tell whether a medicine works better than placebo or not,
  • Summaries of the evidence concerning the efficacy and safety of the main types of alternative treatments.

The Cumming book contains chapters with titles like these. It asks most of the right questions, but fails to answer any of them. There is, time and time again, the usual pious talk about the importance of evidence, but then very little attempt to tell you what the evidence says. When an attempt is made to mention evidence, it is usually partial and out of date. Nowhere are you told clearly about the hazards that will be encountered when trying to find out whether a treatment works.

The usual silly reflexology diagram is reproduced in Cumming’s introductory chapter, but with no comment at all, The fact that it is obviously total baloney is carefully hidden from the reader.. What is the poor medical student meant to think when they perceive that it is totally incompatible with all the physiology they have learned? No guidance is offered.
You will look in vain for a decent account of how to do a good randomised controlled trial, though you do get a rather puerile cartoon, The chapter about evidence is written by a librarian. Since the question of evidence is crucial, this is a fatal omission.

Despite the lack of presentation of evidence that any of it works, there seems to be an assumption throughout the book that is is desirable to integrate alternative medicine into clinical practice. In Cumming’s chapter (page 6) we see

Since it would not be in the interests of patients to integrate treatments that don’t work with treatments that do work, I see only two ways to explain this attitude. Either the authors have assumed than most alternative methods work (in which case they haven’t read the evidence), or they think integration is a good idea even if the treatment doesn’t work. Neither case strikes me as good medical education.

The early chapters are merely vague and uninformative. Some of the later chapters are simply a disgrace.

Most obviously the chapter on homeopathy is highly selective and inaccurate, That is hardly surprising because it is was written by Thomas Whitmarsh, a consultant physician at Glasgow Homeopathic Hospital (one that has still survived). It has all the usual religious zeal of the homeopath. I honestly don’t know whether people like Whitmarsh are incapable of understanding what constitutes evidence, or are simply too blinded by faith to even try. Since the only other possibility is that they are dishonest, I suppose it must be one of the former.

The chapter on “Nutritional therapy” is also written by a convert and is equally misleading piece of special pleading.

The same is true of the chapter on Prayer and Faith Healing. This chapter reproduces the header of the Cochrane Review on “Intercessory prayer for the alleviation of Ill Health”, but then proceeds to ignore entirely its conclusion “Most of the studies show no real differences”).

If you want to know about alternative medicine, don’t buy this book. Although this book was written for medical students, you will learn a great deal more from any of the following books, all of which were written for the general public.


Trick or Treatment

by Simon Singh and Edzard Ernst, Bantam Press, 2008

Simon Singh is the author of many well-known science books, like Fermat’s Last Theorem. Edzard Ernst is the UK’s first professor of complementary and alternative medicine.

Ernst, unlike Cumming et. al is a real expert in alternative medicine. He practised it at an early stage in his career and has now devoted all his efforts to careful, fair and honest assessment of the evidence. That is what this book is about. It is a very good account of the subject and it should be read by everyone, and certainly by every medical student.

Singh and Ernst follow the sensible pattern laid out above, The first chapter goes in detail into how you distinguish truth from fiction (a little detail often forgotten in this area).

The authors argue, very convincingly, that the development of medicine during the 19th and 20th century depended very clearly on the acceptance of evidence not anecdote. There is a fascinating history of clinical trials, from James Lind (lemons and scurvy), John Snow and the Broad Street pump, Florence Nightingale’s contribution not just to hygiene, but also to the statistical analysis that was needed to demonstrate the strength of her conclusions (she became the first female member of the Royal Statistical Society, and had studied under Cayley and Sylvester, pioneers of matrix algebra).

There are detailed assessments of the evidence for acupuncture, homeopathy, chiropractic and herbalism, and shorter synopses for dozens of others. The assessments are fair, even generous in marginal cases.

Acupunture. Like the other good books (but not Cumming’s), it is pointed out that acupuncture in the West is not so much the product of ancient wisdom (which is usually wrong anyway), but rather a product of Chinese nationalist propaganda engineered by Mao Tse-tung after 1949. It spread to the West after Nixon’s visit Their fabricated demonstrations of open heart surgery under acupuncture have been known since the 70s but quite recently they managed again to deceive the BBC It was Singh who revealed the deception. The conclusion is ” . . . this chapter demonstrates that acupuncture is very likely to be acting as nothing more than a placebo . . . ”

Homeopathy. “hundreds of trials have failed to deliver significant or convincing evidence to support the use of homeopathy for the treatment of any particular ailment. On the contrary, it would be to say that there is a mountain of evidence to suggest that homeopathic remedies simply do not work”.

Chiropractic. Like the other good books (but not Cumming’s) there is a good account of the origins of chiropractic (see, especially, Suckers). D.D. Palmer, grocer, spiritual healer, magnetic therapist and fairground quack, finally found a way to get rich by removing entirely imaginary ‘subluxations’. They point out the dangers of chiropractic (the subject of court action), and they point out that physiotherapy is just as effective and safer.

Herbalism. There is a useful table that summarises the evidence. They conclude that a few work and most don’t Unlike homeopathy, there is nothing absurd about herbalism, but the evidence that most of them do any good is very thin indeed.

“We argue that it is now the time for the tricks to stop, and for the real treatments to take priority. In the name of honesty, progress and good healthcare, we call for scientific standards, evaluation and regulation to be applied to all types of medicine, so that patients can be confident that they are receiving treatments that demonstrably generate more harm than good.”

Snake Oil Science, The Truth about Complementary and Alternative Medicine.

R. Barker Bausell, Oxford University Press, 2007

Another wonderful book from someone who has been involved himself in acupuncture research, Bausell is a statistician and experimental designer who was Research Director of a Complementary and Alternative Medicine Specialised Research Center at the University of Maryland.

This book gives a superb account of how you find out the truth about medicines, and of how easy it is to be deceived about their efficacy.

I can’t do better than quote the review by Robert Park of the American Physical Society (his own book, Voodoo Science, is also excellent)

“Hang up your lantern, Diogenes, an honest man has been found. Barker Bausell, a biostatistician, has stepped out of the shadows to give us an insider’s look at how clinical evidence is manipulated to package and market the placebo effect. Labeled as ‘Complementary and Alternative Medicine’, the placebo effect is being sold, not just to a gullible public, but to an increasing number of health professionals as well. Bausell knows every trick and explains each one in clear language”

Bausell’s conclusion is stronger than that of Singh and Ernst.

“There is no compelling, credible scientific evidence to suggest that any CAM therapy benefits any medical condition or reduces any medical symptom (pain or otherwise) better than a placebo”.

Here are two quotations from Bausell that I love.

[Page 22] ” seriously doubt, however, that there is a traditional Chinese medicine practitioner anywhere who ever stopped performing acupuncture on an afflicted body in the presence of similarly definitive negative evidence. CAM therapists simply do not value (and most cases, in my experience, do not understand) the scientific process”

And even better,

[Page39] “But why should nonscientists care one iota about something as esoteric as causal inference? I believe that the answer to this question is because the making of causal inferences is part of our job description as Homo Sapiens.”


Testing Treatments: Better Research for Better Healthcare

by Imogen Evans, Hazel Thornton, Iain Chalmers, British Library, 15 May 2006

You don’t even need to pay for this excellent book (but buy it anyway, eg from Amazon). If you can’t afford, £15 then download it from the James Lind Library.

This book is a unlike all the others, because it is barely mentions alternative medicine. What it does, and does very well, is to describe he harm that can be done to patients when they are treated on the basis of guesswork or ideology, rather than on the basis of proper tests. This, of course, is true whether or not the treatment is labelled ‘alternative’.

It is worth noting that one of the authors of this book is someone who has devoted much of his life to the honest assessement of evidence, Sir Iain Chalmers, one of the founders of the Cochrane Collaboration , and Editor of the James Lind Library .

A central theme is that randomised double blind trial are essentially the only way to be sure you have the right answer. One of the examples that the authors use to illustrate this is Hormone Replacement Therapy (HRT). For over 20 years, women were told that HRT would reduce their risk of heart attacks and strokes. But when, eventually, proper randomised trials were done, it was found that precisely the opposite was true. The lives of many women were cut short because the RCT had not been done,

The reason why the observational studies gave the wrong answer is pretty obvious. HRT was used predominantly by the wealthier and better-educated women. Income is just about the best predictor of longevity. The samples were biassed, and when a proper RCT was done it was revealed that the people who used HRT voluntarily lived longer despite the HRT, not because of it. It is worth remembering that there are very few RCTs that test the effects of diet. And diet differs a lot between rich and poor people. That, no doubt, is why there are so many conflicting recommendations about diet. And that is why “nutritional therapy” is little more than quackery. Sadly, the media just love crap epidemiology. One of the best discussions of this topics was in Radio 4 Programme. “The Rise of the Lifestyle Nutritionists“, by Ben Goldacre.

One of the big problems in all assessment is the influence of money, in other words corruption, The alternative industry is entirely corrupt of course, but the pharmaceutical industry has been increasingly bad. Testing Treatments reproduces this trenchant comment.


Suckers. How Alternative Medicine Makes Fools of Us All

Rose Shapiro, Random House, London 2008

I love this book. It is well-researched, feisty and a thoroughly good read.

It was put well in the review by George Monbiot.

“A fascinating and excoriating book; witty, shocking and utterly convincing”

The chapters on osteopathy and chiropractic are particularly fascinating.

This passage describes the founder of the chiropractic religion.

“By the 1890s Palmer had established a magnetic healing practice in Davenport, Iowa, and was styling himself ‘doctor’. Not everyone was convinced as a piece about him in an 1894 edition of the local paper, the Davenport Leader, shows.”

A crank on magnetism has a crazy notion hat he can cure the sick and crippled with his magnetic hands. His victims are the eak-minded, ignorant and superstitious, those foolish people who have been sick for years and have become tired of the regular physician and want health by the short-cut method . . . he has certainly profited by the ignorance of his victims . . . His increase in business shows what can be done in Davenport, even by a quack”

Over 100 years later, it seems that the “weak-minded, ignorant and superstitious” include the UK’s Department of Health, who have given these quacks a similar status to the General Medical Council.

The intellectual standards of a 19th Century mid-western provincial newspaper leader writer are rather better than the intellectual standards of the Department of Health, and of several university vice-chancellors in 2007.


Healing Hype or Harm

Edited by Imprint Academic (1 Jun 2008)
Download the contents page

My own chapter in this compilation of essays, “Alternative medicine in UK Universities” is an extended version of what was published in Nature last year (I don’t use the term CAM because I don’t believe anything can be labelled ‘complementary’ until it has been shown to work). Download a copy if the corrected proof of this chapter (pdf).

Perhaps the best two chapters, though, are “CAM and Politics” by Rose and Ernst, and “CAM in Court” by John Garrow.

CAM and politics gives us some horrifiying examples of the total ignorance of almost all politicians and civil servants about the scientific method (and their refusal to listen to anyone who does understand it).

CAM in Court has some fascinating examples of prosecutions for defrauding the public. Recent changes in the law mean we may be seeing a lot more of these soon. Rational argument doesn’t work well very well with irrational people. But a few homeopaths in jail for killing people with malaria would probably be rather effective.

Follow-up

Healing, Hype or Harm has had some nice reviews,  That isn’t so surprising from the excellent Harriet Hall at Science-Based Medicine. The introduction to my chapter was a fable about the replacemment of the Department of Physics and Astronomy by the new Department of Alternative Physics and Astrology. It was an unashamedly based on Laurie Taylor’e University of Poppleton column. Hall refers to it as “Crislip-style”, a new term to me. I guess the incomparable Laurie Taylor is not well-known in the USA, Luckily Hall gives a link to Mark Crislip’s lovely article, Alternative Flight,

“Americans want choice. Americans are increasingly using alternative aviation. A recent government study suggests that 75% of Americans have attempted some form of alternative flight, which includes everything from ultralights to falling, tripping and use of bungee cords.”

“Current airplane design is based upon a white male Western European model of what powered flight should look like. Long metal tubes with wings are a phallic design that insults the sensibilities of women, who have an alternative, more natural, emotional, way of understanding airplane design. In the one size fits all design of allopathic airlines, alternative designs are ignored and airplane design utilizing the ideas and esthetics of indigenous peoples and ancient flying traditions are derided as primitive and unscientific, despite centuries of successful use.”

Metapsychology Online Reviews doesn’t sound like a promising title for a good review of Healing, Hype or Harm, but in fact their review by Kevin Purday is very sympathetic. I like the ending.

“One may not agree with everything that is written in this book but it is wonderful that academic honesty is still alive and well.”


Print Friendly

28 Responses to Five good books and a bad one

Leave a Reply

Archives