LOB-vs
Download Lectures on Biostatistics (1971).
Corrected and searchable version of Google books edition

Download review of Lectures on Biostatistics (THES, 1973).

Latest Tweets

Categories

Archives

[This post was transferred from the old Improbable Science page]

It’s not just homeopathy. The Sunday Times, (26 February, 2006) reports that the National Health Service has fallen for another scam.

 “IT COULD be called the Cleopatra Effect. Magnetic therapy, which has held the rich and powerful in thrall from ancient Egypt to modern Downing Street, is about to be made available on the National Health Service.

NHS accountants are so impressed by the cost-effectiveness of a “magnetic leg wrap” called 4UlcerCare that from Wednesday doctors will be allowed to prescribe it to patients.”

This is nicely timed to coincide with an Editorial in the British Medical Journal, by Finegold & Flamm (2006) (click to download pdf file). The editorial title was “Magnet therapy. Extraordinary claims, but no proved benefits“. They conclude

“Patients should be advised that magnet therapy has no proven benefits. If they insist on using a magnetic device, they could be advised to buy the cheapest – this will alleviate the pain in their wallet,”

For example, Carter et al,, 2002 found no detectable effect of magnet therapy for treatment of wrist pain Attributed to Carpal Tunnel Syndrome (30 patients,double blind, careful controls). Winemiller et al., 2003 (Journal of American Medical Association, 290, 1474–78), found no benefit of magnets vs sham-magnets in treatment of plantar heel pain in 101 patients.

Magnets are said to be one of Cherie Blair’s several curious and irrational beliefs. It is alleged, according to the Daily Telegraph, that “Cherie Blair did not allow her youngest child, Leo, to have the controversial MMR vaccine and instead asked a New Age healer to wave a “magic” pendulum over him”. A few more examples are documented here and here.

The accountants at the Prescription Pricing Authority have decided that the “the magnets will save money on bandages and nurses’ time by healing the wounds.” I dare say they could save even more money by removing all effective treatments.

The evidence in favour of the magnetic treatment all seems to come from a Dr Nyjon Eccles. The Sunday Times describes him as an “NHS GP in north London”, but elsewhere he is described as "Founder, CEO and Medical Director of the Chiron Clinic" in Harley Street. A look at their web site shows that they offer a full range of alternative scams. Cancer patients can get

“LYMPH DETOXIFICATION – This is achieved by non-invasive scalar, oxygen-fed light beam therapy. This helps to detoxify the tissues by assisting the body in dissolving lymph blockages and restoring normal lymph flow using the Nobel quantum scalar technology coupled with oxygen for remarkable healing potential.”

This is total gobbledygook, designed to take advantage of the desperate.

The only real evidence to be provided by Dr Eccles that the device works is a small (26 patients) double blind trial that has not yet been published in a peer-reviewed journal, and which suffered from a number of problems (dropouts, outliers). What, I wonder, does NICE think of evidence like this?


More on magnets and the PPA

In the discussion of magnets on the Badscience site, a Michael King says that 4ulcercare will be included in Part IX of the Drug Tariff because it meets the criteria of the Prescription Pricing Authority (PPA).

I presume this Michael King is Director of Planning and Corporate Affairs at the PPA, though he does not say so.

Michael King says

“There is no judgement offered about whether a product in the Drug Tariff

is more (or less) efficacious than any other, or the placebo effect.”

The criteria for inclusion in

Part IX of the Drug Tariff () include, in section 10 iii, “They are cost effective”

Will he please explain how a device can be cost-effective, if it is ineffective (relative to placebo)?

What the PPA says

Michael King has replied to my question by email (1 Mar 2006).  He says

“The cost-effectiveness threshold for inclusion in the Drug Tariff is met if the ‘effectiveness’ of the device, as seen in data submitted by the manufacturer in support of the application, exceeds its cost to the NHS. ”

Sadly this is still ambiguous. It seems to suggest that that whatever data are submitted by the manufacturer are taken at face value, without any attempt to evaluate their quality. So I phoned King to ask if this was the case. He was helpful, but he said that it was not the case. He told me that the data were subject to some sort of low level evaluation, short of the sort of evaluation that NICE would do. This seems to contradict his earlier statement (above) that inclusion in the Tariff implies no judgement about whether a device is better than a placebo.

King said also that listing in the Tariff

“. . . is not a licensing decision nor a recommendation akin to the outcome of a NICE review”

The problem is, of course, that listing is seen as a recommendation by the public, by the Daily

Mail, and certainly by the manufacturer.

One thing, at least, is clear in this case. Whatever evaluation was done, it was done very badly. But in order to try to find out exactly what evaluation was done, and by whom, I’m having to resort to the Freedom of Information Act.

The results are here and here..

What NICE says

Fraser Woodward (Communications Manager, National Institute for Health and Clinical Excellence (NICE)) writes as follows.

 “The test of “cost effectiveness” applied by the PPA when determining whether or not a device should go on the tariff is very different to the way cost effectiviness is assessed by NICE”

That is pretty obvious, but how is the public meant to know that, when they hear that the NHS has declared a treatment to be ‘cost-effective’, that statement can mean two entirely different things according to which part of the bureaucracy the statement comes from?

Jump to Open University course K221

Jump to follow-up

BBC2 TV showed a much-advertised series on alternative medicine, in February 2006. The programmes seem to be linked with a dubious Open University course.

The programmes are presented by Kathy Sykes, who is professor of the public understanding of science at Bristol University. She has done some excellent work in that area, for example, in the Rough Science TV series.

The first programme: acupuncture

The first programme, on acupuncture, was shown on 24th January, 2006. The programme did not start
in a very promising way. Just lots of testimonials from happy patients, the staple diet of all snake oil salesmen.
They are watchable, of course, but don’t do anything at all to promote public understanding of what constitutes evidence.

There is, of course, little doubt that sticking needles into your body can produce physiological responses. Two things remain uncertain.

  • Just how useful are these responses in helping particular conditions?
  • Is there anything at all in the mumbo jumbo of meridians and chi?.

With a big flourish we were shown “a 21-year-old Shanghai factory worker undergoing open-heart surgery with only the needles to control her pain”. It turns out that this was a sham. The patient was doped on opiates and local anaesthetics. The needles were merely cosmetic. Why were we not told?

The apparently contradictory trials suggested that, at least the alleged principles of acupuncture are nonsense. The programme concentrated on a trial by Berman (Ann Intern Med. 2004,
141, 901-10 ) which used ‘sham acupuncture’, with ‘stage dagger needles’, on osteoarthritis of the knees. In this sort of trial there is no actual penetration, and the sham needles are placed on the places dictated by the mumbo jumbo. This procedure was justly criticised by a subsequent letters in the same journal (Ann
Intern Med
2005, 142, 871
).

Another large study was ignored by the TV programme altogether. This was by Linde et al. (Journal of the American Medical Association. 2005 293(17):2118-25). This study concluded

“Acupuncture was no more effective than sham acupuncture in reducing migraine headaches although both interventions were more effective than a waiting list control.”

As pointed out above, this study is, in many ways, much more interesting than Berman’s, because the control group did not have ‘sham acupuncture’. Needles were really inserted, but they were inserted in points that have nothing to do with the mumbo jumbo of meridians. The fact that the controls were much the same as the treated group suggests that, whatever effect the needles produce, it doesn’t matter much where they are inserted. The only obvious interpretation of this is that the ‘principles’ on which acupuncture is based are so much nonsense (and, therefore, it is not a subject that can possibly be taught in a university).

This crucial point was ignored by the TV programme. A big fuss was made of a functional magnetic resonance experiment, staged for TV, that showed that the effects on brain ‘activation’ are different for superficial needling and for real needling. There is nothing in the least surprising in the observation that have a needle pushed into you affects the brain, but it really does not help at all in answering the important questions. Incidentally that experiment had already been done anyway.

In summary, the first programme, failed to give a fair assessment of current knowledge about acupuncture, and failed to consider the important questions of what sort of controls are appropriate, and whether talk of meridians means anything whatsoever. Sadly, I can’t agree with the boast that “It’s the deepest investigation into the efficacy of alternative medicine ever attempted on TV”.  Let’s hope the second programme is a bit more critical.

The second programme: healing

The second programme (31st January, 2006). I liked this programme much better than the first, even if it left the crucial questions unresolved.

The programme started with a healing meeting by the notorious Benny Hinn.  The meeting had all the mass hysteria of a Nuremburg rally, though no mention was made of the fact that this (very rich) man’s financial malpractice had been revealed by a CBC TV programme.

On the right is his receipt for £3347 for two nights at the Lanesborough hotel in London (that did not include $1700 he gave in tips).

The lovely Ghanaian lady who cleans my office and lab every morning gives gives money to this mega-rich man because “he needs it to preach the gospel”.

The National Institutes of Health provided $1.8 m of US taxpayers’ money for this project which seems not to do real research at all. After seeing a demonstration of the “Gas Discharge Visualization”, GDV, or Kirlian camera, given by a very gullible Dr Melinda Connor, Sykes comments that this ‘research’ “is not so much trying to find the evidence for ‘healing energy’,  but is rather working on the basis that there is one”

Kathy Sykes did, though show a pretty healthy degree of scepticism about the people who pretend to photograph “auras” and other imaginary “force fields”. She visited the “Center for Frontier Medicine in Biofield Science” at the University of Arizona.

In other words, the ‘research’ is a con. Once again (see above) we see money given by well-intentioned governments diverted form the purpose for which it was given.  For more first class boloney on ‘imaging’, see for example, Biofield Sciences in Exeter (UK)  and ‘electro-crystal therapy’.  The list is endless.

Kathy Sykes went on to show several interesting experiments on placebo effects. For example sham healers (played by actors) do at least as well as ‘real’ healers.  And sham knee surgery may be as effective as real surgery, though the programme failed to mention the obvious possibility that this could mean nothing more than that real knee surgery is itself pretty ineffective.  As so often in this series, the producers failed to talk to the right people.

She concludes “healing does not work beyond placebo”.

So I’m right with her, though it would have been better if there had been a more critical mention of the fact that not all placebo effects are real.  Many probably depend on the natural fluctuations in the intensity of the patient’s condition.  Anything can ‘cure a cold’, because you recover from a cold in a few days anyway,

Sykes concludes, speaking of the placebo effect, “I want to see that power properly harnessed -we’d be mad not to”.  But that, disappointingly, was the end of the programme.  That point is where the problems begin.  How do you harness the placebo effect?  How do you justify lying to the patient in order to maximize the effect?  How do you train the ‘healers’? Are they themselves to believe the same lies, or are they to be trained in the art of deception?  As pointed out in a recent review of the neurobiology of placebos (Colloca and Benedetti, 2005)

“For example, the assertion that placebos, fake therapies, fresh water and sugar pills could  positively affect the brain biochemistry in the appropriate psychosocial context might lead to a dangerous justification for deception, lying and quackery”.

These are the central dilemmas of sCAM, as listed at the top of this page. The programme did nothing to solve them, or even to draw attention to them.

The link with Open University Course K221

The blurb on this programme on the Open University/BBC site concludes

“So, could the power of the mind explain the benefits people experience from healers? And have healers tapped into this power somehow? The conclusion throws new light on all healing processes, and has a surprising and inspirational message for every practitioner and patient.”

But what is to be done about this “inspiration”? Nothing is said about that.  The TV programme was immediately followed by voice-over that advertised an Open University pamphlet, which is publicity for their course K221. That course, judging from what is posted on the web, is run by true believers who are a lot less sceptical than Sykes. She says that she did the voice-over but has not yet been shown the contents of the course.

The third programme: herbalism

Oooh dear. The third programme was, in my view, by far the worst.  Hardly a single critical voice was heard. Despite the odd word of reservation,  the programme left the impression of being an advertisement for the herbal medicine industry. Did the BBC not think of asking a pharmacologist?  In my view, this programme was a disservice to human knowledge.  Let’s look at some of the details.

The programme once again starts with dramatic testimonials from satisfied customers. No hint is given to the viewer of the total unreliability of such testimonials. References, in awed voice, are mad to “a vast body of ancient knowledge that herbalists draw on”.  No mention of the superb track record that ‘ancient knowledge’ has for turning out to being dead wrong.  It was 11 minutes into the programme before the question of evidence was even mentioned and then we had a herbalist wandering through a field. At 13 minutes, the herbalist, Simon Mills, was interviewed -he rattled on about dampness. marshy conditions. “There
are herbs for heating and drying”. Sheer gobbledygook. And still no discussion of evidence.

Sutherlandia At 18 minutes “To get another view I’m going to a country where herbs are claimed to have dramatic effects”. Off to Africa to spend a good 10 minutes on Sutherlandia, a totally unverified treatment for AIDS.  Why spend all this time (and licence-payers money) to end up with the conclusion that clinical trials have
not been done yet, and we have no real idea whether it works or not?  A search of Pubmed for Sutherlandia and AIDS produces a mere five papers.  Mills et al. Nutrition Journal 2005, 4:19 write as follows.

“Despite the popularity of their use and the support of Ministries of Health and NGOs in some  African countries, no clinical trials of efficacy exist, and low-level evidence of harm identifies the potential for drug interactions with antiretroviral drugs.”

(and one of the authors on that paper is from the Canadian College of Naturopathic Medicine: hardly likely to have a bias against herbs). The comments made in the programme about AIDS were irresponsible and potentially dangerous:  they could kill people..

It took until almost half way through the programme, before we got round to the question of whether any of these claims are true. Very impressive to learn that the Nazis pushed herbal medicine, but totally uninformative (or does it mean that herbalism appeals to nutters?). We are shown the German herbal bible, but again it is pointed out that it contains no evidence about their efficacy. So no further forward yet.  Then we are introduced to chromatography: very pretty, but still no evidence about whether herbs help people.

At 9.34 pm we are last get round to some evidence. Or do we? Not yet, just another personal testimonial about the the wonders of St John’s Wort. St John’s Wort (Hypericum) is an interesting case, because there is at least some evidence that it works, though certainly not enough for it to be described as a “superherb”, as Sykes did.  Of course depression (like knee surgery -above) makes a pretty good case for herbalists, because conventional antidepressants are so very unsatisfactory themselves.  It doesn’t take much to do better than Seroxat (Paxil, paroxetine).  At 9.38 pm we get the first actual numbers. And very selective numbers they are
too. The view presented in the programme was desperately over-optimistic about the wondrous effects of St John’s wort.  Consider the recent review by Linde et al. (2005 Brit J. Psychiatry, 186, 99-107) (read
it yourself –download pdf file).  The conclusion was as follows.

“Current evidence regarding Hypericum extracts inconsistent and confusing. In patients who meet criteria for major depression, several recent placebo-controlled trials suggest that Hypericum has minimal beneficial effects while other trials suggest that Hypericum and standard antidepressants have similar beneficial effects. ”

And another trial, again not mentioned in the programme, was published in Journal of the American Medical Association, 2002, 287, 1807 – 1814) [download the pdf file]. This paper was interesting because it compared placebo, St John’s Wort and sertraline (Zoloft), a drug of the same class as Seroxat).  All three were indistinguishable (on the two primary outcome measures).  So St John’s Wort was as good as Zoloft, but only because Zoloft was no better than placebo either.  The paper concluded thus.

“This study fails to support the efficacy the efficacy of H. perforatum [St John’s Wort] in moderately severe major depression. The result may be due to low assay sensitivity of the trial, but the  complete absence of trends suggestive of efficacy for H. perforatum is noteworthy.”

Why were we not told about trials like these?

At 9.43 pm, almost three quarters of the way through the programme, we are eventually told that ginseng, echinacea and evening primrose oil do not work. What took so long?

9.46 pm. Off to South Africa to look at research in Johannesburg on Sutherlandia by Carl Albrecht (more of him below).  Some impressive stuff about flavonoids but no results.   Flavonoids can’t be absorbed, but, aha, it contains saponins too. Perhaps they allow the flavonoids into cells. Well perhaps.
But this is not information, it is idle speculation.

At 9.51, we get back to brain imaging, this time at Imperial College. Professor Sykes seems to be excessively impressed by brain imaging. We are then treated to more idle speculation about how ginko might help in Alzheimer’s disease. Dr Warner is running a clinical trial to find out whether ginko really helps. But there were no results yet. In that case why not wait until there is a result, before telling us all about it?

We are told that herbs now “have to go through rigorous quality standards”.  It was NOT made clear that the standards don’t include anything about the herb actually doing anything useful.  The standards may give some protection against your being poisoned.  They do nothing at all to guarantee you’ll be helped.

The conclusions

“What’s really impressed me is the way that different ingredients from particular herb can combine together and have really powerful effects on us humans. So I believe that herbs are going to play a key role in medicines of the future”

“What started as an ancient wisdom may just might provide new medicines that will help us all live longer, fuller lives”

These statements are quite outrageous!   The first statement has no basis whatsoever.  It is sheer idle speculation.  It could be true, but there is no reason to believe it is.

The second statement is content-free.  Yes, it “may just” do that. On the other hand it may not.

The web site for the third programme. (7th February, 2006, 2100-2200) concludes thus.

“So, what’s their secret? Working with fellow scientists, Kathy discovers that plants contain much  more than a single – or even two or three – active ingredients. They are enormously complex

Chemical cocktails that have medicinal properties modern pharmaceuticals simply can’t reproduce.”

Just one snag (apart from the misleading implication the Sykes was doing pharmacological experiments), There is not the slightest reason, thus far, to think there is any advantage in using an “enormously complex chemical cocktail”.


Stop press: on Saw palmetto (one of the “superherbs” of the TV series)

The New England Journal of Medicine, for February 9th 2006 (354, 557 – 566), reports a clinical trial of “Saw Palmetto for Benign Prostatic Hyperplasia”. This is what they say.

“Saw palmetto is used by over 2 million men in the United States for the treatment of benign prostatic hyperplasia and is commonly recommended as an alternative to drugs approved by the Food and Drug Administration.”

“In this double-blind trial, we randomly assigned 225 men over the age of 49 years who had moderate-to-severe symptoms of benign prostatic hyperplasia to one year of treatment with saw palmetto extract (160 mg twice a day) or placebo.”

“Conclusions. In this study, saw palmetto did not improve symptoms or objective measures of benign prostatic hyperplasia.”

I hope that the BBC, the Open University and Prof Sykes now appreciate the folly of judging treatments before the results are in.

Postscript. Some reviews of the TV programmes

  • There has been some lively discussion of the BBC2 series on a forum of the James Randi Educational Foundation, on the BBC2/Open University site, on Ben Goldacres’s Badscience site, and at ebm-first.com.

  • The Times TV critic was unenthusiastic.

    “So having started out as a sceptic, Sykes ended the programme chirruping, like a born-again Christian, about how herbs contain complex combinations of chemicals that scientists cannot yet reproduce&”;

  • Simon Singh writes in the Daily Telegraph (14 Feb., 2006): "Did we really witness the ‘amazing power’ of acupuncture?

    “A BBC series on unorthodox therapies was devoid of scepticism and rigour, says Simon Singh.”

    “Although the second programme was indeed a rational look at the placebo effect, the other two episodes were little more than rose-tinted adverts for the alternative medicine industry.”

    “For example, the scene showing a patient punctured with needles and undergoing open heart surgery left viewers with the strong impression that acupuncture was providing immense pain relief. In fact, in addition to acupuncture, the patient had a combination of three very powerful sedatives (midazolam, droperidol, fentanyl) and large volumes of local anaesthetic injected into the tissues on the front of the chest.
    With such a cocktail of chemicals, the acupuncture needles were apparently cosmetic. In short, this memorable bit of telly was emotionally powerful, but scientifically meaningless in building a case for acupuncture. ”

    “This TV series pretended to be scientific and had the chance to set the record straight, but instead it chickened out of confronting the widespread failure of alternative medicine. ”

  • Advertisers cash in. Sadly, but predictably, the programme on herbalism has
    already been exploited by vendors of unproven treatments. While it is true that the programme did not actually assert that this herb cured AIDS, it certainly left the impression that it was good stuff.  Here is an example: “As seen on BBC2”

    “In South Africa, BBC 2 TV presenter, Professor Kathy Sykes learnt of the herb Sutherlandia, which is being touted as a new weapon in the fight against HIV and ”

    “It is with thanks to programmes such as Alternative Medicine shown on BBC 2 on Tuesday 7th February, and the work carried out by Professor Kathy Sykes that medicinal herbs can receive the acknowledgement they truly deserve, and this knowledge be passed on to the general public.”

    “Bioharmony Sutherlandia is available from Revital Ltd in 60 x 300mg tablets for £19.99rrp. ”

  • A groundbreaking experiment … or a sensationalised TV stunt?

    Simon Singh, in The Guardian (25 March 2006) followed through with some more details on the BBC2 series. It’s not only pharmacologists who were unhappy about it. So were several of the people who advised the BBC and/or appeared on the programme.

    “But this week scientists involved in the series have complained that elements of the programmes were misleading, the production team was uninformed, and scientists were used as “marionettes” ”

    At the end of the first programme a “hugely ambitious” imaging experiment was shown with an enormous flourish. The outcome was, roughly speaking, that pushing needles into yourself produces a signal in the brain. Good heavens! Who’d have thought it? Even George Lewith, normally an apologist for CAM, was critical.

    “The interpretation of the science in this particular programme was not good and was inappropriately sensationalised by the production team. I think all of us on the experiment felt like that.”

    “The experiment was not groundbreaking, its results were sensationalised and there was insufficient time to analyse the data properly and so draw any sound conclusions. It was oversold and over-interpreted. We were encouraged to over-interpret, and proper scientific qualifications that might suggest alternative interpretations of the data appear to have been edited out of the programme.”

    Edzard Ernst, professor of complementary medicine at Exeter University, and the main consultant for the series says:

    “The BBC decided to do disturbingly simple storylines with disturbingly happy endings. But none of these stories is as simple as they presented, nor do they have such happy endings. Even when the evidence was outright negative, they somehow bent over backwards to create another happy ending.
    “I feel that they abused me in a way. It was as if they had instructions from higher up that this had to be a happy story about complementary medicine without any complexity, and they used me to give a veneer of respectability.”

    The BBC, thus far, remain unapologetic

    “We take these allegations very seriously and we strongly refute them.We used two scientific consultants for the series, Prof Ernst and Dr Jack Tinker, dean emeritus of the Royal Society of Medicine, both of whom signed off the programme scripts.”

    This is the same Jack Tinker who, as Chairman of the Ethics Committee of the Dr Foster organisation, also approved their “COMPLEMENTARY therapists Guide 2004”, and the utterly uncritical complementary practioner directory. The ‘Dr Foster’ organisation is a commercial business that supplies "management information", "market research services", "marketing services" and "information for the public". Let’s hope their services in conventional health care are a bit more critical than their evaluation of CAM. Their “Guide to [CAM] therapies” repeats all the usual pseudo-scientific gobbledygook in a totally uncritical way.

    Singh’s article ended with some quotations from this site, concerning Sutherlandia and AIDS, with the remark made above, highlighted: "Comments about Aids were irresponsible and potentially dangerous".

  • Science accuses BBC of medical quackery

    Lois Rogers, in the Sunday Times for 26th March, reports on the same topic.

    “Ernst yesterday released the contents of a letter that he has written to Martin Wilson, the series producer, criticising him for promoting “US-style anti-science”.

    He said he felt “abused” by the programme makers: “It was as if they had instructions from higher up that this had to be a happy story about complementary medicine without any complexity, and they used me to give a veneer of respectability.”  “

    “This is no longer a fringe game played by new age people,” said Colquhoun. “It is beginning to erode intellectual standards at real universities.”

Later a letter appeared in defence of the programmes. Investigation showed that this letter had actually been written by the BBC and not all of the ‘signatories’ had seen it.This is dealt with in a separate post, Alternative Medicine series: dirty tricks at the BBC?

This is a slightly modified version of some thoughts from the old improbable science page, where they formed part of the review of a BBC2 series on alternative medicine. It has been moved to the new blog because of the comments posted here.

Evolution of plants

Plants didn’t evolve for our benefit. Natural selection ensures that plants, like every other living thing, evolve in a way that maximises their own chance of survival. To ensure that, plants should be as toxic as possible to anything that might eat them. The more harm a plant does to humans, the better its chance of survival. It is sheer luck that some of the toxic principles evolved by plants occasionally turn out to be useful.

Memo to: The members of the Kansas Board of Education
From: God

Re: Your decision to eliminate the teaching of evolution as science


Thank you for your support. Much obliged. Now, go forth and multiply. Beget many children. And yea, your children shall beget children. And their children shall beget children, and their children’s children after them. And in time the genes that made you such pinheads will be eliminated through natural selection. Because that is how it works.’ . . . ;

By Gene Weingarten, Washington Post Staff Writer.

Saturday, August 14, 1999; Page C01

Naturalness

Here are some products of nature. That doesn’t mean they are good for you.

Lead, uranium, radon, arsenic, thallium, strychnine, cyanide (in Sorghum and Prunus species), Stinging nettles, poison ivy, yew, deadly nightshade, castor beans (ricin), tobacco, curare, foxglove, fly agaric, (muscarine), death cap (amanita phalloides), . . ..

Foxgloves, heart failure and biological standardisation

Here is a bit of relevant pharmacological history.

The 24th edition of Martindale’s Extra Pharmacopoeia (1958) describes Digitalis Leaf (B.P., I.P.), also known as Digit. Fol.; Digitalis; Foxglove Leaf; Feuille de Digitale; Fingerhutblatt; Hoja de digital.

It was defined as “the dried leaves of Digitalis purpurea (Scrophulariaceae).”At that time it was sometimes prescribed as Prepared Digitalis (BP),
which is “Digitalis leaf reduced to powder, no part being rejected, and biologically assayed the strength being stated in units per g. For therapeutic purposes it must be adjusted to contain 10 units in 1 g.”Sometimes foxglove leaf was prescribed as Tincture of Digitalis (B.P., I.P.).
“It may be made from unstandardised leaf, the tincture being subsequently biologically assayed, or it may be made from prepared digitalis, using a quantity containing 1000 units per litre, by percolation or maceration, with alcohol (70 %). It contains 1 unit per ml. I.P. allows also 1 unit per g. Dose: 0.3 to 1 ml. (5 to 15 minims). ”



Although these preparations are now totally defunct, they were still better than the sort of thing that is now advocated by herbalists. Why? They were better because they were standardised.

Foxglove leaves contain several chemical compunds that are useful in certain forms of hear failure. But the margin of safety is quite low. Take a bit too much and it kills you not cures you. One batch of foxglove leaves will contain different amounts of active compounds from the last batch, and that endangered patients.

From the 1930s onwards, pharmacologists developed methods of biological assay that overcame this problem. An international
standard digitalis leaf sample was established. Every new batch had to be assayed against this standard, and diluted to a fixed level of biological activity. This ensured that each batch of digitalis powder had the same biological potency as the last batch. It was a great pharmacological advance in its time. But of course it did involve the use of animals for the biological assay.

All this was solved when the active principles were purified from the foxglove leaves. There was no longer any need to uses animals for biological assays. The right amount of pure digoxin or digitoxin could be weighed out.

Fortunately herbalists are not allowed to prescribe anything as potentially dangerous as digitalis. But in general herbalists are happy to use pre-1930, unstandardised plant extracts.

I can think of no case in which there is the slightest reason to think that the mixture of chemicals in the plant is any better than the purified active principles. Of course there could be such cases of synergy. But that is just idle speculation.

No surprise there then, because idle speculation is the stuff of alternative medicine. It’s a great deal easier than making the effort to find out what works, and probably more lucrative too.

This entry has been transferred from the old IMPROBABLE SCIENCE page..

The Open University is a great institution.  Its first vice-chancellor was Walter Laing Macdonald Perry . Before he took that job, he was professor of Pharmacology in Edinburgh (and one of my Ph.D. supervisors). He must be turning in his grave at the new OU course, K221 – Perspectives on Complementary and Alternative Medicine .

The course description sounds harmless enough, ” This course provides an accessible but rigorous introduction to complementary and alternative approaches to health.”.  But just how rigorous is it?  The game is given away when you see that the “experts” seem mostly to be true believers, people who make their living from alternative medicine. How can such people be expected to the merits of the systems of beliefs
that are the basis of their livelihood?   It is rather like having a rigorous discussion about the existence of god in which all the course tutors are priests. Take some examples.

The bit about Testing Therapies is by Elaine Weatherley-Jones, She is in clinical practice as a homeopath. And as you might expect the three pages on the web about ‘testing therapies’ are highly partisan and selective.  Try these quotations.

“In homeopathy, the vital force is said to be responsible for maintaining health, combating disease by recruiting the body’s natural tendency to cure itself. In the homeopathy model, disease occurs when the vital force is not working efficiently to keep the balance of health.”

“. . . in The Manual of Conventional Medicine for Alternative Practitioners : “The essence of alternative medical thought is that there is a vitalistic principle behind and encompassing any physical object”, explaining that “vitalistic” means that there are “objects which are non-physical in part or whole”. The vital
force of homeopathy and qi of TCM are non-physical – it’s impossible to see them, no matter how powerful an electron microscope was used. Qi and the vital force are ideas that are put forward to explain how the body heals itself.”

“Vital forces”? Which century are we living in? If this were offered as social studies, perhaps it would not matter, but the Open University is offering this course as part of a B.Sc. degree. Then we get the usual weasel words about the impossiblity of testing empirically whether alternative medicines (CAM) produce an effect, regardless of how they work. That is the important question. After all we are pretty vague about how some conventional drugs work. In a mind-boggling passage we are told that it is impossible to test CAM against a placebo, but quite possible to test CAM against an orthodox treatment.

It seems, incidentally, that the author’s grasp of pharmacology, and of the literature, is a bit weak. “. . . when Belon and his colleagues reported research in 2004 in the journal Inflammation Research , they showed that ultra-high dilutions of histamines (which are proteins involved in allergic reactions and causes, for example, inflammation of the breathing tubes in asthma) are active in influencing human cell activity. ”

Histamine (there is only one), is not a protein? This is meant to be a university course! Belon, of course, is a committed homeopath. This passage conveniently ignores the fact that his experiments have been repeated at least twice by respectable scientists, and they find no such effect. Surprisingly enough, they find that no drug gives no response. Amazingly, it seems that the OU would have us think otherwise.

The rest of the course seems to be much the same. The dispassionate expert on Herbal medicine is a herbal practitioner who makes his living from it, and is just as uncritical as one might expect in that circumstance. Unlike Weatherley–Jones, though, he does come clean (more or less) about the lack of evidence as to efficacy of herbal remedies (though that evidently does not deter him from practising the subject).

The “expert” on acupuncture , Rosey Grandage, is a bit more interesting on the history of her subject, but is every bit as committed to CAM as the others. She works at the University of Westminster as course leader of the Diploma in Qi Gong Tuina and also lectures on the BSc Acupuncture course. ” Rosey practices as a physiotherapist, acupuncturist and tuina practitioner in West London”. Hardly an unbiased observer. “ . . . it is this growing popularity which answers the question of whether acupuncture has a place in the modern world.”

Is it not obvious that the long persistence, and popularity, of an idea cannot possibly be used as an index of truth? One merely has to think of the long-persistent and popular ideas about the ‘one true god’. Clearly at most one of these can be true. The history of medicine is replete with popular and persistent ideas that turned out to be untrue. Take nux vomica . For hundreds of years conventional medicine regarded strychnine as a ‘tonic’. That persisted right up to the 1950s. But it became apparent that it just did not work, and strychnine, and the very word ‘tonic’, vanished from the vocabulary of rational medical people. You
won’t be surprised to find, though, that is still widely touted by fraudulent herbalists.

It could be argued that the course is intended as sociology rather than science, though the course description does not say so, and the course can count towards a BSc. Even as sociology though, it would seem better if the viewpoint of the tutors was rather broader.

The course books

I have now obtained copies of the three course books that were used for K221last year.  They are indeed written largely as sociology not as science.  But it is a highly biased sort of sociology, as one might have expected from the commitments of the authors.   Although there are occasional references to lack of evidence, this does not seem to deter the authors from their relentless pursuit of the ‘integration’ of CAM into medical practice.

A superficial reading by a naive student might give an impression that the books are a “rigorous introduction to CAM”.  The more sophisticated student is likely to see them as subtle, even insidious, propaganda. I won’t claim to have read all three books. Opening almost any page makes obvious their not-very-hidden agenda.

Here are a few examples from CAM: Structures and Safeguards (eds. Geraldine Lee–Treweek, Tom Heller, Hilary MacQueen, Julie Stone and Sue Spurr).

Chapter 5 (Homeopathy: principles practice and controversies) contains a ludicrously biased account ot the affair of Jacques Benveniste (see here and here ).  There is no mention of the fact that his results were disproved at the time, and at least twice since, I know of only one group that has claimed similar results, and that group, like Benveniste’s, contained committed homeopaths. There is no mention of Beneveniste’s two Ignobel prizes. There is no mention of the fact that after he left France in disgrace, he went on to claim that the properties of the alleged memory of water could be sent by email, a claim so absurd that it has not persisted even within CAM.

This chapter uses the standard CAM trick of redefining the word efficacy.   Rather than its usual meaning of having an effect greater than placebo, it is conveniently redefined to mean, roughly, ‘patients say they feel better’,

“Although the issue of whether or how homeopathic remedies ‘work’ is a major bone of contention for medical scientists, the fact that they perceive that the remedies do work makes it attractive to many orthodox medical practitioners.”

So that’s OK then. Don’t trouble yourself with what’s true.

Chapter 1 of ‘CAM: Structures and Safeguards’ has the title “Knowledge, names, fraud and trust”, and is by Geralidine Lee-Treweek. It is a fine example of relativism -almost post-modernist in style. This is a discussion of knowledge in which the words ‘true’ and ‘false’ barely appear. The student who lent me
the book has scribbled in the margin “If it is not true and right –then it is not knowledge”. Pass the student, fail the tutor . (in fairness, it has to be pointed out that the student passed with distinction, despite her scepticism.)

So does Open University course K221 really give you a “rigorous introduction to complementary and alternative approaches to health. ” No it seems that it does not. Here are some more reasons.

A student who has successfully completed course K221 has told me that

  • “It was very anti-science and anti-orthodoxy in places”
  • “I had several ‘discussions’ with her [OU tutor] in our online tutor group, some about her anti-vaccination stance and the fact that she was happy to give homeopathic anti-malarials to travellers.”

The argument that homeopaths at least do no harm ( see above ) seems to be destroyed by their advocacy of policies that will lead to more children getting measles, and which will contribute to the spread of malaria. There is an ultimate irony in OU tutors preaching against vaccination. Walter Perry, the first vice-chancellor of the OU, before he was my supervisor in Edinburgh, had been Director of Biological Standards at the Medical Research Council’s labs. In that job he had responsibility for introduction of polio vaccine in UK. That effectively eliminated the scourge of polio.

This is not what a real university should be doing, as part of a B.Sc. degree.

On 21 November, 2005, Dr David Spence appeared on the BBC’s Today Programme. He was being interviewed about a report that, he said, provided evidence for the effectiveness of homeopathy. In fact it does nothing of the sort.

Dr Spence’s paper was published in the Journal of Alternative and Complementary Medicine. It is not really research at all. They simply asked 6544 patients who had had homeopathic treatment whether they felt better or not. Half the patients (50.7%) said they were ‘better’ ot ‘much better’. A further 20% said they were ‘slightly better’. The patients who had homeopathic treatment were not compared with anything whatsoever!

This is reported in a straighforward way. What is quite ludicrous is the stated conclusion of the paper:

“The study results show that homeopathic treatment is a valuable intervention”.

It is obvious that there is not the slightest reason to attribute the answers given by patients to the fact that they had been given homeopathic treatment. That would be the crudest form of post hoc ergo propter hoc error. It does not even show that the homeopathic treatment was producing a placebo effect.

Papers like this do not add to human knowledge, they detract from it. By reverting to pre-enlightment forms of argument, they mislead rather than enlighten. To make matters worse, this work was done at public expense, by the Directorate of Homeopathic Medicine, United Bristol Healthcare, National Health Service Trust, Bristol, United Kingdom.

What on earth is a respectable hospital and medical school, like those in Bristol, wasting money with this sort of mediaeval hindrance to medical knowledge? We are truly living in an age of delusions.

Download the paper and see for yourself [ Spence DS, Thompson EA, Barron SJ. J Altern Complement Med. 2005, 11, 793-8. pdf file, 74 kb].

The Daily Telegraph recently published two reports about acupuncture. One said it worked. The other said it didn’t work. Needle cure effect ‘is not all in the mind’ By Catriona Davies, starts

Acupuncture has a genuine ability to relieve pain, scientists have found.

The other report was

Doubt cast on needle therapy for migraine By Nic Fleming, Health Correspondent, starts

Acupuncture is no better at reducing migraines than fake treatment, researchers say today.

A study involving more than 300 patients found the healing method did reduce headaches, but only by the same amount as placing needles at non-acupuncture points.

Unfortunately the Daily Telegraph‘s reporters did nothing to help the confused reader. No comment was made on the apparent contradiction. In this particular case, there is an obvious explanation.

The first (favourable) article was said to be published in Nature, though in fact it was published in Neuroimage (Pariente J., White P., Frackowiak , Richard S. J. & Lewith G. Neuroimage, 25. 1161 – 1167, 2005). Presumably the reporter had picked it up from a rather uncritical synopsis on the news site, nature.com. It was conducted on 14 patients with painful osteoarthritis. Contrary to the first line of the Telegraph‘s report (“Acupuncture has a genuine ability to relieve pain”), the work did not measure pain at all. In fact the summary of the paper says

“The three interventions, all of which were sub-optimal acupuncture treatment, did not modify the patient’s pain.”

What the paper actually did was to use positron emission tomography (PET) to measure ‘activation’ of certain parts of the brain when needles were inserted. Some parts were activated more by having acupuncture needles piercing ths body than by ‘stage-dagger’ needles which retracted and did not pierce the body. I have no criticism of these findings: my purpose here is to explore the apparent contradiction between this trial and another.

The second, unfavourable, report was of a much bigger study, 302 patients with migraine headaches. It was published by Linde et al. (Journal of the American Medical Association. 2005 293(17):2118-25). This study concluded

“Acupuncture was no more effective than sham acupuncture in reducing migraine headaches although both interventions were more effective than a waiting list control. ”

These two studies were on quite different conditions, used different methods, and very different numbers of patients. But suppose we take them at face value, are they not contradictory? No, not necessarily, because they used quite different sorts of control group.

The study that was reported as showing that acupuncture worked compared patients that had real acupuncture with patients who had treatment with “stage dagger” needles that appeared to pierce the skin but did not.

The study that was reported as showing that acupuncture did not work (Linde et al.) used a different sort of control group, acupuncture needles that pierced the skin but were inserted in the wrong places (as defined by the ‘principles’ of acupuncturists). No difference was found between ‘real acupuncture’ and control.

There is nothing incompatible about these two studies if one adopts the view that piercing the skin with a needle can produce a physiological response that makes the patient feel that other sorts of pain are less painful, but that it does not matter where the skin is pierced. The latter hypothesis means, of course, that all talk about “meridians”` and “energy flow” that acupuncturists use, is no more than mumbo jumbo.

Dr Lewith is quoted as saying, of the negative study, “This is a badly conceived study that just adds more confusion to the debate because it uses non-site specific acupuncture as a control.” Quite on the contrary, the Linde study seems more interesting to me, because it unveils the mumbo jumbo of meridians (at least for the particular points used in this study). Dr Lewith may find this conclusion unpalatable, but it is the obvious implication of this pair of studies.

Another study confirms that the principles of acupuncture are nonsense

A trial by Melchart and colleagues on 270 patients shows conclusively that acupuncture can indeed produce amelioration of tension-type headache when compared with no treatment, BUT the relief is produced whether or not needles are inserted at ‘acupuncture points’. Very similar results were found with ‘superficial needling at non-acupuncture points’. British Medical Journal, 15 August 2005. Yet again it is shown that the mumbo-jumbo of meridians and magic points is nonsense. Yet again, that is a good reason why universities cannot be expected to train acupuncturists. Our business is to minimise mumbo-jumbo, not to propagate it.

This letter to The Times points out the folly of making regulations that do not require any demonstration that the product works. Can you imagine a regulation for television sets that required only that they do no harm, but did not specify that they should show a picture?

From Professor D. Colquhoun, FRS

Sir, Congratulations on your report on the deficiencies found in complementary medicine practitioners (Body & Soul, January 10).

In the face of such evidence it is natural to ask for more effective regulation of this very profitable industry. But the question is quis custodiet ipsos custodes? This question has serious implications for the universities as well as for the public (and the industry).The House of Lords report and the Government’s response to it, pointed out that the first step was to find out whether the complementary treatment worked (better than a placebo). They recommended that the Department of Health should fund research on complementary medicine, the first priority being to find out whether each therapy worked. The problem is that you cannot regulate properly an area when it is not, in most cases, known whether the product being offered has no effect above that of wishful thinking.

This raises a serious question for universities, because it leads, naturally enough, to demands for better training. But how can a university run a course on a subject about which there is so little hard evidence?  Tragically (for their own reputation), some of the new universities are running three-year BSc courses in such subjects as complementary therapies. I’m quite happy to believe that nice smells produce good placebo effects, but aromatherapy is not, by any stretch of the imagination, science, and in my view it is not honest to award Bachelor of Science degrees in it.The effect of such courses will be not to promote better regulation, but to give spurious respectability to an industry that, according to the Government, should (but does not) have, as its first priority, to find out what works and what doesn’t.

Yours faithfully,

DAVID COLQUHOUN,

A. J. Clark Professor of Pharmacology,
University College London,
Gower Street, WC1E 6BT.
d.colquhoun@ucl.ac.uk
January 10.

This is the story of my first incursion in to the fantasy world of alternative medicine.

I was asked by the producer of a television programme (QED) to look at a paper that claimed a beneficial effect of homeopathic treatment of fibrositis (Fisher, P., Greenwood, A., Huskisson, E. C., Turner, P., & Belon, P. (1989). Effect of homoeopathic treatment on fibrositis (primary fibromyalgia) British Medical Journal 299, 365-366.) [download pdf].

The homeopath, Peter Fisher, was kind enough to give me the raw data for re-analysis. Curiously. the two medical co-authors (apparently guest authors), neither of them homeopath, were reluctant to hand over the raw data.

It appeared from the paper that the crossover trial had been analysed incorrectly (each patient had been counted twice). When the results were analysed correctly, no significant effects were found.

Astonishingly, the British Medical Journal declined to publish the correction, but their rival, the Lancet, did so with alacrity (Colquhoun, D. (1990). Reanalysis of a clinical trial of a homoeopathic treatment of fibrositis. Lancet 336, 441-442.).[ download pdf ].

Incidentally, the result of this exercise, despite the fact that it had been commissioned by the television producer, was entirely misrepresented in the final TV programme. The producer was evidently less interested in discovering the truth, than in giving the public what he thought they wanted, i.e. wishful thinking. In this he must have been successful, because the first letter that I received after the programme was from a lady in Fulham, who asked me to recommend a source of homeopathic flu jabs for her cat.

It’s interesting, but not surprising that this correction has been universally ignored by advocates of homeopathy. Whether this is incompetence or dishonesty is impossible to say.

Both the House of Lords report on Complementary and Alternative Medicine, and the Government’s response to it, state clearly “. . . we recommend that three important questions should be addressed in the following order:”. (1) does the treatment offer therapeutic benefits greater than placebo? (2) the treatment safe? (3) how does it compare, in medical outcome and cost-effectiveness, with other forms of treatment?

These recommendations seem admirable, but they have not been followed. The money has gone, almost (if not completely) to projects that address the second and third questions, before it has been established that the treatments have anything other than a placebo effect. This interesting case is debated in Debate: UK government funds CAM research (Focus on Alternative and Complementary Therapies , 8, 397-401 (DC’s bit, pp 400-401)).
[Get PDF of whole debate]

Both the House of Lords report and the Government response to it, state clearly “… we recommend that three important questions should be addressed in the following order: . . .

  • (1) does the treatment offer therapeutic benefits greater than placebo?
  • (2) is the treatment safe?
  • (3) how does it compare, in medical outcome and cost-effectiveness, with other forms of treatment?

These aims seem admirable, but to what extent do the projects that have been funded match these recommendations?

The answer, sad to say, is that they do not seem to follow the recommended order of priorities at all.

None of the studies in the first initiative (Tovey, Corner and Shaw) appears to address the question that the recommendations specify should be done first. With the possible exception of White, none of those in the second phase (Shaw, Barry, Weatherley-}ones, White and MacPherson) do. (Dr White, like most other recipients, has declined to provide any information about his project so it is impossible to be sure.) None really addresses ,the second priority directly. The third recommendation is worded much more vaguely than the first two, but it is
only this third aim, the one that was meant to be done last, that might be furthered by most of these studies.

The rationalisation given by some of the applicants for uncontrolled, or ‘pragmatic’ trials is that they are conducted under real clinical conditions and tell you what the patient actually thinks. It is quite true that, from the point of view of the patient, it does not matter in the least whether they feel better because of a placebo effect or because of a specific effect of the treatment. That is an important consideration but it is not the only one.

If the first priority had been addressed first (which it has not) it is quite possible that the outcome could be that the entire effect could be a placebo effect. Such a possibility has been envisaged by no less a luminary of the CAM world than Peter Fisher (ref 1). If that were to turn out to be the case it might matter little to the patient but it would matter a great deal to universities, which are under continual pressure from CAM people to run degree courses (though only a few have acquiesced).

If the whole effect were placebo, it follows that the ‘principles’ of homoeopathy, reflexology, etc. are mere mambo jumbo and so not appropriate for teaching in universities (or, indeed, anywhere else). The question of courses and ‘training’ cannot be considered until the first question is answered because, until then, we do not know if there is anything real to train people about. That is why it is the first priority. There would however, be a dilemma for clinical practice. The placebo effect does appear to be useful, so the question would then become how best to produce a good. placebo effect without too much intellectual dishonesty. Perhaps that is a question that deserves more research.

The fact of the matter is that the Department of Health has ended up spending £1.3 million of public money in a way that directly contravenes the recommendations of the House of Lords and of the government (with one possible exception). They claim that this happened because very few applications were received that addressed the government’s first priority. That alone says something about the extent to which the CAM world is interested in tests against placebo hardly surprising since a negative result would destroy their livelihood. But, arguably, if few applications were received that addressed the first priority, then the funding should have been postponed until appropriate applications were forthcoming. The reason that this did not happen is, I fear, only too obvious. The judging panel was dominated by CAM people who clearly share the lack of interest shown by the rest of the CAM community in answering the most important question first. If such research must be done, because of public demand for it, it should have been organised by the Medical Research Council using the same criteria they would use for any other treatment.

References

1 Fisher P, Scott DL. A randomized controlled trial
of homeopathy in rheumatoid arthritis. Rheumatology
(Oxford) 2001; 40: 1052-55.

David Colquhoun, FRS
A. J. Clark Professor of Pharmacology, University College London;
Hon. Director, Wellcome Laboratory for Molecular Pharmacology,
Department of Pharmacology,
University College London,
Gower Street,
London WClE 6BT, UK.

This post was one of the earliest on my old improbable science page (late 2002). It has been reproduced here because of the interest attached to Stephen Senn’s analysis of what appears to be a rather dubious trial, and because the subject purports to be regular medicine rather than alternative quackery. It is unlikely that all of the links will still be valid.


During research for a talk on quack medicines. I came across several web sites that were selling the common co-enzyme, NADH, for a whole variety of ailments. One of these sites, Wizards Gate, no longer sells NADH (“because the profit is too small”). but continues to carry the following advertisement for it.

A recent Georgetown University Medical Center clinical pilot study among CFS patients reports that participants taking ENADA® were four times more likely than those taking a placebo to experience a reduction in symptoms.

Please click on the links below for more information.

pilot study graph.jpg

What this does NOT mention is that the huge column on the left represents eight people out of 26, and the tiny one on the right represents two people (out of 26)!

The reader is then referred to ImmuneSupport.com, who describe NADH as follows (with my comments in italics).


  • Improves cellular energy [This, as it stands, is meaningless]

  • Elevates mental clarity [There is no evidence that this is true]


  • Promotes mental alertness and concentration [Again, there is no evidence that this is true]

The ImmuneSupport site goes on to say

“NADH is proven to trigger energy production through ATP generation [true].
A naturally occurring coenzyme found in all living cells, NADH helps supply cells with energy.[true].
Thus, as NADH levels rise in the body, the cells become more energized, helping the body feel stronger and more vitalized. [This sentence confuses energy in the technical sense that is used in physics, with the everyday use of the term to describe a mental and physical state, In fact there is no reason to think that NADH “helps the body feel stronger” to any greater extent than a placebo].

Other sites that promote NADH heavily are Enada.com, and the Menuco Corporation, of which more below

The clinical trial referred to in all of these advertisements is Forsyth, L. M., Preuss, H. G., MacDowell, A. L., Chiazze, L., Jr., Birkmayer, G. D., & Bellanti, J. A. (1999). Therapeutic effects of oral NADH on the symptoms of patients with chronic fatigue syndrome. Ann.Allergy Asthma Immunol. 82, 185-191.

It is very clear from the letter below that this trial is flawed, but it is impossible to see exactly how flawed it is because the authors, in breach of all guidelines on such matters, have refused, for over a year, to make available the raw data that would allow a correct analysis to be done.

Annals of Allergy, Asthma, & Immunology ©2000;84:639-640

To the Editor:

We should like to comment on the paper by Forsyth et al. 1 This paper claims, on the basis of a randomized double-blind clinical trial, that NADH is effective in the treatment of chronic fatigue syndrome. It seems that there are several problems with the work described in this paper.

  • On page 187, the ‘raw symptom score’ is defined as a score from a questionnaire normalized to lie between 0 and 1 (the questionnaire had 50 questions, each scored 1, 2, 3 or 4, so raw scores are in the range 50 to 200). This was normalized to lie between 0 and 1 by first subtracting 50 and then dividing by 150. However, patients with chronic fatigue syndrome are said to have scores ranging from 10.9 to 55.8, so the normalized score has presumably been expressed as a percentage, whereas in the case of normal patients the values of 0.01 to 0.13 suggest that proportions have been used. Furthermore, even using percentages rather than proportions, we cannot reproduce the ranges given for patients. For example, a raw score of 66 yields a normalized score of 10.7 whereas the next possible value 67 translates to 11.3.  How in that case could Forsyth et al get 10.9? Similarly 133 yields a score of 55.3 but 134 yields 56.  So what raw score can have produced 55.8?
  • It is stated that, “an individual was considered improved if s/he demonstrated at least a 10% improvement,” but then it is also stated that, “the degree of improvement for the results to show a positive effect was an improvement by one point in 10 questions or by 2 points in 5 questions” (p 187). This corresponds to a decrease in the raw score of 10 points, not a “10% improvement.” Table 5 gives the scores (presumably normalized score x 100) for each of the 26 patients on each of the tests. It is stated that 8/26 improve on NADH and 2/26 improve on placebo. This appears to be a change in normalized score (x 100) by 10, not to a change in raw score of 10.
  • The method of analysis given on page 187 is a test of difference between two independent estimates of a binomial proportion. This is entirely inappropriate for the analysis of a crossover trial. 2
  • Most importantly, nothing whatsoever is said about which of the 26 patients in Table 5 received which treatment. Without knowing which treatment was given in which period it is impossible for the reader to repeat the analysis using methods more appropriate for crossover trials. The simple addition of one column here, giving the sequence to which the patient was allocated, would have permitted others to have performed the necessary calculations.

Drs Forsythe and Bellanti were asked in June 1999 to provide the raw data, in particular the crucial information on sequences that would allow re-analysis of their results. Despite several further requests we have not had any response. This is made all the more disturbing in view of fact that their paper is being used to justify vigorous marketing 3 of NADH as a treatment that “Enhances energy naturally, elevates mental clarity, improves alertness, and concentration,” which treatment is also being claimed by some to be useful for everything from improving athletic performance to Alzheimer’s disease. In view of this it seems that the least that it is owed the readers of your journal is for the authors to produce the original data so that their claims can be verified.

David Colquhoun

Stephen Senn

Professor of Pharmacology, University College London, Hon, Director Wellcome Laboratory for Molecular Pharmacology, Dept of Pharmacology, University College London, London, United Kingdom and ,

Professor of Pharmaceutical and Health Statistics, Department of Epidemiology & Public Health,
and Department of Statistics, University College London, London, United Kingdom

 

REFERENCES

1. Forsythe LM, Preuss HG, MacDowell AL, et al. Ann Allergy Asthma Immunol
1999;82:185-191.

2. Senn SJ. Cross-over trials in clinical research. Wiley, Chichester,
1993.

3. See, for example, http://www.enada.com/,http://www.wizardsgate.com/
and many others.

Professor Bellanti’s response:

We have reviewed the two “Letters to the Editor” from Professors Colquhoun and Senn and from Professor Lorden and wish to respond to both. Recognizing that there are various methods and points of view concerning statistical analyses, since this was a preliminary study and not a definitive one, we chose to analyze the data in the manner described. This was done to determine whether there was a basis for the therapeutic efficacy of NADH in chronic fatigue syndrome and, if so, then to proceed to a larger, more definitive study. We feel that we have accomplished this goal in obtaining sufficient evidence to proceed further. We are grateful for the constructive elements of the readers’ comments and we shall take these into consideration as we plan the definitive studies.

Joseph A Bellanti, MD

Professor of Pediatrics and Microbiology-Immunology

Georgetown University School of Medicine

Washington, DC

This reply from Professor Bellanti stands in contrast to his own statement in a recent report presented to Annual Meeting of American College of Allergy, Asthma and Immunology in which he says “We have recently demonstrated the clinical effectiveness of reduced nicotinamide adenine dinucleotide (NADH) in a group of 26 patients with CFS in a double-blinded, placebo-controlled, crossover study”.

There is no mention here (or in any of the advertisements) of the study being “preliminary” or “not definitive”. (The measurement of 5-HIAA urinary concentration as a possible predictive marker of disease activity and therapeutic efficay of NADH in the chronic fatigue syndrome. Joseph A Bellanti MD; Linda M Forsyth MD; Ana Luiza MacDowell-Carneiro MD; Dawn B. Wallerstedt, MSN, FNP; Harry G Preuss MD and Georg D Birkmayer MD. PhD ).

This reply gives no reason at all for refusing to disclose the raw data, to allow others to check the conclusions. It does say that “there are various methods and points of view concerning statistical analyses”, but does not (unsurprisingly) refer to anyone who thinks that a test of difference between two independent estimates of a binomial proportion is an appropriate way to analyse a cross over trial.

Financial interests

The editor of Annals of Allergy, Asthma, & Immunology, Dr Edward O’Connell, tells me that the journal requires authors to disclose any financial interest in the work, and that no such interests were declared in the case of Forsyth et al. This seems very odd, since the address of one of the authors, Dr G.D Birkmayer is given as Birkmayer Pharmaceuticals (Vienna), who market the ENADA brand of NADH, on which Dr Birkmayer holds patents. Birkmayer Pharmacuticals supported the work by a grant, and analysed the data

Latest news The case of the Bellanti paper was referred to the Georgetown University’s Research Integrity Committee for review, in February 2001. That committee has reported that “no violation within the purview of the Code has been committed”. In the light of the facts concerning (a) improperly analysed data, (b) non-disclosure of financial interest and (c) the refusal to allow independent analysis of the data, it seems the the University has a novel view of what constitutes research integrity!

Follow-up

This item appeared originally on my old Religion and Education page. It has been moved here because of the discussion that followed my review in the BMJ of Unscientific America and the discussion that followed, on this blog and on P.Z. Myers Pharyngula blog.


 

Until recently, the idea that the earth was created 6000 years ago was largely restricted to right wing religious fundamentalists in the USA. Now we have a government in the UK which seems to be happy that such “fruit-cake” nonsense should be taught at the taxpayers’ expense.

The following article, Good God Almighty, was commissioned by Punch magazine, which folded before it could be published.
It refers to the fuss that followed the discovery that a state-funded school was being run by extreme ‘young earth’ creationists. If you want to see just how extreme, look at the speech made by Steven Layfield, the head of science at Emmanuel School. The views expressed are so extreme that the speech was actually deleted from the web site of the Christian Institute as soon as the fuss blew up. Luckily, thanks to the Google cache, this did not work and you can still read it at here.


Good God Almighty!

or Jurassic Theology

[This article was commissioned by Punch, but the magazine went out of business before it was published]

OK class, settle down. Here is your quiz. Compare and contrast the following.

(1) ” . . if the Bible really is the Word of God – and the internal evidence is overwhelming – true Science will always agree with it.”

(2)Science teachers should

“Note every occasion when an evolutionary/old-earth paradigm (millions or billions of years) is explicitly mentioned or implied by a text-book, examination question or visitor and courteously point out the fallibility of the statement and, wherever possible, give the alternative (always better) Biblical explanation of the same data”.
[Steven Layfield, Head of Science at Emmanuel School Gateshead, 2000]

(3) “There are those that argue that Science and Christianity can be harmoniously reconciled . . . We cannot subscribe to this view”
[John Burn & Nigel McQuoid; ex-head, and head, of Emmanuel School Gateshead, 2002]

(4) “Then there is science. Science is a God-given activity. Scientists are 5using their God-given minds and God-given creativity to explore and utilise God-given nature. Sadly, biblical literalism brings not only the bible but Christianity itself into disrepute.”
[The Bishop of Oxford. The Rt. Revd. Richard Harries, 2002]

(5) “God created the world and everything in it.” “It is about 6000 years old”
[Kate and Simon, pupils of Emmanuel School, Gateshead, interviewed by Mike Thomson, BBC]

(5) [Jenny Tonge, MP:]   “Is the Prime Minister happy to allow the teaching of creationism alongside Darwin’s theory of evolution in state schools?”

[Tony Blair]. First I am very happy.”  
“Secondly, I know that the honourable lady is referring to a school in the north-east, and I think that certain reports about what it has been teaching have been somewhat exaggerated”.

The prime minister is right. Something has been exaggerated:

The idea that teachers realise that the scientific method involves not declaring the outcome in advance.

Clearly, the head of science at Emmanuel doesn’t. To proclaim, before looking, that one view (the literal interpretation of the bible) is “always better” might be expected of an itinerant preacher in the deepest bible belt, but that is the view of the man in charge of educating children at a state-funded English school.

It is a view that offends the Bishop of Oxford, and it is contrary to the views expressed by Pope John Paul II. But don’t worry, our leader is “very happy” with it. To give equal time to a simple assertion (that the earth was created 6000 years ago), and to the wealth of hard-won reasons for thinking it to be untrue, is deeply offensive to every scientist who is trying to fumble towards the best approximation to the truth that can be found. Rarely can a couple of teachers and one prime minister have managed to offend so many people, everyone from bishops to professors, at a single sweep.

Of course neither the prime minister nor anybody else knows exactly what Emmanuel School has been teaching (though the reports from the children themselves give us a good idea). The glowing report from OFSTED was actually the result of a “short inspection” which does not look at such details. The school has not been given a full inspection since 1994, before the serious zealots took control. Views such as those in the first two quotations are so obviously relevant to the quality of science teaching that you may well ask how OFSTED managed not to notice that the views of the head of science were at the very extreme edge of fundamentalism.

Consider also the following amazing coincidence. The team of inspectors who found no fault with the unusual science (biblical is always best) teaching at Emmanuel was almost the same as the team that inspected the Huntington School, York. At that school they said that there was not enough teaching of religion (in the narrow sense, as opposed to spiritual values in general). The Huntington head teacher Chris Bridge lodged a complaint against that report, and it was partially upheld by OFSTED.

Does all this mean that extreme fundamentalism has infiltrated OFSTED, or does it simply mean that the inspectors did not do their homework?

The latter is the more charitable interpretation, but the inspectors are not going to get the chance to ‘try harder next time’. Instead Her Majesty’s Chief Inspector, wrote a letter to the chairman of the governors, asking for “clarification”. Unlike most schools, the Emmanuel School does not publish the names of its governors in the school prospectus, and neither OFSTED, nor the Department of Education, knows who they are. OFSTED does give the name of the chairman as “Dr `Peter Vardy” (as does the school if you phone). It turns out that the learned Dr Vardy is one and the same person as Sir Peter Vardy, the man who gave the government two million pounds to pay for the school. Don’t worry about the “Dr” bit though –it is perfectly genuine. The University of Sunderland was generous enough to give him an honorary doctorate in business administration (after he generously gave the University £1 million). The Chief Inspector (by then changed) recieved a suitably emollient letter from Sir Peter, but did no investigation whatsoever. Ofsted gives the firm impression of having born yesterday.

It is hard to appreciate the manic fervour of Mr Layfield’s notorious speech from only two quotations. It used to be on the web site of the Christian Institute (www.Christian.org.uk), and that site is still worth a visit. Mr McQuoid is a major contributor (see quotation no 2), but other bits are well worth reading too, like the vigorous defence of beating children (don’t get me wrong –I know there are lots of sites that deal with that sort of thing, but this is different; it is holy beating).

Mr Layfield’s speech suddenly vanished from this web site shortly after Tania Brannigan (of the Guardian) told us what was going on. Luckily, though, some kind folks thought it was such an outstanding piece of work that should still be available to everyone (just go to http://www.darwinwars.com/lunatic/liars/layfield.html).

Time’s up folks. Put down your pens and now go out and vote.

This item appeared originally on my old Religion and Education page. It has been moved here because of the discussion that followed my review in the BMJ of Unscientific America and the discussion that followed, on this blog and on P.Z. Myers Pharyngula blog. Evan Harris MP and I were up against the head of fundamentist school that had been started by wealthy used-car dealer, Peter Vardy. Being rather daunted by the Oxford Union, I actually wrote down my bit. The order of speaking because Evan Harris had to rush back to the House of Commons to vote against the Iraq war. The rest is history.


Oxford Union talk on ‘faith schools’ and creationism

Monday 25th November 2002

This debate is meant to be about ‘faith schools’, but I suspect that the differences between the views of my two friends opposite are rather large. Mr Brady will, I presume, advocate schools that teach conventional religious views, and that I take to be the topic under discussion. First let’s get the issue of Mr McQuoid’s view on creation out of the way before coming to the serious matter of the debate. We have to be clear about one thing -there is a spectrum of religious views and Mr McQuoid is at the extreme fruit cake end of that spectrum. A true flat-earther.

An article by John Burn & Nigel McQuoid; (ex-head, and head, of Emmanuel School) stated:

“There are those that argue that Science and Christianity can be harmoniously reconciled . . . We cannot subscribe to this view”

This appeared on the web site of the Christian Institute, and that site is well worth a look. Apart from promoting ‘young-earth creationism’, it has two other obsessions, the evilness of homosexuality and the desirability of corporal punishment of small boys. It is truly bizarre. Well, at least Mr McQuoid’s job is not to teach science, but the head of science in his school, Steven Layfield, said in a speech published on the same web site (in 2000)

“…if the Bible really is the Word of God – and the internal evidence is overwhelming – true Science will always agree with it.”
Science teachers should… “Note every occasion when an evolutionary/old-earth paradigm (millions or billions of years) is explicitly mentioned or implied by a text-book, examination question or visitor and courteously point out the fallibility of the statement and, wherever possible, give the alternative (always better) Biblical explanation of the same data”.

He is free to express that view, but it disqualifies him from teaching science. The speech from which this is quoted, incidentally, disappeared form the web site of christian.org as soon as the row became public -one does wonder why, but don’t worry you can still ready it at the quaintly named site http://www.darwinwars.com/lunatic/liars/layfield.html However offensive it may be that such extremism is subsidised by taxpayers money, however offensive it may be that no full inspection of his school has been carried out since he came to power, and however offensive it may be that the prime minister supported his views, it is an irrelevant issue for the purposes of serious discussion. The pope does not believe that the earth was created 6000 years ago, the Anglican church does not believe that the earth was created 6000 years ago, hardly anyone but Mr. McQuoid thinks that the earth was created 6000 years ago (except in some parts of the south USA). The number of people who have that capacity to deny the obvious will never be more than handful, and to that extent we need not bother any further about them, and we can get on with the real issue of religious schools. As the Bishop of Oxford (The Rt. Revd. Richard Harries) said (2002)

“Then there is science. Science is a God-given activity. Scientists are using their God-given minds and God-given creativity to explore and utilise God-given nature. Sadly, biblical literalism brings not only the bible but Christianity itself into disrepute.”

Now to the real question…

What is wrong with allowing parents to choose a school for their child that teaches the parents’ religious belief? At first sight this sounds quite innocent, apart from the fact that it is the parents’ religious belief, not the child’s. In any case there are not many parents in the country who have religious (well Christian) beliefs. No more than 7.5 % now go to church regularly, and recently, for the first time, the proportion who profess even a nominal attachment to the church fell below half (as did the proportion of parents who have children baptised).

Why, then, are religious schools quite popular? The reason appears to be that they are selective. The rules say that selection must be on religious grounds only, but it simply does not work out like that. It is bad enough that children’s access to education should depend on their parents’ religious views (or, not infrequently, on their parents’ skill at lying about their religious views). But of course, selection of some implies exclusion of others, and, it turns out, not only because of their religion. Religious selection leads inevitably to racial selection too. In Oldham, scene of the recent race riots, the christian schools are almost exclusively white. Selection by wealth occurs too. In Accrington a C of E school has 12.5% of special needs pupils, while its neighbouring non-religious school has 69.8%; eligibility for free school meals is and 5% in the C of E school and 33% in its neighbour (on average the factor is almost 2).

How could such a disgraceful state of affairs come about? I believe that, as so often, a look at history might help, so let’s go back to see how the world looked in 1820.

Apart from 9 years I have spent all my working life at University College London.
The relevance of that will be obvious if we recall that in 1820, there were only two universities in England, Oxford and Cambridge (though 3 in Scotland). Both Oxford and Cambridge restricted entry to members of the Church of England – not only did they exclude Jews, atheists and muslims, but they also excluded many christians -catholics, protestants and dissenters. They would, no doubt, have excluded Mr McQuoid from getting higher education. It was in this context that liberals and dissenters in London founded the University of London, soon to become known as University College London, with the specific aim of admitting good students regardless of their religious beliefs, and with providing them with non-sectarian education in science as well as humanities (there was little science in Oxford and Cambridge at that time). Despite prolonged opposition from Oxford and Cambridge, UCL opened its doors in 1828.

At that time the fiercely conservative Duke of Wellington had just, albeit reluctantly, made the transition from general to prime minister. Even his government passed measures to increase the rights of catholics and dissenters, but his implacable opposition to electoral reform led to his resignation two years later, to be replaced by the Whig administration of Earl Grey.

The country was in ferment and change was rapid -in 1830 the Liverpool-Manchester railway opened and Faraday’s work on electricity was well-developed. But the electoral system was still mediaeval. The ‘rotten borough’ of Old Sarum (an iron age fort and 7 inhabitants) had two MPs while Manchester, already busy with the cotton trade, had none at all. In 1831 three attempts at electoral reform failed in parliament, but Earl Grey had told William IV “it is the spirit of the age which is triumphing and that to resist it is certain destruction”. The next year, in 1832, Lord John Russell (grandfather of Bertrand Russell) successfully got parliament to pass the (first) Reform Act and the modern age of democracy had begun.

The political ferment was paralleled by scientific ferment -people were no longer constrained by dogma and authority and new ideas came into medicine and biology too.

After another 40 years, Oxford and Cambridge caught up, when the Universities Test Act of 1870 eliminated religious criteria for university entrance. After 1870 it became quite unthinkable that a selection for higher education should be dependent on what a person believed, or did not believe in their private lives about religion.

That was 1870. Now it is 2002.. Can you imagine Oxford reintroducing religious selection of undergraduates now? Yet my friends opposite, and the government, are supporting a view of education that started to vanish in 1828 with the foundation of UCL, and finally died in 1870 with the Test Acts. They are trying to set back progress not by 10 or 20 years but by 175 years.

What next? Bring back rotten boroughs?