LOB-vs
Download Lectures on Biostatistics (1971).
Corrected and searchable version of Google books edition

Download review of Lectures on Biostatistics (THES, 1973).

Latest Tweets
Categories
Archives

National Institutes of Health

Jump to follow-up

Reply to David Katz.

The Atlantic is an American magazine founded (as The Atlantic Monthly) in Boston, Massachusetts, in 1857. It is a literary and cultural magazine with a very distinguished history. Its contributors include Mark Twain and Martin Luther King. So it was pretty exciting to be asked to write something for it, even with a 12 hour deadline.

atlantic logo

Sadly though, in recent years, the coverage of science in The Atlantic has been less than good The inimitable David Gorski has explained the problem in Blatant pro-alternative medicine propaganda in The Atlantic. The immediate cause of the kerfuffle was the publication of an article, The Triumph of New-Age Medicine. It was written by a journalist, David Freedman. It is very long and really not very good. It has been deconstructed also by Steven Novella.

Freedman’s article is very long, but it boils down to saying I know it doesn’t work but isn’t it nice. The article was followed up with Fix or Fraud: a ‘debate’., The debate is rather disappointing. It suffers from the problem, not unknown at the BBC, of thinking that ‘balance’ means giving equal time to people who think the earth is flat as it gives to people who think it is a oblate spheroid. The debate consists of 800 word contributions from seven people, six of whom are flat earthers, and one of which is very good. Try Steven Salzberg, A ‘triumph’ of hype over reality for some real sense. One of the flat earthers is director of a National Institutes of Health institute, NCCAM.

And this is a magazine that published not only Mark Twain, but Abraham Flexner, the man who, in 1910, put US medical education on a firm scientific footing, You can read Flexner in their archive. Mark Twain said

[A reply to letters recommending remedies]:

Dear Sir (or Madam):–I try every remedy sent to me. I am now on No. 67. Yours is 2,653. I am looking forward to its beneficial results. – quoted in My Father Mark Twain, by Clara Clemens

and

"allopathy is good for the sane and homeopathy for the insane"

So here is the piece, produced rather rapidly, for the debate. This is the original unedited version, slightly longer than appears in The Atlantic.

The title for The Atlantic piece, America, Land of the Health Hucksters, was theirs not mine. There is no shortage of health hucksters in the UK. but at least they mostly haven’t become as embedded within univerities and hospitals as much as in the USA.

David Freeman’s article, “The Triumph of New Age Medicine” starts by admitting that most alternative treatments don’t work, and ends by recommending them.  He takes a lot more words to say it, but that seems a fair synopsis.  It is the sort of thing you might expect in a cheap supermarket magazine, not in Atlantic.

The article is a prime example of rather effective sales technique, much beloved of used car salesmen and health hucksters. It’s called bait and switch.

It’s true that medicine can’t cure everything. That’s hardly surprising given that serious research has been going on for barely 100 years, and it turns out that the humans are quite complicated.  But the answer to the limitations of medicine is not to invent fairy stories, which is what the alternative medicine industry does.  There is no sensible option but to keep the research going and to test its results honestly.  It’s sad but true that Big Pharma has at times corrupted medicine, by concealing negative results.  But that corruption has been revealed by real scientists, not by health hucksters.  In the end, science is self-correcting and the truth emerges.  Health hucksters, on the other hand, seem incapable of giving up their beliefs whatever the evidence says.

The idea of patient-centered care is fashionable and care is great, if you can’t cure.  But there’s a whole spectrum in the wellbeing industry, from serious attempts to make people happier, to the downright nuts.  The problem is that caring for patients make a very good bait, and the switch to woo tends to follow not far behind.

I write from the perspective of someone who lives in a country that achieves health care for all its citizens at half the cost of the US system, and gets better outcomes in life expectancy and infant mortality. The view from outside is that US medicine rather resembles US religion. It has been taken over by fundamentalists who become very rich by persuading a gullible public to believe things that aren’t true.

One of Freedman’s problems is, I think, that he vastly overestimates the power of the placebo effect.  It exists, for sure, but in most cases, it seems to be small, erratic and transient.  Acupuncture is a good example.  It’s quite clear now that real acupuncture and sham acupuncture are indistinguishable, so it’s also quite clear that the ‘principles’ on which it’s based are simply hokum.  If you do a non-blind comparison of acupuncture with no acupuncture, there is in some trials (not all) a small advantage for the acupuncture group.  But it is too small to be of much benefit to the patient.

By far the more important reason why ineffective voodoo like acupuncture appears to work is the “get better anyway” effect (known technically as regression to the mean).  You take the needles or pills when you are at your worst, the next day you feel better.  It’s natural to attribute the fact that you feel better to the needles or pills, when all you are seeing is natural fluctuations in the condition.  It’s like Echinacea will cure your cold in only seven days when otherwise it would have taken a week.

If the article itself was naïve and uncritical, the follow up was worse. It is rather surprising to me that a magazine like Atlantic should think it worth printing an advertorial for Andrew Weil’s business.

Surely, though, Josephine Briggs, as director of an NIH institute is more serious?  Sadly, no.  Her piece is a masterpiece of clutching at straws. The fact is that her institute has spent over $ 2 billion of US taxpayers’ money and, for all that money it has produced not a single useful treatment.  All that NCCAM has done is to show that several things do not work, something we pretty much knew already. If I were a US taxpayer, I’d be somewhat displeased by that. It should be shut down now.

At first sight Dean Ornish sounds more respectable.  He bases his arguments on diet and life style changes, which aren’t alternative at all.  He’s done some research too.  The problem is that it’s mostly preliminary and inconclusive research, on the basis of which he vastly exaggerates the strength of the evidence for what can be achieved by diet alone. It’s classical bait and switch again.  The respectable, if ill-founded, arguments get you the foot in the door, and the woo follows later.

This is all very sad for a country that realized quite early that the interests of patients were best served by using treatments that had been shown to work. The Flexner report of 1910 led the world in the rational education of physicians. But now even places like Yale and Harvard peddle snake oil to their students through their "integrative medicine" departments. It’s hard to see why the USA is in the vanguard of substituting wishful thinking for common sense and reason.

The main reason, I’d guess, is money. Through NCCAM and the Bravewell Collaborative, large amounts of money have been thrown to the winds and businesses like Yale and Harvard have been quick to abandon their principles and grab the money. Another reason for the popularity of alternative medicine in parts of academia is that it’s a great deal easier to do ‘science’ when you are allowed to make up the answers. The "integrative medicine" symposium held at Yale in 2008 boggled the mind. Dr David Katz listed a lot of things he’d tried and which failed to work, His conclusion was not that they should be abandoned, but that we needed a "more fluid concept of evidence".  You can see it on YouTube,

Senator Tom Harkin’s promotion of NCCAM has done for the U.S. reputation in medicine what Dick Cheney did for the U.S. reputation in torture. It is hard to look at the USA from outside without thinking of the decline and fall of the Roman Empire.

One had hoped that era was over with the election of Obama, but the hucksters won’t give up without a fight. They are making too much money to do that.

Follow-up

The comments that appeared in The Atlantic on this piece were mostly less than enlightening -not quite what one expected of an intellectual magazine. Nevertheless I tried to answer all but the plain abusive comments.

More interesting, though, was an editorial by Jennie Rothenberg Gritz, the Atlantic Senior Editor who asked me to contribute. The Man Who Invented Medical School. It picked up on my mention of Abraham Flexner, and his famous 1910 report [download from Carnegie Foundation] which first put US medial education on a form rational footing. based on science. Now, 100 years later that’s being unpicked both in the USA and here. ms Gritz seemed to think that Flexner would have approved of Dean Ornish. In a response I begged to differ. I’m pretty sure that Felxner would have been furious of he could have seen the reecent march of quackademia, particularly, but not exclusively, in the USA. It is exactly the sort of thing his report set out, successfully, to abolish. He wrote, for example,

“the practitioner is subjected, year in, year out, to the steady bombardment of the unscrupulous manufacturer, persuasive to the uncritical, on the principle that “what I tell you three times is true.” Against bad example and persistent asseveration, only precise scientific concepts and a critical
appreciation of the nature and limits of actual demonstration can protect the young physician.” (Flexner report, 1910, pp 64-65)

It is this very “appreciation of the nature and limits of actual demonstration” that is now being abandoned by the alternative medicine industry. despite the fact that real medicine was in its infancy at the time he w as writing, he was very perceptive about the problems. Perhaps Freedman should read the report.

David Katz seems to have spotted my piece in The Atlantic, and has responded at great length in the Huffington Post (quite appropriate, given the consistent support of HuffPo for nonsense medicine). HuffPo allows only short comments with no links so I’ll reply to him here.

I fear that Dr Katz doth protest a great deal too much. He seems to object to a comment that I made about him in The Atlantic.

“… [He] listed a lot of things he’d tried and which failed to work. His conclusion was not that they should be abandoned, but that we needed a ‘a more fluid concept of evidence.'”

You don’t have to take my word for it. You can take it from the words of Dr Katz.

"What do we do when the evidence we have learned, or if we care to be more provocative, with which we have been indoctrinated, does now fully meet the needs of our patients"

It seems odd to me to regard teaching about how you distinguish what;s true and what isn’t as "indoctrination", though I can understand that knowledge of that subject could well diminish the income of alternative practitioners. You went on to say

"Some years ago the CDC funded us with a million dollars to do what they referred to initially as a systematic review of the evidence base for complementary and alternative medicine,  Anybody who’s ever been involved in systematic reviews knows that’s a very silly thing. . . . Well we knew it was silly but a million dollars sounded real [mumbled] took the money and then we figurered we’d figure out what to do with it [smiles broadly]. That’s what we did ". . .

I do hope you told the CDC that you did not spend the million dollars for the sensible purpose for which it was awarded.

This infusion of calcium, magnesium and D vitamins and vitamin C ameliorates the symptoms of fibromyalgia.  . . .  We did typical placebo controlled randomized double-blind trial for several months . . . we saw an improvement in both our treatment and placebo groups . . .

You then describe how you tested yoga for asthma and homeopathy for attention deficit hyperactivity disorder, Neither of them worked either. Your reaction to this string of failures was just to say “we need a more fluid concept of evidence”

After telling an anecdote about one patient who got better after taking homeopathic treatment you said £I don’t care to get into a discussion of how, or even whether, homeopathy works”.  Why not?  It seems it doesn’t matter much to you whether the things you sell to patients work or not.

You then went on to describe quite accurately that anti-oxidants don’t work and neither do multivitamin supplements for prevention of cardiovascular problems,  And once again you fail to accept the evidence, even evidence you have found yourself. Your response was

“So here too is an invitation to think more fluidly about of evidence. Absence of evidence is not evidence of absence.” 

That last statement is the eternal cry of every quack.  It’s true, of course, but that does not mean that absence of evidence gives you a licence to invent the answer.  But inventing the answer is what you do, time after time, You seem quite incapable of saying the most important thing that anyone in your position should. I don’t know the answer.

Jump to follow-up

The King’s Fund recently published Assessing complementary practice Building consensus on appropriate research methods [or download pdf].

Report title

It is described as being the “Report of an independent advisory group”. I guess everyone knows by now that an “expert report” can be produced to back any view whatsoever simply by choosing the right “experts”, so the first things one does is to see who wrote it.  Here they are.

  • Chair: Professor Dame Carol Black
  • Harry Cayton, Chief Executive, Council for Healthcare Regulatory Excellence
  • Professor Adrian Eddleston, then Vice-Chairman, The King’s Fund
  • Professor George Lewith, Professor of Health Research, Complementary and Integrated Medicine Research Unit, University of Southampton
  • Professor Stephen Holgate, MRC Clinical Professor of Immunopharmacology, University of Southampton
  • Professor Richard Lilford, Head of School of Health and Population Sciences, University of Birmingham

We see at once two of the best known apologists for alternative medicine, George Lewith (who has appeared here more than once) and Stephen Holgate

Harry Cayton is CEO of Council for Healthcare Regulatory Excellence (CHRE) which must be one of the most useless box-ticking quangos in existence. It was the CHRE that praised the General Chiropractic Council (GCC) for the quality of its work.  That is the same GCC that is at present trying to cope with 600 or so complaints about the people it is supposed to regulate (not to mention a vast number of complaints to Trading Standards Offices).  The GCC must be the prime example of the folly of giving government endorsement to things that don’t work. But the CHRE were not smart enough to spot that little problem.  No doubt Mr Cayton did good work for the Alzheimer’s Society.  His advocacy of patient’s choice may have helped me personally.  But it isn’t obvious to me that he is the least qualified to express an opinion on research methods in anything whatsoever. According to the Guardian he is “BA in English and linguistics from the University of Ulster; diploma in anthropology from the University of Durham; B Phil in philosophy of education from the University of Newcastle.”

Adrian Eddlestone is a retired Professor of Medicine. He has been in academic administration since 1983. His sympathy for alternative medicine is demonstrated by the fact that he is also Chair of the General Osteopathic Council, yet another “regulator” that has done nothing to protect the public
from false health claims (and which may, soon, find itself in the same sort of trouble as the GCC).

Richard Lilford is the only member of the group who has no bias towards alternative medicine and also the only member with expertise in clinical research methods  His credentials look impressive, and his publications show how he is the ideal person for this job. I rather liked also his article Stop meddling and let us get on.. He has written about the harm done by postmodernism and relativism, the fellow-travellers of alternative medicine.

Most damning of all, Lewith, Eddlestone and Holgate (along with Cyril Chantler, chair of the King’s Fund, and homeopaths, spiritual healers and Karol Sikora) are Foundation Fellows of the Prince of Wales Foundation for Magic Medicine, an organisation that is at the forefront of spreading medical misinformation.

I shall refer here to ‘alternative medicine’ rather than ‘complementary medicine’ which is used in the report. It is not right to refer to a treatment as ‘complementary’ until such time as it has been shown to work. The term ‘complementary’ is a euphemism that, like ‘integrative’, is standard among alternative medicine advocates whose greatest wish is to gain respectability.

The Report

Kings Fund logo

The recommendations

On page 10 we find a summary of the conclusions.

The report identifies five areas of consensus, which together set a framework for moving forward. These are:

  • the primary importance of controlled trials to assess clinical and cost effectiveness.
  • the importance of understanding how an intervention works
  • the value of placebo or non-specific effects
  • the need for investment and collaboration in creating a sound evidence base
  • the potential for whole-system evaluation to guide decision-making and subsequent research.

The first recommendation is just great. The rest sound to me like the usual excuses for incorporating ineffective treatments into medical practice. Notice the implicit assumption in the fourth point
that spending money on research will establish “a sound evidence base". There is a precedent, but it is ignored. A huge omission from the report is that it fails to mention anywhere that a lot of research has already been done.

Much research has already been done (and failed)

The report fails to mention at all the single most important fact in this area. The US National Institutes of Health has spent over a billion dollars on research on alternative medicines, over a period
of more than 10 years. It has failed to come up with any effective treatments whatsoever. See, for example Why the National Center for Complementary and Alternative Medicine (NCCAM) Should Be Defunded;   Should there be more alternative research?;   Integrative baloney @ Yale, and most recently, $2.5B Spent, No Alternative Med Cures found. .

Why did the committee think this irrelevant? I can’t imagine. You guess.

The report says

“This report outlines areas of potential consensus to guide research funders, researchers, commissioners and complementary practitioners in developing and applying a robust evidence base for complementary practice.”

As happens so often, there is implicit in this sentence the assumption that if you spend enough money evidence will emerge. That is precisely contrary to the experence in the USA where spending a billion dollars produced nothing beyond showing that a lot of things we already thought didn’t work were indeed ineffective.

And inevitably, and tragically, NICE’s biggest mistake is invoked.

“It is noteworthy that the evidence is now sufficiently robust for NICE to include acupuncture as a treatment for low back pain.” [p ]

Did the advisory group not read the evidence used (and misinterpeted) by NICE? It seems not. Did the advisory group not read the outcome of NIH-funded studies on acupuncture as summarised by Barker Bausell in his book, Snake Oil Science? Apparently not. It’s hard to know because the report has no references.

George Lewith is quoted [p. 15] as saying “to starve the system of more knowledge means we will continue to make bad decisions”. No doubt he’d like more money for research, but if a billion dollars
in the USA gets no useful result, is Lewith really likely to do better?

The usual weasel words of the alternative medicine industry are there in abundance

“First, complementary practice often encompasses an intervention (physical treatment or manipulation) as well as the context for that intervention. Context in this setting means both the physical setting for the delivery of care and the therapeutic relationship between practitioner and patient.” [p. 12]

Yes, but ALL medicine involves the context of the treatment. This is no different whether the medicine is alternative or real. The context (or placebo) effect comes as an extra bonus with any sort of treatment.

“We need to acknowledge that much of complementary practice seeks to integrate the positive aspects of placebo and that it needs to be viewed as an integral part of the treatment rather than an aspect that should be isolated and discounted.” [p. 13]

This is interesting. It comes very close (here and elsewhere) to admitting that all you get is a placebo effect, and that this doesn’t matter. This contradicts directly the first recommendation of the House of Lords report (2000).. Both the House of Lords report on Complementary and Alternative Medicine, and the Government’s response to it, state clearly

“. . . we recommend that three important questions should be addressed in the following order”. (1) does the treatment offer therapeutic benefits greater than placebo? (2)  is the treatment safe? (3) how does it compare, in medical outcome and cost-effectiveness, with other forms of treatment?.

The crunch comes when the report gets to what we should pay for.

“Should we be prepared to pay for the so-called placebo effect?

The view of the advisory group is that it is appropriate to pay for true placebo (rather than regression to the mean or temporal effects).” [p 24]

Perhaps so, but there is very little discussion of the emormous ethical questions:that this opinion raises: 

  • How much is one allowed to lie to patients in order to elicit a placebo effect?
  • Is is OK if the practitioner believes it is a placebo but gives it anyway?
  • Is it OK if the pratitioner believes that it is not a placebo when actually it is?
  • Is it OK for practitioners to go degrees taught by people who believe that it is not a placebo when actually it is?

The report fails to face frankly these dilemmas.  The present rather absurd position in which it is considered unethical for a medical practitioner to give a patient a bottle of pink water, but
perfectly acceptable to refer them to a homeopath. There is no sign either of taking into account the cultural poison that is spread by telling people about yin, yang and meridians and such like preposterous made-up mumbo jumbo.  That is part of the cost of endorsing placebos. And just when one thought that believing things because you wished they were true was going out of fashion

Once again we hear a lot about the alleged difficulties posed by research on alternative medicine. These alleged difficulties are, in my view, mostly no more than excuses. There isn’t the slightest
difficulty in testing things like herbal medicine or homeopathy, in a way that preserves all the ‘context’ and the ways of working of homeopaths and herbalists. Anyone who reads the Guardian knows
how to do that.

In the case of acupuncture, great ingenuity has gone into divising controls. The sham and the ‘real’ acupuncture always come out the same. In a non-blind comparison between acupuncture and no acupuncture the latter usually does a bit worse, but the effects are small and transient and entirely compatible with the view that it is a theatrical placebo.

Despite these shortcomings, some of the conclusions [p. 22] are reasonable.

“The public needs more robust evidence to make informed decisions about the use of complementary practice.

Commissioners of public health care need more robust evidence on which to base decisions about expenditure of public money on complementary practice.”

What the report fails to do is to follow this with the obvious conclusion that such evidence is largely missing and that until such time as it is forthcoming there should be no question of the NHS paying for alternative treatments.

Neither should there be any question of giving them official government recognition in the form of ‘statutory regulation’. The folly of doing that is illustrated graphically by the case of chiropractic which is now in deep crisis after inspection of its claims in the wake of the Simon Singh defamation case. Osteopathy will, I expect, suffer the same fate soon.

In the summary on p.12 we see a classical case of the tension

Controlled trials of effectiveness and cost-effectiveness are of primary importance

We recognise that it is the assessment of effectiveness that is of primary importance in reaching a judgement of different practices. Producing robust evidence that something works in practice – that it is effective – should not be held up by the inevitably partial findings and challenged interpretations arising from inquiries into how the intervention works.

The headline sounds impeccable, but directly below it we see a clear statement that we should use treatments before we know whether they work.  “Effectiveness”, in the jargon of the alternative medicine business, simply means that uncontrolled trials are good enough. The bit about “how it works” is another very common red herring raised by alternative medicine people. Anyone who knows anything about pharmacology that knowledge about how any drug works is incomplete and often turns out to be wrong. That doesn’t matter a damn if it performs well in good double-blind randomised controlled trials.

One gets the impression that the whole thing would have been a lot worse without the dose of reality injected by Richard Lilford. He is quoted as a saying

“All the problems that you find in complementary medicine you will encounter in some other kind of treatment … when we stop and think about it… how different is it to any branch of health care – the answer to emerge from our debates is that it may only be a matter of degree.” [p. 17]

I take that to mean that alternative medicine poses problems that are no different from other sorts of treatment. They should be subjected to exactly the same criteria. If they fail (as is usually the case) they should be rejected.  That is exactly right.  The report was intended to produce consensus, but throughout the report, there is a scarcely hidden tension between believers on one side, and Richard Lilford’s impeccable logic on the other.

Who are the King’s Fund?

The King’s Fund is an organisation that states its aims thus.

“The King’s Fund creates and develops ideas that help shape policy, transform services and bring about behaviour change which improve health care.”

It bills this report on its home page as “New research methods needed to build evidence for the effectiveness of popular complementary therapies”. But in fact the report doesn’t really recommend ‘new research methods’ at all, just that the treatments pass the same tests as any other treatment. And note the term ‘build evidence’.  It carries the suggestion that the evidence will be positive.   Experience in the USA (and to a smaller extent in the UK) suggests that every time some good research is done, the effect is not to ‘build evidence’ but for the evidence to crumble further

If the advice is followed, and the results are largely negative, as has already happened in the USA, the Department of Health would look pretty silly if it had insisted on degrees and on statutory regulation.

The King’s Fund chairman is Sir Cyril Chantler and its Chief Executive is Niall Dickson.  It produces reports, some of which are better than this one. I know it’s hard to take seriously an organisation that wants to “share its vision” withyou, but they are trying.

“The King’s Fund was formed in 1897 as an initiative of the then Prince of Wales to allow for the collection and distribution of funds in support of the hospitals of London. Its initial purpose was to raise money for London’s voluntary hospitals,”

It seems to me that the King’s Fund is far too much too influenced by the present Prince of Wales. He is, no doubt, well-meaning but he has become a major source of medical misinformation and his influence in the Department of Health is deeply unconstitutional.  I was really surprised to see thet Cyril Chantler spoke at the 2009 conference of the Prince of Wales Foundation for Integrated Health, despite having a preview of the sort of make-believe being propagated by other speakers. His talk there struck me as evading all the essential points. Warm, woolly but in the end, a danger to patients. Not only did he uncritically fall for the spin on the word “integrated”, but he also fell for the idea that “statutory regulation” will safeguard patients.

Revelation of what is actually taught on degrees in these subjects shows very clearly that they endanger the public.

But the official mind doesn’t seem ever to look that far. It is happy ticking boxes and writing vacuous managerialese. It lacks curiosity.

Follow-up

The British Medical Journal published today an editorial which also recommends rebranding of ‘pragmatic’ trials.  No surprise there, because the editorial is written by Hugh MacPherson, senior research fellow, David Peters, professor of integrated healthcare and Catherine Zollman, general practitioner. I find it a liitle odd that the BMJ says “Competing Interests: none. David Peters interest is obvious from his job description. It is less obvious that Hugh MacPherson is an acupuncture enthusiast who publishes mostly in alternative medicine journals. He has written a book with the extraordinary title “Acupuncture Research, Strategies for Establishing an Evidence Base”. The title seems to assume that the evidence base will materialise eventually despite a great deal of work that suggests it won’t. Catherine Zollman is a GP who is into homeopathy as well as acupuncture. All three authors were speakers at the Prince of Wales conference, described at Prince of Wales Foundation for magic medicine: spin on the meaning of ‘integrated’.

The comments that follow the editorial start with an excellent contribution from James Matthew May. His distinction between ‘caring’ and ‘curing’ clarifies beautifully the muddled thinking of the editorial.

Then a comment from DC, If your treatments can’t pass the test, the test must be wrong. It concludes

“At some point a stop has to be put to this continual special pleading. The financial crisis (caused by a quite different group of people who were equally prone to wishful thinking) seems quite a good time to start.”