# DC's Improbable Science

### Truth, falsehood and evidence: investigations of dubious and dishonest science

#### September 25th, 2012 · 16 Comments

 This is a very important book. Buy it now (that link is to Waterstone’s Amazon don’t pay tax in the UK, so don’t use them). When you’ve read it, do something about it. The book has lots of suggestions about what to do. Stolen from badscience.net

Peter Medawar, the eminent biologist, in his classic book Advice to a Young Scientist, said this.

“Exaggerated claims for the efficacy of a medicament are very seldom the consequence of any intention to deceive; they are usually the outcome of a kindly conspiracy in which everybody has the very best intentions. The patient wants to get well, his physician wants to have made him better, and the pharmaceutical company would have liked to have put it into the physician’s power to have made him so. The controlled clinical trial is an attempt to avoid being taken in by this conspiracy of good will.”

There was a lot of truth in that 1979, towards the end of the heyday of small molecule pharmacology.  Since then, one can argue, things have gone downhill.

First, though, think of life without general anaesthetics, local anaesthetics, antibiotics, anticoagulants and many others.  They work well and have done incalculable good.  And they were developed by the drug industry.

But remember also that remarkably little is known about medicine.  There are huge areas in which neither causes nor cures are known.  Treatments for chronic pain, back problems, many sorts of cancer and almost all mental problems are a mess.  It just isn’t known what to do.  Nobody is to blame for this.  Serious medical research has been going on for little more than 60 years, and it turns out to be very complicated.  We are doing our best, but are still ignorant about whole huge areas. That leads to a temptation to make things up. Clutching at straws is very evident when it comes to depression, pain and Alzheimer’s disease, among others.

In order to improve matters, one essential is to do fair tests on treatments that we have.  Ben Goldacre’s book is a superb account of how this could be done, and how the process of testing has been subverted for commercial gain and to satisfy the vanities of academics.

Of course there is nothing new in criticisms of Big Pharma.  The huge fines levied on them for false advertising are well known.  The difference is that Goldacre’s book explains clearly what’s gone wrong in great detail, documents it thoroughly, and makes concrete suggestions for improving matters.

Big Pharma has undoubtedly sometimes behaved appallingly in recent years. Someone should be in jail for crimes against patients.  They have behaved in much the same way that bankers have. In any huge globalised industry it is always possible to blame someone in another department for the dishonesty.  But they aren’t the only people to blame.  None of the problems could have arisen with the complicity of academics, universities, and a plethora of regulatory agencies and professional bodies.

The biggest scandal of all is missing data (chapter 1).  Companies, and sometmes academics, have suppressed of trials that don’t favour the drugs that they are trying to sell.  The antidepressant drug, reboxetine, appeared at first to be good. It had been approved by the Medicines and Healthcare products Regulatory Agency (MHRA) and there was at least one good randomized placebo-controlled trial (RCT) showing it worked.  But it didn’t.  The manufacturer didn’t provide a complete list of unpublished trials when asked for them.  After much work it was found in 2010 that, as well as the published, favourable trial, there were six more trials which had not been published and all six showed reboxetine to be no better than placebo .  In comparisons with other antidepressant drugs three small studies (507 patients) showed reboxetine to be as good as its competitors.  These were published. But it came to light that data on 1657 patients had never been published and these showed reboxetine to be worse than its rivals.

When all the data for the SSRI antidepressants were unearthed (Kirsch et al., 2008) it turned out that they were no better than placebo for mild or moderate depression. This selective suppression of negative data has happened time and time again. It harms patients and deceives doctors, but, incredibly, it’s not illegal.

Disgracefully, Kirsch et al. had to use a Freedom of Information Act request to get the data from the FDA.

“The output of a regulator is often simply a crude, brief summary: almost a ‘yes’ or ‘no’ about side effects. This is the opposite of science, which is only reliable because everyone shows their working, explains how they know that something is effective or safe, shares their methods and their results, and allows others to decide if they agree with the way they processed and analysed the data.”

 “the NICE document discussing whether it’s a good idea to have Lucentis, an extremely expensive drug, costing well over £ 1,000 per treatment, that is injected into the eye for a condition called acute macular degeneration. As you can see, the NICE document on whether this treatment is a good idea is censored. Not only is the data on the effectiveness of the treatment blanked out by thick black rectangles, in case any doctor or patient should see it, but absurdly, even the names of some trials are missing, preventing the reader from even knowing of their existence, or cross referencing information about them.Most disturbing of all, as you can see in the last bullet point, the data on adverse events is also censored.”

The book lists all the tricks that are used by both industry and academics. Here are some of them.

• Regulatory agencies like the MHRA, the European Medicines Agency (EMA) and the US Food and Drugs Administration (FDA) set a low bar for approval of drugs.
•  Companies make universities sign gagging agreements which allow unfavourable results to be suppressed, and their existence hidden.
• Accelerated approval schemes are abused to get quick approval of ineffective drugs and the promised proper tests often don’t materialise
• Disgracefully, even when all the results have been given to the regulatory agencies (which isn’t always). The MHRA, EMA and FDA don’t make them public. We are expected to take their word.
• Although all clinical trials are meant to be registered before they start, the EMA register, unbelievably, is not public.  Furthermore there is no check that the results if trials ever get published.  Despite mandates that results must be published within a year of finishing the trial, many aren’t.  Journals promise to check this sort of thing, but they don’t.
• When the results are published, it is not uncommon for the primary outcome, specified before it started, to have been changed to one that looks like a more favourable result.  Journals are meant to check, but mostly don’t.
• Companies use scientific conferences, phony journals, make-believe “seed trials” and “continuing medical education” for surreptitious advertising.
• Companies invent new diseases, plant papers to make you think you’re abnormal, and try to sell you a “cure”.  For example, female sexual dysfunction , restless legs syndrome and social anxiety disorder (i.e. shyness).  This is called disease-mongering, medicalisation or over-diagnosis. It’s bad.
• Spin is rife. Companies, and authors, want to talk up their results. University PR departments want to exaggerate benefits. Journal editors want sensational papers. Read the results, not the summary. This is universal (but particularly bad in alternative medicine).
• Companies fund patient groups to lobby for pills even when the pills are known to be ineffective.  The lobby that demanded that Herceptin should be available to all on the breast cancer patients on the NHS was organised by a PR company working for the manufacturer, Roche.  But Herceptin doesn’t work at all in 80% of patients and gives you at best a few extra months of  life in advanced cases.
• Ghostwriting of papers is serious corruption.  A company writes the paper and senior academics appear as the authors, though they may never have seen the original data.  Even in cases where academics have admitted to lying about whether they have seen the data, they go unpunished by their universities. See for example, the case of Professor Eastell.
• By encouraging the funding of “continuing medical education” by companies, the great and the good of academic medicine have let us down badly.

This last point is where the book ends, and it’s worth amplification.

“So what have the great and good of British medicine done to help patients, in the face of this endemic corruption, and these systematic flaws? In 2012, a collaborative document was produced by senior figures in medicine from across the board, called ‘Guidance on Collaboration Between Healthcare Professionals and the Pharmaceutical Industry’. This document was jointly approved by the ABPI, the Department of Health, the Royal Colleges of Physicians, Nursing, Psychiatrists, GPs, the Lancet, the British Medical Association, the NHS Confederation, and so on. ”

“It contains no recognition of the serious problems we have seen in this book. In fact, quite the opposite: it makes a series of assertions about them that are factually incorrect.”

“It states that drug reps ‘can be a useful resource for healthcare professionals’. Again, I’m not sure why the Royal Colleges, the BMA, the Department of Health and the NHS Confederation felt the need to reassert this to the doctors of the UK, on behalf of industry, when the evidence shows that drug reps actively distort prescribing practices. But that is the battle you face, trying to get these issues taken seriously by the pinnacle of the medical establishment.”

This is perhaps the most shameful betrayal of all.  The organisations that should protect patients have sold them out.

You may have been sold out by your “elders and betters”, but you can do something. The “What to do” sections of the book should be produced as a set of flash cards, as a reminder that matters can be improved.

It is shameful that this book was not written by a clinical pharmacologist, or a senior doctor, or a Royal College, or a senior academic.  Why has the British Pharmacological Society said nothing?

It is shameful too that too was not written by one of the quacks who are keen to defend the $60 billion alternative medicine industry (which has cured virtually nothing) and who are strident in their criticism of the 600 billion dollar Pharma industry. They haven’t done the work that Goldacre has to analyse the real problems. All they have done is to advocate unfair tests, because that is the only sort their treatments can pass. It’s weird that medicine, the most caring profession, is more corrupt than any other branch of science. The reason, needless to say, is money. Well, money and vanity. The publish or perish mentality of senior academics encourages dishonesty. It is a threat to honest science. Goldacre’s book shows the consequences: harm to patients and huge wastage of public money. Read it. Do something. ### Follow-up 7 October, 2012, The Observer Goldacre wrote "I think it’s really disappointing that nobody, not the Royal Colleges, the Academy of Medical Sciences, the British Pharmacological Society, the British Medical Association, none of these organisations have stood up and said: selective non-publication of unflattering trial data is research misconduct, and if you do it you will be booted out. And I think they really urgently should." Exactly. ### 16 responses so far ↓ • 1 Andrew // Sep 25, 2012 at 23:07 Thanks for the review, David – I shall certainly order a copy. Unfortunately, none of those bullet points above come as any surprise – I have also seen them appear in slightly different forms in academia. (For example, I once recollect hearing a postgraduate student complaining that his study didn’t come out the way he wanted, therefore he and his supervisor would have to run it again.) My only concern about this book is that it will be [wrongly] used as evidence for why some of the quirkier forms of CAM are good and orthodox medicine is bad. Andrew Gilbey, NZ • 2 David Colquhoun // Sep 25, 2012 at 23:45 @Andrew Gilby I’ve no doubt you are right. But the alt med fantasists are increasingly irrelevant. An army of nerds has arisen and exposed their delusions, and the harm that they sometimes do, The problems of real medicine seem to me to be much more important. Although Ben (and I) have been writing about it for years, it hasn’t stopped the accusations that we are in the pockets of Big Pharma. A minor benefit of this book might be to stop such accusations. Though I doubt it. • 3 Majikthyse // Sep 26, 2012 at 00:01 I haven’t read the book yet, but I think I can trust what Ben is saying. However, to pick up on “drug reps actively distort prescribing practices”, it would be very odd if they didn’t. They are paid to influence doctors. Is there something immoral about that? There is not a product on this planet that doesn’t require somebody to sell it. Important new drugs (a tiny minority admittedly) would simply not be prescribed if their companies didn’t promote them. I do hope that Ben has acknowledged the grinding effort that thousands of industry scientists expend to obtain clinical data fairly. The message seems to be that they are let down by their business colleagues, something that I have repeatedly seen over nearly 40 years with Big Pharma. • 4 robbo // Sep 26, 2012 at 09:24 All at a time when the funding agencies and senior academics are forcing us down the path of “translational” science. When what we should be doing in universities is undirected, curiosity driven science that is not output driven, but REF2014 will put a stop to that! • 5 Lindy // Sep 26, 2012 at 09:53 *I’ve not read the book yet, but taking the general thrust of what Ben is saying, it makes you wonder how health services will develop when the NHS is taken over completely by private firms. How many unecessary and expensive treatments will be carried out just to make profit? How many people will be seen by unqualified ‘health-care providers’ rather than doctors of nurses – just to make profit? This combined with the tendency of big Pharma to promote not only its products but various diseases, (see writings by Ray Moynihan and others) it seems likely that all health services will be very expensive indeed, to the benefit of the few rather than for the health and care of the many. • 6 David Colquhoun // Sep 26, 2012 at 10:37 @Majikthyse I agree entirely that scientists in Big Pharma are, by and large, totally honest. There is reason to believe that trials designed by Big Pharma are often of higher quality than those in academia. And as Ben and I both say, we must never forget that it is the industry that has discovered many treatments that do work well. We can’t do without them. But the industry spends twice as much on marketing as it spends on research and development. That seems to be where all the problems arise. All big companies spend a lot on marketing. The job of the marketer is to exaggerate and deceive. Is this immoral? I guess it is just a fact of life and we have to live with it. But dishonest marketing is bad enough in the case of banks. If the outcome is to harm patients, it’s disgraceful. It’s only to be expected that a big industry will behave in a dishonest way to further its own interests. Pharma, like banks, need to be properly regulated to curb their worst excesses. The real concern is the failure of regulators, academics and professional bodies to do the job expected of them. They have, to a greater or lesser extent, sold out. That’s unforgivable and something must be done about it. • 7 David Colquhoun // Sep 26, 2012 at 10:59 @Lindy I fear your are quite right. The government seems intent on heading towards a US system of medicine which costs twice as much and doesn’t work as well. The only way that the private providers can make money is by selling unnecessary and expensive treatments. Or by charging the taxpayer a lot for cheap and ineffective treatments. It’s no surprise that companies like Virgin Care and Connect Physical Health are happy to sell acupuncture and “craniosacral therapy”. I can only hope that the government pay the price for their attempts to destroy the NHS. It was all summed up for me by the most trenchant tweet that I saw in the run-up to the passing of the wretched Health and Social Care Act. It was from @HeardInLondon, on March 15, 2012. “For a brief period during 20th century, people gave a fuck and looked after each other. Unfortunately this proved unprofitable.” • 8 Majikthyse // Sep 26, 2012 at 11:24 You are quite right David, as with the banks the problem is with regulation. It has been repeatedly shown that self-regulation is inadequate, eg the GMC, the supine ABPI, the GCC etc. The MHRA is a bit better than the dreadful FDA, and it’s widely acknowledged that the EU GCP and Clinical Trials Directives have their toughest implementation in the UK. I wonder if the problem is with the way the MHRA is funded, from product licence fees and inspection fees, and not taxation. It is in the MHRA’s interest to ramp up the number of licence applications. Rather like academic publishing, it’s a numbers game. I am very doubtful that medical assessors at the MHRA really look at the data as carefully as they should, because of limited resource. Even more importantly, they don’t challenge the submission package and obtain assurances that they are seeing ALL the data. • 9 stephenemoss // Sep 26, 2012 at 16:58 Our copy of Bad Pharma arrived today, funnily enough not long after I took delivery of ‘Genentech: the Beginnings of Biotech’ by Sally Smith Hughes. Genentech is the company that developed Lucentis, the drug for which NICE produced the bizarrely censored document you show above. I’m mystified why NICE would do this, particularly since in this case the drug is actually fairly effective and the incidence of adverse side-effects is low. Since becoming the front-line treatment for AMD there have been more than 1000 peer-reviewed publications on its efficacy – whether its worth it at £1000 per monthly injection is another question, but for those patients for whom it works, it means they can continue to read, drive etc. Minor point for Ben come the second edition. AMD is ‘age-related macular degeneration’, not acute macular degeneration. • 10 adzcliff // Sep 28, 2012 at 19:01 “It is shameful too that too was not written by one of the quacks who are keen to defend the$60 billion alternative medicine industry (which has cured virtually nothing) and who are strident in their criticism of the 600 billion dollar Pharma industry.”

Not quite sure what’s meant by that statement?  Is it just accusing the lt med community of being all mouth and no trousers?  I guess I try to occupy a position where I favour honest evidence-based healthcare, and oppose dishonest/delusional nevidence-based healthcare wherever I see it.  I guess I don’t need or expect any one group of the latter writing authoritatively on the ills of another?

• 11 David Colquhoun // Sep 28, 2012 at 20:21

Your position is much the same as mine.

For years, any criticism whatever of the evidence for an alternative therapy has been met with a blast about the evils of big pharma.  If the alties had any sense they’d have martialled the evidence properly as Goldacre has done.  In fact just about every valid criticism of the status quo has been rooted out by real scientists, not by the quacks.  In that sense, science really is self-correcting, though it can be a slow process.

• 12 Should my wife volunteer for Synexus osteoporosis trial? // Nov 17, 2012 at 12:55

[...] dose of something not very effective, is one of the stratagems listed in Chapter 4, Bad trials, in Ben Goldacre’s Bad Pharma. It makes the new drug look good, but it asks the wrong [...]

• 13 Transparent pills will make the best medicine | My Blog // Dec 25, 2012 at 00:57

[...] David Colquhoun, Richard Smith (former British Medical Journal Editor), Alice Bell (for the New Left Project), and Michael Rawlins of NICE, among others, have already reviewed the book in some depth. While some interesting points of criticism were raised, all generally applauded the forensic attention that Goldacre has brought to the drugs business. The accuracy of his aim is underscored by the weakness of the riposte from Stephen Whitehead, CEO of the Association of the British Pharmaceutical Industry. [...]

• 14 Junking ‘Junk Science?’ « Lee Turnpenny // Feb 8, 2013 at 12:11

[...] Hmmm. Yet, when it comes to comment on the ‘vested interests’ of Big Pharma, hasn’t the rug been pulled from under CAM’s feet by the ‘misleading and vacuous opinions’ of that renowned sceptic, Ben Goldacre? Now, where was it I saw particularly good review and comment on his book? Oh yeah… [...]

• 15 Junking ‘Junk Science?’ | The Mawk Moth Profligacies // Feb 8, 2013 at 12:59

[...] Hmmm. Yet, when it comes to comment on the 'vested interests' of Big Pharma, hasn't the rug been pulled from under CAM's feet by the 'misleading and vacuous opinions' of that renowned sceptic, Ben Goldacre? Now, where was it I saw particularly good review and comment on his book? Oh yeah… [...]

• [...] above all, to Ben Goldacre (1), more people than ever know that the big pharma companies have been concealing evidence of the [...]