[This an update of a 2006 post on my old blog]
The New York Times (17 January 2006) published a beautiful spoof that illustrates only too clearly some of the bad practices that have developed in real science (as well as in quackery). It shows that competition, when taken to excess, leads to dishonesty.
More to the point, it shows that the public is well aware of the dishonesty that has resulted from the publish or perish culture, which has been inflicted on science by numbskull senior administrators (many of them scientists, or at least ex-scientists). Part of the blame must attach to "bibliometricians" who have armed administrators with simple-minded tools the usefulness is entirely unverified. Bibliometricians are truly the quacks of academia. They care little about evidence as long as they can sell the product.
The spoof also illustrates the folly of allowing the hegemony of a handful of glamour journals to hold scientists in thrall. This self-inflicted wound adds to the pressure to produce trendy novelties rather than solid long term work.
It also shows the only-too-frequent failure of peer review to detect problems.
The future lies on publication on the web, with post-publication peer review. It has been shown by sites like PubPeer that anonymous post-publication review can work very well indeed. This would be far cheaper, and a good deal better than the present extortion practised on universities by publishers. All it needs is for a few more eminent people like mathematician Tim Gowers to speak out (see Elsevier – my part in its downfall).
Recent Nobel-prizewinner Randy Schekman has helped with his recent declaration that "his lab will no longer send papers to Nature, Cell and Science as they distort scientific process"
The spoof is based on the fraudulent papers by Korean cloner, Woo Suk Hwang, which were published in Science, in 2005. As well as the original fraud, this sad episode exposed the practice of ‘guest authorship’, putting your name on a paper when you have done little or no work, and cannot vouch for the results. The last (‘senior’) author on the 2005 paper, was Gerald Schatten, Director of the Pittsburgh Development Center. It turns out that Schatten had not seen any of the original data and had contributed very little to the paper, beyond lobbying Scienceto accept it. A University of Pittsburgh panel declared Schatten guilty of “research misbehavior”, though he was, amazingly, exonerated of “research misconduct”. He still has his job. Click here for an interesting commentary.
The New York Times carried a mock editorial to introduce the spoof..
One Last Question: Who Did the Work?
By NICHOLAS WADE
In the wake of the two fraudulent articles on embryonic stem cells published in Science by the South Korean researcher Hwang Woo Suk, Donald Kennedy, the journal’s editor, said last week that he would consider adding new requirements that authors “detail their specific contributions to the research submitted,” and sign statements that they agree with the conclusions of their article.
A statement of authors’ contributions has long been championed by Drummond Rennie, deputy editor of The Journal of the American Medical Association,
Explicit statements about the conclusions could bring to light many reservations that individual authors would not otherwise think worth mentioning. The article shown [below] from a future issue of the Journal of imaginary Genomics, annotated in the manner required by Science‘s proposed reforms, has been released ahead of its embargo date.
The old-fashioned typography makes it obvious that the spoof is intended to mock a paper in Science.
The problem with this spoof is its only too accurate description of what can happen at the worst end of science.
Something must be done if we are to justify the money we get and and we are to retain the confidence of the public
My suggestions are as follows
- Nature Science and Cell should become news magazines only. Their glamour value distorts science and encourages dishonesty
- All print journals are outdated. We need cheap publishing on the web, with open access and post-publication peer review. The old publishers would go the same way as the handloom weavers. Their time has past.
- Publish or perish has proved counterproductive. You’d get better science if you didn’t have any performance management at all. All that’s needed is peer review of grant applications.
- It’s better to have many small grants than fewer big ones. The ‘celebrity scientist’, running a huge group funded by many grants has not worked well. It’s led to poor mentoring and exploitation of junior scientists.
- There is a good case for limiting the number of original papers that an individual can publish per year, and/or total grant funding. Fewer but more complete papers would benefit everyone.
- Everyone should read, learn and inwardly digest Peter Lawrence’s The Mismeasurement of Science.
3 January 2014.
Yet another good example of hype was in the news. “Effect of Vitamin E and Memantine on Functional Decline in Alzheimer Disease“. It was published in the Journal of the American Medical Association. The study hit the newspapers on January 1st with headlines like Vitamin E may slow Alzheimer’s Disease (see the excellent analyis by Gary Schwitzer). The supplement industry was ecstatic. But the paper was behind a paywall. It’s unlikely that many of the tweeters (or journalists) had actually read it.
The trial was a well-designed randomised controlled trial that compared four treatments: placebo, vitamin E, memantine and Vitamin E + memantine.
Reading the paper gives a rather different impression from the press release. Look at the pre-specified primary outcome of the trial.
The primary outcome measure was
" . . the Alzheimer’s Disease Cooperative Study/Activities of Daily Living (ADCSADL) Inventory.12 The ADCS-ADL Inventory is designed to assess functional abilities to perform activities of daily living in Alzheimer patients with a broad range of dementia severity. The total score ranges from 0 to 78 with lower scores indicating worse function."
It looks as though any difference that might exist between the four treaments is trivial in size. In fact the mean difference between Vitamin E and placebos was only 3.15 (on a 78 point scale) with 95% confidence limits from 0.9 to 5.4. This gave a modest P = 0.03 (when properly corrected for multiple comparisons), a result that will impress only those people who regard P = 0.05 as a sort of magic number. Since the mean effect is so trivial in size that it doesn’t really matter if the effect is real anyway.
It is not mentioned in the coverage that none of the four secondary outcomes achieved even a modest P = 0.05 There was no detectable effect of Vitamin E on
- Mean annual rate of cognitive decline (Alzheimer Disease Assessment Scale–Cognitive Subscale)
- Mean annual rate of cognitive decline (Mini-Mental State Examination)
- Mean annual rate of increased symptoms
- Mean annual rate of increased caregiver time,
The only graph that appeared to show much effect was The Dependence Scale. This scale
“assesses 6 levels of functional dependence. Time to event is the time to loss of 1 dependence level (increase in dependence). We used an interval-censored model assuming a Weibull distribution because the time of the event was known only at the end of a discrete interval of time (every 6 months).”
It’s presented as a survival (Kaplan-Meier) plot. And it is this somewhat obscure secondary outcome that was used by the Journal of the American Medical Assocciation for its publicity.
Note also that memantine + Vitamin E was indistinguishable from placebo. There are two ways to explain this: either Vitamin E has no effect, or memantine is an antagonist of Vitamin E. There are no data on the latter, but it’s certainly implausible.
The trial used a high dose of Vitamin E (2000 IU/day). No toxic effects of Vitamin E were reported, though a 2005 meta-analysis concluded that doses greater than 400 IU/d "may increase all-cause mortality and should be avoided".
In my opinion, the outcome of this trial should have been something like “Vitamin E has, at most, trivial effects on the progress of Alzheimer’s disease”.
Both the journal and the authors are guilty of disgraceful hype. This continual raising of false hopes does nothing to help patients. But it does damage the reputation of the journal and of the authors.
This paper constitutes yet another failure of altmetrics. (see more examples on this blog). Not surprisingly, given the title, It was retweeted widely, but utterly uncritically. Bad science was promoted. And JAMA must take much of the blame for publishing it and promoting it.
Today is a good day for anyone who deplores dangerous confidence tricksters. In particular it is a good day for Ben Goldacre, and for the Guardian which defended him at potentially enormous expense.
|Matthias Rath, the Dutch (or is it German) vitamin salesman has dropped his libel action against the Guardian. He is the man who is, without doubt, responsible for many deaths form AIDS in Africa, as a result of peddling vitamin pills as cures. The action was taken after Goldacre said, in the Guardian, that Rath aggressively sells his message to Aids victims in South Africa that Rath vitamin pills are better than medication”.
Here is some of what has appeared already today
Fall of the doctor who said his vitamins would cure Aids – from The Guardian, with a video of the villain.
Then more in the Guardian the next day, Chris McGreal investigates the Rath Foundation
Let’s be clear about what the words mean. Nutritional therapists are not like dietitians, and they are not like nutritionists. Nutritional therapists are solidly in the camp of alternative medicine practitioners, Don’t
take my word for it. They say so themselves.
“For nutritional therapists (who practise Complementary and Alternative Medicine) optimum nutrition encompasses individual prescriptions for diet and lifestyle in order to alleviate or prevent ailments and to promote optimal gene expression through all life stages. Recommendations may include guidance on natural detoxification, procedures to promote colon health, methods to support digestion and absorption, the avoidance of toxins or allergens and the appropriate use of supplementary nutrients, including phytonutrients.”
They love to use imaginary words like “detoxification”, and, much more dangerously, they love to pretend that they can cure diseases by changes in diet. As long as you buy from them a stack of expensive “supplement” pills, of course. That means they are selling medicines, but by pretending they are selling food supplements they manage to evade the law that requires medicines to be safe and effective. That will not be so easy under new legislation though, and we can look forward to a few prosecutions soon.
The University of Westminster
On their web site we learn that the Course Leader is Heather Rosa, and the Deputy Course Leader is Val Harvey. Harvey qualified in the subject at the Institute of Optimum Nutrition, the private college run by none other than the famous pill-peddler, Patrick Holford, about whom so very much has been written (try Holfordwatch, or the masterly chapter in Goldacre’s Bad Science)
We don’t know much about what is taught on the Nutritional Therapy course because the University of Westminster has refused repeated requests to say (but watch this space).. One can only assume that, whatever it is, they are not very proud of it. It seems a little unlikely that they will go as far as Matthias Rath and claim to cure AIDS -we’ll just have to wait and see. Meanwhile we can get an inkling by looking elsewhere.
Course leader, Heather Rosa, pops up for example, on the expert panel of a web site called Supplements Compared.com. “Supplements Compared is designed to help you find the best dietary supplement product for your health needs.” And what sort of advice do you find there? Try the page that compares 10 brands of CoQ10 (that is the stuff I wrote about recently, in “Boots reaches new level of dishonesty with CoQ10 promotion” – their advertising was deemed improper by the ASA ). It isn’t a recommended treatment for anything at all, but you certainly wouldn’t guess that from what is written by the ‘expert panel’. The winners are, according to the ‘expert panel’, Boots’ CoQ10 and Holland and Barrett’s CoQ10. Winners? Perhaps the explanation for that comes elsewhere, under “How are we funded?”. “Manufacturers who are awarded “best product” and “worth a look” are given the opportunity to promote this fact throughout the site for an additional fee.”. Well well.
Deputy Course leader, Val Harvey has her own web site and business (I do hope thar Westminster does not pay these people a full time salary too). What can we glean from there? It has the usual scare tactics “Why
you are at risk?“. Never fear; buy enough vitamin pills and you’ll be saved.
Her home page makes some pretty drastic claims.
“Potential health benefits of your nutritional programme
An appropriate Nutritional Programme can benefit many conditions including:
Chronic degenerative diseases
Chronic fatigue, ME
Depression, mood swings
Digestive or bowel problems
Eczema, psoriasis, other skin problems
Hypertension or elevated cholesterol
Irritable bowel syndrome
Parasitic and fungal infections
Premenstrual syndrome (PMS)
and many others ….
These are just some of the wide range of health problems that may be helped by nutritional therapy. Even those who consider themselves well and healthy may be able to enhance their physical and mental health, as well as their performance, including athletic performance, by improving their nutrition.”
There is, in my view, not the slightest bit of good evidence that swallowing vitamin pills can benefit most of these conditions.
But at least the list doesn’t contain AIDS, so is all this really relevant to the case of Matthias Rath?
Yes, I believe it is. The University of Westminster may well not support the views of Matthias Rath (they won’t say), but we have heard no choruses of protests about him from any nutritional therapists, as far as I’m aware. There is no mention of him at all on the web site of the British Association of Nutritional Therapists (BANT), the UK club for these people. BANT, by the way, has a rather curious code of ethics. It allows its members to take undisclosed financial kickbacks for the pills they prescribe to patients. If doctors were caught doing that they’d be struck off the register.
It is the existence of degrees in subjects like “nutritional therapy” that gives the subject a spurious air of respectability which allows seriously dangerous people like Rath to flourish with very little criticism. In an indirect way, the vice-chancellors who allow it to flourish (and Universities UK who do nothing about it) must bear some small part of the responsibility for the deaths of thousands of people from AIDS.
It is about time they did something about it.
ANH. The first reaction from the supplement-peddling industry comes from the Alliance for Natural Health on 16th September. It contains not one word of condemnation for Rath’s murderous activities. It’s hard to believe how low they will sink.
The Prince’s Foundation for Integrated Health remains totally silent about Rath. HRH’s concern for health seems to dry up if things don’t suit his views.
The British Association of Nutritional Therapists shows it’s total irresponsibility after a letter was sent to them to ask about their reaction. Their answer , on jdc325’s weblog was “The association has no opinion to offer on Dr Raths vitamin trials.”.