Download Lectures on Biostatistics (1971). Corrected and searchable version of Google books edition
Download review of Lectures on Biostatistics (THES, 1973).
BBC
It must be admitted that the human genome has yet to live up to its potential. The hype that greeted the first complete genome sequence has, ten years on, proved to be a bit exaggerated. It’s going to take longer to make sense of it than was thought at first. That’s pretty normal in science. Commerce, though, can’t wait. Big business has taken over and is trying to sell you all sorts of sequencing, with vastly exaggerated claims about what you can infer from the results.
There are two main areas that are being exploited commercially, health and ancestry. Let’s look at an example of each of them.
Private health screening is wildly oversold
There has been a long-running controversy about the value of screening for things like breast cancer. For a superb account, read Dr Margaret McCartney’s book, "The Patient Paradox: Why sexed-up medicine is bad for your health", “Our obsession with screening swallows up the time of NHS staff and the money of healthy people who pay thousands to private companies for tests they don’t need. Meanwhile, the truly sick are left to wrestle with disjointed services and confusing options”.
Many companies now offer DNA sequencing. It’s become so bad that a website, http://privatehealthscreen.org/ has been set up to monitor the dubious claims made by these companies. It’s run by Dr Peter Deveson, Dr Margaret McCartney, Dr Jon Tomlinson and others. They are on twitter as @PeteDeveson, @MgtMcCartney, @mellojonny. They explain clearly what’s wrong. They show examples of advertisements and explain the problems.
Genetic screening for ancestry is wildly-oversold
Many companies are now offering to tell you about your ancestors on the basis of your DNA. I’ll deal with only one example here because it is a case where legal threats were used to try to suppress legitimate criticism. The problem arose initially in an interview on BBC Radio 4’s morning news programme. Play the interview.
Today. I should make it clear that I’m a huge fan of Today. I listen every morning. I’m especially enthusiastic when the presenters include John Humphrys and James Naughtie. The quality of the interviews with politicians is generally superb. Humphrys’ interview with his own boss, George Entwhistle, was widely credited with sealing Entwhistle’s resignation. But I have often thought that is not always as good on science as it is on politics. It has suffered from the "false balance" problem, and from the fact that you don’t get to debate directly with your opponent. Everything goes through the presenter who, only too often, doesn’t ask the right questions.
These problems featured in Steve Jones’ report on the quality of BBC Science reporting, which was commissioned by the BBC Trust, and reported in August 2011. The programmes produced by science departments are generally superb. Just think of the Natural History Unit and David Attenborough’s programmes. But the news departments are more variable. One of Jones’ recommendations was that there should be an overall science editor. In January 2012, David Shukman was appointed to this job. But it seems that neither Shukman, nor Today‘s own science editor, Tom Feilden, was consulted about the offending interview.
My knowledge of genetics is not good enough to provide a critical commentary. But UCL has world-class experts in the area. The account that follows is based mainly on a draft written by two colleagues, Mark Thomas and David Balding.
Vincent Plagnol has posted on Genomes Unzipped a blog that explains in more detail the abuse of science, and the resort to legal threats by a public figure to try to cover up his errors and exaggerations.
|
The interview was with the Rector of St Andrews University, Alastair Moffat, who also runs several businesses, including Britains DNA, Scotlands DNA, Irelands DNA, and Yorkshires DNA.
|
 “Inside all of us lies a hidden history, the story of an immense journey told by our DNA.” |
All four sites are essentially identical (including lack of apostrophes. These companies will, for a fee, type some genetic markers in either the maternally-inherited mtDNA (mitochondrial DNA), or the male-only Y chromosome, and provide the customer with a report on their ancestry.
We’ll come back to the accuracy of these reports below, but as a preliminary guide, consider some of the claims made by Moffat on the BBC Radio 4 Today programme on July 9, 2012:
- there is scientific evidence for the existence of Adam and Eve
- nine people in the UK have the same DNA as the Queen of Sheba
- there is a man in Caithness who is “Eve’s grandson” because he differs by only two mutations from her DNA
- 33% of all men are extremely closely associated with the founding lineages of Britain
- we found people that have got Berber and Tuareg ancestry from the Saharan nomads
There was a persistent theme in the interview that DNA testing by Moffat’s company was “bringing the Bible to life” — of course their activities neither support nor detract from anything in the Bible. If you want to either laugh or cry at the shocking range of errors and exaggerations in the interview, listen to it yourself and then read the Genomes Unzipped blog. For example, Piagnol points out that::
"A bit of clarification on chromosome Y and mtDNA: these data represent only a small portion of the human genome and only provide information about the male (fathers of fathers of fathers…) and female (mother of mothers of mothers…) lineages. As an illustration, going back 12 generations (so 300 years approximately) we each have around 4,000 ancestors. mtDNA and chromosome Y DNA only provide information about 2 of them. So these markers provide a very limited window into our ancient ancestry."
Of most concern is that Moffat’s for-profit business was presented as a scientific study in which listeners were twice invited to participate. He admitted that they have to pay but “we subsidise it massively”. This phrase is important because it suggests that this is a genetic history project of such importance and public interest that it has been subsidised by the government or a charitable body. This doesn’t seem plausible – Britains DNA charges £170 for either mtDNA or Y typing, which is comparable with their competitors. There is nothing on the Britain’s DNA website to indicate that it is subsidised by anybody.
Instead of producing evidence, Moffat paid a lawyer to suppress criticism
When challenged by my two academic colleagues, Mark Thomas and David Balding, on this and other problems arising from the interview, Moffat failed to either clarify or withdraw the "massively subsidised" claim. Instead, his two challengers received letters from solicitors threatening legal action for defamation unless they fulfil conditions such as that they will not state that Mr Moffat’s science is wrong or untrue. In the face of so many obvious dubious claims in the Today interview, it would be a dereliction of the duty of an academic not to point out what is wrong.
The interview certainly sounded exaggerated to me. Luckily, a colleague of Thomas and Balding, Vincent Plagnol (lecturer in statistical genetics), has written a detailed refutation of many of Moffat’s claims on his post Exaggerations and errors in the promotion of genetic ancestry testing, on the Genomes Unzipped blog.
Thomas and Balding maintain that oversimpliifed and incorrect statements appear also on the web site of Britains DNA.
The interview sounded more like advertising than science
It is clearly unacceptable for a person in high public office to make a claim on national radio that appears to be untrue and intended to support his business interests, yet to refuse to withdraw or clarify it when challenged.
Complaints were made both by Thomas and Balding to the BBC, but they met with the usual defensive reply. The BBC seemed to be more interested in entertainment value than in science, in this case.
A few more aspects of this story are interesting. One of them is the role of Moffat’s business partners, Dr Jim Wilson of Edinburgh University and Dr Gianpiero Cavalleri of the Royal College of Surgeons in Ireland. It was these two academics to whom both Thomas and Balding sent emails about their concerns. And it was these emails that elicited the legal threat. They refused to clarify the “massively subsidised” claim, and they have not publicly disassociated themselves from the many misleading statements made by Moffat, from which as business partners they stand to benefit financially (they are both listed as directors and shareholders of The Moffat Partnership Ltd at Companies House). They have both responded to Thomas and Balding, misrepresenting their statements in a way that would support legal action.
Did James Naughtie think his interview was entertainment, not science?
Another interesting but depressing aspect to the story is the role of the BBC’s interviewer Jim Naughtie. In the face of the most outrageous claims about Eve, the Queen of Sheba and "bringing the Bible to life”, that a moment’s thought would have suggested can’t be true, Naughtie asked no sceptical, challenging or probing question. He even commented twice on individuals having "pure" DNA, which is appalling: nobody’s DNA is any purer than another’s; has he not heard of eugenics? Naughtie gave Moffat two opportunities to promote his business, even with details of the web address.
It turns out that Naughtie and Moffat are old friends: as Chancellor of the University of Stirling, Naughtie invited Moffat to sit on its Court, and he posted on YouTube a short video of him supporting Moffat’s campaign for Rector of St Andrews. No such connection was mentioned during the interview.
This interview is not the first. Naughtie also interviewed Moffat (broadcast on 1st June 2011) about a rather silly exercise in claiming – on the basis of Y-chromosome data – that Jim Naughtie is an Englishman . Again Naughtie gave Moffat ample opportunity to promote his business and as far as I am aware none of Moffat’s commercial rivals has been given any comparable opportunity for free business promotion on the BBC.
No suggestion is being made that there is anything corrupt about this. Naughtie is not a scientist, and couldn’t be expected to challenge scientific claims. The most likely interpretation of events is that Naughtie thought the subject was interesting (it is) and invited a friend to talk about it. But it was a mistake to do this without involving Today‘s science editor, Tom Feilden, or the new BBC science editor, David Shukman, and especially to fail to invite a real expert in the area to challenge the claims. There are lots of such experts close to the BBC in London.
Many other companies cash in on the ancestry industry.
Britains DNA is only one of many genetic ancestry companies that make scientifically unsupported claims about what can be inferred from Y chromosome or from mtDNA variants about an individual’s ancestry. This is a big industry, and arouses much public interest, yet in truth what can reliably be inferred from such tests is limited.
Our number of ancestors roughly doubles every generation in the past, so the maternal-only and paternal-only lineages rapidly become negligible among the large numbers of ancestors that each of us had even just 10 or 20 generations ago. Because migration is ubiquitous in human history, those ancestors are likely to have had diverse origins and the origins of just two of them (paternal-only and maternal-only lineages) may not give a good guide to your overall ancestry. Moreover if you have a DNA type that is today common in a particular part of the world, it doesn’t follow that your ancestors came from that location: such genetic tests can say little about where your ancestors were at different times in the past.
Faced with these limitations, but desiring to sell genetic tests, many companies succumb to the temptation to exaggerate and mislead potential clients about the implications of their tests, in some cases leading to disappointed clients who feel betrayed by the scientists in whom they placed trust.
If you have a few hundred pounds to spare, by all means get yourself sequenced for fun. But don’t imagine that the results will tell you much.
Follow-up
25 February 2013, Mark Thomas follwed up this post in the Guardian
“To claim someone has ‘Viking ancestors’ is no better than astrology. Exaggerated claims from genetic ancestry testing companies undermine serious research into human genetic history”
10 March 2014
After more than a year of struggling, the BBC did eventually uphold a complaint about this affair. Full accuonts can be found on the web site of UCL’s Molecular and Cultural Evolution Lab and on Debbie Kennett’s site
|
Steve Jones, UCL’s star geneticist, has been commissioned by the BBC Trust to write a report on the impartiality of science journalism on the BBC. It covers both TV and radio, and all channels. Current programmes can be found by the BBC Science home page. |
|
It is not uncommon for bloggers to be critical of science reporting in the mainstream media. Now is our chance to do something constructive about it. If you have opinions about this, please leave them in the comments here, and/or email them to
Here are some of my own opinions, to get things going. Many programmes I haven’t seen/heard, so my selection may not be representative, but it is wide enough to include examples that are superb and examples of some that I think are not good enough.
Problem areas
There are two particular topics that are real problems for broadcasters. One is the whole area of alternative medicine and the ‘supplement’ industry. The other is anything to do with climate change. Both have formidable lobby groups which, to the inexperienced journalist, may sound like quite plausible scientists (some even have academic titles). Creationists can also be a problem. though not many programmes take them very seriously. Both quacks and climate deniers rarely have anything to say that is real science. They have different motivations. Examples are given below.
Many programmes are superb
David Attenbrough is an obvious example. His programmes can’t be bettered. The photography is breathtakingly beautiful and the science is always accurate. For me, they alone are worth the licence fee, and I don’t want the licence fee to be reduced. It helps that Attenborough knows the science so well. It also helps that most of the time the science isn’t very difficult and isn’t very controversial either.
There have been many other superb programmes. Steve Jones own 6-part TV series "In the blood" was a beautiful example. The fact that his comments are sought frequently by the BBC is greatly to their credit. Much depends on producers being sufficiently well-informed to know whom to ask.
More recently Brian Cox’s "Wonders" series has provided an excellent example of how science programmes can be made popular without being inaccurate,
Also excellent were Jim Al-Khalili’s Chemistry: A Volatile History and Michael Moseley’s Medical Mavericks.
Simon Singh has made consistently good programmes. His wonderful documentary on Fermat’s Last Theorem was a masterpiece.. He is a master at making programmes that make really difficult subjects accessible to the public, without making them misleading.
Tim Harford’s programm, More or Less has made a great contribution to public understanding of statistics.
Ben Goldacre‘s two part Radio 4 Programme, The Rise of the Lifestyle Nutritionists was a superb explanation of a contentious area.
It would be useful if all presenters of programmes with any scientific content had to listen to Harford or Goldacre. It might make them a bit more critical about the problem of causality that beset the observational epidemiology items that predominate among the items picked out from press releases by so many news programmes.
These programmes were so successful because they were made by people who know what they are talking about. They confirm the view that the best science programmes are made by scientists, not by journalists. There are exceptions of course. It could be argued that some of Robert Winston’s programmes have strayed too far from his area of expertise to reach the same high standards. And some journalists have produced excellent programmes. Two examples follow.
Geoff Watts has kept up a consistently high standard on Radio 4. from Science Now, through Medicine Now to Leading Edge, the standard has always been high. It is good straight science in its social context. He avoids controversies, for example his excellent programme about Charles Darwin does not include a creationist to provide (phony) ‘balance’.
After the demise of Medicine Now in 1998, Watts wrote in the BMJ thus.
In the early days of the programme you could have listened for several weeks in a row without hearing from patients. I used to defend this on the grounds that Medicine Now was there to talk about disease and its treatment, not the experience of disease and its treatment. I was wrong. To make that distinction is simply to parallel the fault for which doctors themselves take a deal of stick—being interested in the illness to the exclusion of the person who’s suffering from it. I was persuaded, reluctantly, to accept more lay voices, and I am now embarrassed that I didn’t sooner see the need for them.
The swing of the pendulum may push the whole patient experience thing too far: to a point at which it’s professional knowledge and objective analysis that is elbowed into the wings. One of the vogue concepts among BBC managers in recent years has been “accessibility”. But this is a weasel word, too easily used as justification for editing out anything that might require the audience to concentrate and think. If a patient has a rash, it’s a lot simpler for the reporter to inquire about the urge to scratch than to explore the events in the immune system that caused the skin to redden, swell, and itch in the first place. How sad if people lose an opportunity to hear from the researchers, speaking their own words, who are actually trying to find out.
This summarises a lot of the problems of science programmes. They too easily become trivial vox pops, and Watts resisted this tendency very successfully.
A great problem for programmes about medicine arises from the pressure exerted by the alternative medicine industry (ot which more later). Watts would not tolerate nonsense. He says
Medicine Now was stabled in the BBC’s Science Unit, and it was the broad acceptance of science and its methodology which shaped editorial choices. When complementary medicine was on the agenda, we expected evidence from our contributors not testaments of faith.
Material World is another good Radio 4 programme. Quentin Cooper does, on the whole, a good job. But sometimes even he falls foul of the phony balance argument After my piece in Nature on the shameful degrees in pseudo-scientific medicine got discussed on Material World, (audio here) but my opponent was not a scientist at all, but the head of “Complementary Therapies” at the University of Westminster, a man who presides over courses that teach “amethysts emit high Yin energy“. It is simply impossible to have a proper scientific discussion with people who believe nonsense like that. They don’t accept the ground rules at all. It is a good example of phony balance (see below).
Some programmes are quite bad
Alternative Medicine: the evidence. This series if three TV programmes was shown in February 2006 on BBC 2, in conjunction with the Open University. It illustrates well three problems with science programming. (1) Despite the title, tt was surprisingly weak at showing evidence, (2) It showed the defensive and unhelpful response that, only too often, the BBC shows when complaints are made. And (3) it showed that association with a university is not, per se, enough to guarantee quality.
Because of the title, I’d looked forward to this programme, and made minute by minute notes, which are recounted in BBC2 and the Open University on Alternative Medicine. It turned out that the evidence was thin on the ground, and what there was was not always accurate I complained to the BBC, but got nowhere [download my complaint and some subsequent correspondence]. I was fobbed off with defensive PR. (Much the same happened when I complained to the Open University.)
Worse still, a letter in defence of the programme that appeared in the Guardian, turned out to have been written by the BBC and was not even seen by some of its "signatories" -see .Alternative Medicine series: dirty tricks at the BBC? All this took a lot of work and got nowhere.
Simon Singh, the eminent science author, wrote two articles that exposed the very misleading portrayal of anaesthesia with acupuncture In the Guardian he wrote A groundbreaking experiment … or a sensationalised TV stunt?, and in the Daily Telegraph he wrote Did we really witness the ‘amazing power’ of acupuncture?. Singh also sent complaints to the BBC, but he persisted after the complaints were fobbed off and eventually his complaints reached the BBC Trustees. Two of his three serious complaints were upheld.
Phone-in programmes are notoriously bad for both balance and phony balance. In the alternative medicine field, equal time is always given to scientists, astrologers and crystal healers. The presenters are usually ill-informed and the callers are usually even less well informed. A particularly bad example follows.
Call You and Yours. The Radio 4 programme, You and Yours, at its best, can be quite good. It did a good job on a "snoring remedy" that I’d investigated, though it omitted some things that should, in my view, have been included. But they also have phone in versions of the programme. On 29th February 2010, they ran a phone-in programme about herbal medicine, hosted by Julian Worricker, someone who clearly was totally unaware of the controversies that surrounded this subject and, particularly, its regulation. The worst thing about this programme was that it featured a resident ‘expert’ That was Michael McIntyre who is chair of the European Herbal & Traditional Medicine Practitioners Association. McIntyre is a well known advocate of alternative medicine, who constantly fudges the need for proper evidence.
I went through the programme carefully, making detailed notes, which appear at Some truly appalling reporting of science by the BBC. It was one of the most biassed programmes on the topic I have ever heard. i sent a complaint to the BBC, referring to the detailed analysis which had already appeared here. To my dismay, they wouldn’t accept a complaint in the form. They wanted me to type the whole thing in a little box on the complaint site. where there is no formatting and no live links. I protested in vain that if they wanted a printed version, all the had to do was print the web page. At this point I decided that there was no point in spending yet more time to cope with the inflexibility of the complaints procedure.
Today programme. I’m an avid listener toToday, the best news programme on radio John Humphrys has no greater fan than I. For politics it is superb. But for science it is, sad to say, not always so good. One reason is that the presenters don’t know enough about the topics to ask the same sort of tough questions that they fire at politicians. Another reason is that they suffer badly from the phony balance problem (see below). A third reason is that they tend to pick up on silly survey press releases (the sort of ‘men with long big toes are better in bed’ pseudo-science); They may quite rightly laugh at them but this sort of thing doesn’t count as science reporting.. The Today programme is admirably serious about politics, but the science is often dumbed down and uncritical.
What needs to be done to improve BBC science
Link to the sources. Despite pressure from bloggers, the BBC web site still does not usually link to original sources, the paper on which claims are based. The whole virtue of the web is that it makes this very easy to do.
Anonymity of reporters. Too often reports of science on the BBC web site are anonymous. There is no excuse for that. Every report should carry the name and email address of the person who wrote it, Most newspapers do this, but the BBC is lagging behind.
Reaction to criticism. In most cases that I’ve tried, the reaction to constructive criticism has been obstructive and defensive. Producers seem very reluctant to admit that any mistakes were made. That needs to be changed.
Science correspondents are too often uncritical. A few more with the approach of investigative journalists would improve standards. An example is provided by a recent report “It’s good to think – but not too much, scientists say“. This is typical of the sort of work that many people find a bit hard to take seriously, but the report reads a bit like a regurgitation of press releases. There is no link to original sources and no attempt at evaluation.
Press releases. One reason for misleading reports stems from misleading press releases. Press releases often come from media departments who regard their job as getting their university into the headlinse, rather than explaining science. Worse still, sometimes the misleading hyps stems from the authors themselves (one example here, but there are hundreds to choose from). This makes it very important that science reporters should read the paper and have good enough critical faculties to read through the hype.
Complaints procedure needs to be improved. Complaints should be accepted in any form, The present web form is suitable only for short and simple criticicisms. An email address should be provided and it should accept attached documents. Certainly complaints in the form of web pages should be welcomed, because the live links provide the simplest way to refer to source documents.
The problem of phony balance. This is biggest problem of the lot.
In the wake of the report by the Science and Technology Committee (STC) on the lack of evidence for homeopathy, and the Chinese medicine poisoning, the BBC carried at least three very bad reports. Being a strong supporter of the BBC that saddens me. These cases are summarised at Some truly appalling reporting of science by the BBC. The worst was the case of Call You and Yours. There was also a totally imbalanced and ill-informed report on statutory regulation, and a very irresponsible video of a woman who claimed homeopathy cured her cancer. .
The question of balance is important. Ofcom imposes an obligation that reporting should reflect the balance of viewpoints. Section 5 of Ofcom’s broadcasting code says (emphasis is mine).
“Section 5: Due Impartiality and Due Accuracy and Undue Prominence of Views and Opinions”
“To ensure that news, in whatever form, is reported with due accuracy and presented with due impartiality.”
“Meaning of “due impartiality”: “Due” is an important qualification to the concept of impartiality. Impartiality itself means not favouring one side over another. “Due” means adequate or appropriate to the subject and nature of the programme. So “due impartiality” does not mean an equal division of time has to be given to every view, or that every argument and every facet of every argument has to be represented.”
The BBC Trust has a very similar definition of “due impartiality”.
Thus the rules stare quite explicitly that "impartial" does not mean giving equal time to any view, however batty,
In practice, though, producers often seem to play it too safe, and choose to give the same time time to the view that the earth is flat as is does to the view that the earth is spherical (OK, an oblate ellipsoid). This often gives a quite misleading impression of the state of play of informed opinion. Inappropriate use of “equal time” is the most common cause for misleading reports
on science.
Minority views should be heard of course, but they should not be given equal prominence to views that are held by the vast majority of informed people. Inevitably the worst cases arise in the areas of quack medicine, climate change and evolution.
Somebody said recently, it is as though after an air crash one gave equal time to the air accident investigator and a representative gravity-deniers association. That is scarcely an exaggeration of what happens on the BBC too often.
Worse still, far more time was given (especially on ‘Call You and Yours’) to the viewpoint that any scientist, indeed any informed person, would regard as quackery.
One thing that could be done about this false balance is to have better informed producers, or, more likely, to have better informed science reporters who can give advice on the state of opinion (and to make sure that their advice is sought).
Unless the BBC starts to be more critical in some of its reports, it could lose its preeminence. In the last few years it has become increasingly the case that the best critical evaluations of science are to be found not in the BBC or other mainstream media, but on blogs written by working scientists. Perhaps the BBC should ask them more often than it does at present.
Now give your opinions, below or email them to trust.science@bbc.co.uk
Follow-up
The Open University is a wonderful institution
I should have made it clear the Open University has played a big role in producing some of the best programmes. I was, quite rightly, corrected by a letter from an OU scientist. I’ll quote from it.
David Attenborough’s series on Life In Cold Blood, Charles Darwin and the Tree Of Life, Life – these are all Open University commissions. Life In Cold Blood also won a BAFTA and we have a string of other awards. I believe this is some of the very best science broadcasting the Open University puts out and I’m not surprised to see it at the top of your list.
More Or Less is an Open University commission – see e.g. http://www.open2.net/moreorless/
Material World is another one of our occasional commissions -see e.g. http://www.open2.net/materialworld/index.html
Geoff Watts contributed to the BBC Darwin season last year, which was heavily supported by Open University programmes.
See for example http://www.bbc.co.uk/darwin/We support the series with a considerable amount of on-line material both for credit and not for credit. This material is accessible to the public at these open2.net sites, and at http://www.open.ac.uk/openlearn/.
It is clear that the criticisms levelled at the Alternative Medicine series are very much the exception to the usual excellent work of the OU. I’m told that that programme had nothing to do with the science faculty. Clearly it was an unusual aberation. I presume it was connected with the OU’s course K221, which I wrote about in 2006, under the title Open University Quacks. That sort of thing is quite atypical of the Open University, and something of an embarrassment to the many top rate people who work there. As usual, the blame lies not with scientists but with senior managers. After hearing about course K221, I had a long correspondence with Professor David Vincent, a pro-vice chancellor. He made sympathetic noises, but did absolutely nothing. That’s par for the course with senior administrators.
The Open University has been a magnificent success from the outset. Its first vice-chancellor was Walter Laing Macdonald Perry . Before he took that job, he was professor of Pharmacology in Edinburgh (and one of my Ph.D. supervisors). He did a great job.
Failure to report negative results
A classic example of a sin of omission by the BBC (and the rest of the mainstream media too) occurred recently in the reporting of the alleged effect of B vitamins on the development of Alzheimer’s disease. A positive trial was widely reported, but two weeks later a trial appeared that measured the eight thing -cognitive deficiency – and that showed no effect at all. As far as I can tell it was barely reported at all, The details are at https://www.dcscience.net/?p=3516
This is a follow-up of the poat on BBC2 and the Open University on Alternative Medicine.
Following the article by Simon Singh in the Guardian (25 March 2006), two letters appeared on April 1, 2006. The first, from Prof. Edzard Ernst, confirmed that he felt the BBC had ignored and misrepresented his advice.
|
In its response to our criticism of the Alternative Medicine series, the BBC says “it is extremely unusual that Professor Ernst should make these comments so long after the series was aired” (Report, March 25). I made my criticism in writing two months before the programme was broadcast. The reason why I reiterated them when I did was simply because Simon Singh interviewed me in my capacity as adviser to the BBC. Extremely unusual? Long after? I don’t think so.
Prof Edzard Ernst Peninsula Medical School, Exeter |
The second letter defended the BBC. It was unequivocal in its support of the entire series of programmes, and its appearence surprised me. In the light of all that has been written, one might have hoped that the BBC would listen and learn from its mistakes. The letter has ten signatories.
|
We are all scientists involved as consultants or contributors to the BBC2 series, Alternative Medicine. We do not in any way recognise the experience of working on the series as described in your article (Was this proof of acupuncture’s power … or a sensationalised TV stunt?, Science, March 25), nor do we share the views of those scientists you have quoted in it. In all its dealings with us, the BBC asked for advice and input where needed, took on board our feedback and incorporated our comments into the final edit of the programme as transmitted, where appropriate. Far from feeling dissatisfied with the final outcome, we feel the series seemed well balanced and informative, doing full justice to the subject matter it addressed. Dr Jack Tinker Prof Brian Berman Prof Liz Williamson Dr Andrew Vickers Dr James Warner Dr Mike Cummings Prof Gary Green Dr Carl Albrecht Dr Jen Cleland Professor Irving Kirsch |
But all is not what it seems. Contrary to appearances, this letter was actually written by the BBC who also compiled the signatories (it seems to have been the responsibility of Kim Creed, of BBC Factual Publicity).
- One of the signatories. Dr James Warner, had never seen the letter until after it was published, and tells me that “[I] substantially do not agree with the sentiments expressed therein. Indeed, we had to resist attempts by the programme makers to sensationalise our work”.
The Guardian has published a correction. - Six other signatories tell me that their approval was limited to the way their own contribution was treated, and was not intended as approval of the whole series. One commented ” I’ve obviously been naïve, and I am very fed up with this whole thing”. Another says ” I suppose I (foolishly by the sounds of things) extrapolated from my own programme and experience, without considering the wider implications of the concluding sentence”.
- Only one of the eight signatories whom I’ve asked has actually seen all three programmes, as they were transmitted. This makes it rather odd that they should appear to endorse so unequivocally the whole series.
- One of the signatories, Carl Albrecht, gives his address as “University of Johannesburg”, but oddly the BBC forgot to mention that Dr Albrecht is co-owner (at least until very recently) of the South African Company, Phyto Nova, that makes, promotes and sells the untested herb, Sutherlandia, for treatment of AIDS (see, for example, here).
He is, therefore, highly biassed. He is also exceedingly controversial. One of his strongest critics has been Stuart Thomson, Director of the Gaia Research Institute,
hardly an organisation that is biassed against “natural medicines”. Albrecht is indeed a very curious choice of advisor for a programme about science. - Three of the signatories (Berman, Cummings and Albrecht) are heavily committed to CAM, and so unlikely to be critical of anything that favours it, even apart from financial interests in the outcome. Brian Berman even has is own an entry in Quackwatch. So several of the signatories are pretty much committed in advance. Asking them if they endorse the programmes is about as informative as asking a group of priests if the endorse god.
- It gets worse. This morning, 6th April, I heard from Andrew Vickers, of the Memorial Sloan-Kettering Cancer Center, New York. This is what he says.
“I didn’t sign this letter ”
“I was shown the text of the letter but didn’t fully agree with it and told them so. I said something along the lines that the series didn’t do ‘full justice to the subject matter’ (how could it possibly?) but that what they did was fair and reasonable within the constraints set by the medium. You are also right to point out that my comments only go so far as the acupuncture episodes (which I saw) rather than the other two shows (which I did not). No doubt had I been shown a final version for signature I would have also pointed this out.”
The BBC brought us superb programmes like Life on Earth and Planet Earth. They bring us superb news (I’m listening to the incomparable John Humphrys on the Today Programme right now). They have suffered unjustly at the hands of spin-meisters like Alastair Campbell and the execrable Hutton Report (If the Hutton Report had been an undergraduate essay, it would have scored alpha-plus for collection of evidence and gamma-minus for ability to connect evidence to conclusions).
How ironic it is, then, to see the BBC behaving in this case like spin artists. Deny everything, and, if necessary, falsify the evidence.
Jump to Open University course K221
BBC2 TV showed a much-advertised series on alternative medicine, in February 2006. The programmes seem to be linked with a dubious Open University course.
| The programmes are presented by Kathy Sykes, who is professor of the public understanding of science at Bristol University. She has done some excellent work in that area, for example, in the Rough Science TV series. |  |
The first programme: acupuncture
The first programme, on acupuncture, was shown on 24th January, 2006. The programme did not start
in a very promising way. Just lots of testimonials from happy patients, the staple diet of all snake oil salesmen.
They are watchable, of course, but don’t do anything at all to promote public understanding of what constitutes evidence.
There is, of course, little doubt that sticking needles into your body can produce physiological responses. Two things remain uncertain.
- Just how useful are these responses in helping particular conditions?
- Is there anything at all in the mumbo jumbo of meridians and chi?.
With a big flourish we were shown “a 21-year-old Shanghai factory worker undergoing open-heart surgery with only the needles to control her pain”. It turns out that this was a sham. The patient was doped on opiates and local anaesthetics. The needles were merely cosmetic. Why were we not told?
The apparently contradictory trials suggested that, at least the alleged principles of acupuncture are nonsense. The programme concentrated on a trial by Berman (Ann Intern Med. 2004,
141, 901-10 ) which used ‘sham acupuncture’, with ‘stage dagger needles’, on osteoarthritis of the knees. In this sort of trial there is no actual penetration, and the sham needles are placed on the places dictated by the mumbo jumbo. This procedure was justly criticised by a subsequent letters in the same journal (Ann
Intern Med 2005, 142, 871).
Another large study was ignored by the TV programme altogether. This was by Linde et al. (Journal of the American Medical Association. 2005 293(17):2118-25). This study concluded
“Acupuncture was no more effective than sham acupuncture in reducing migraine headaches although both interventions were more effective than a waiting list control.”
As pointed out above, this study is, in many ways, much more interesting than Berman’s, because the control group did not have ‘sham acupuncture’. Needles were really inserted, but they were inserted in points that have nothing to do with the mumbo jumbo of meridians. The fact that the controls were much the same as the treated group suggests that, whatever effect the needles produce, it doesn’t matter much where they are inserted. The only obvious interpretation of this is that the ‘principles’ on which acupuncture is based are so much nonsense (and, therefore, it is not a subject that can possibly be taught in a university).
This crucial point was ignored by the TV programme. A big fuss was made of a functional magnetic resonance experiment, staged for TV, that showed that the effects on brain ‘activation’ are different for superficial needling and for real needling. There is nothing in the least surprising in the observation that have a needle pushed into you affects the brain, but it really does not help at all in answering the important questions. Incidentally that experiment had already been done anyway.
In summary, the first programme, failed to give a fair assessment of current knowledge about acupuncture, and failed to consider the important questions of what sort of controls are appropriate, and whether talk of meridians means anything whatsoever. Sadly, I can’t agree with the boast that “It’s the deepest investigation into the efficacy of alternative medicine ever attempted on TV”. Let’s hope the second programme is a bit more critical.
The second programme: healing
The second programme (31st January, 2006). I liked this programme much better than the first, even if it left the crucial questions unresolved.
| The programme started with a healing meeting by the notorious Benny Hinn. The meeting had all the mass hysteria of a Nuremburg rally, though no mention was made of the fact that this (very rich) man’s financial malpractice had been revealed by a CBC TV programme. |  |
 |
On the right is his receipt for £3347 for two nights at the Lanesborough hotel in London (that did not include $1700 he gave in tips).
The lovely Ghanaian lady who cleans my office and lab every morning gives gives money to this mega-rich man because “he needs it to preach the gospel”.
The National Institutes of Health provided $1.8 m of US taxpayers’ money for this project which seems not to do real research at all. After seeing a demonstration of the “Gas Discharge Visualization”, GDV, or Kirlian camera, given by a very gullible Dr Melinda Connor, Sykes comments that this ‘research’ “is not so much trying to find the evidence for ‘healing energy’, but is rather working on the basis that there is one”
Kathy Sykes did, though show a pretty healthy degree of scepticism about the people who pretend to photograph “auras” and other imaginary “force fields”. She visited the “Center for Frontier Medicine in Biofield Science” at the University of Arizona.
In other words, the ‘research’ is a con. Once again (see above) we see money given by well-intentioned governments diverted form the purpose for which it was given. For more first class boloney on ‘imaging’, see for example, Biofield Sciences in Exeter (UK) and ‘electro-crystal therapy’. The list is endless.
Kathy Sykes went on to show several interesting experiments on placebo effects. For example sham healers (played by actors) do at least as well as ‘real’ healers. And sham knee surgery may be as effective as real surgery, though the programme failed to mention the obvious possibility that this could mean nothing more than that real knee surgery is itself pretty ineffective. As so often in this series, the producers failed to talk to the right people.
She concludes “healing does not work beyond placebo”.
So I’m right with her, though it would have been better if there had been a more critical mention of the fact that not all placebo effects are real. Many probably depend on the natural fluctuations in the intensity of the patient’s condition. Anything can ‘cure a cold’, because you recover from a cold in a few days anyway,
Sykes concludes, speaking of the placebo effect, “I want to see that power properly harnessed -we’d be mad not to”. But that, disappointingly, was the end of the programme. That point is where the problems begin. How do you harness the placebo effect? How do you justify lying to the patient in order to maximize the effect? How do you train the ‘healers’? Are they themselves to believe the same lies, or are they to be trained in the art of deception? As pointed out in a recent review of the neurobiology of placebos (Colloca and Benedetti, 2005)
“For example, the assertion that placebos, fake therapies, fresh water and sugar pills could positively affect the brain biochemistry in the appropriate psychosocial context might lead to a dangerous justification for deception, lying and quackery”.
These are the central dilemmas of sCAM, as listed at the top of this page. The programme did nothing to solve them, or even to draw attention to them.
The link with Open University Course K221
The blurb on this programme on the Open University/BBC site concludes
“So, could the power of the mind explain the benefits people experience from healers? And have healers tapped into this power somehow? The conclusion throws new light on all healing processes, and has a surprising and inspirational message for every practitioner and patient.”
But what is to be done about this “inspiration”? Nothing is said about that. The TV programme was immediately followed by voice-over that advertised an Open University pamphlet, which is publicity for their course K221. That course, judging from what is posted on the web, is run by true believers who are a lot less sceptical than Sykes. She says that she did the voice-over but has not yet been shown the contents of the course.
The third programme: herbalism
Oooh dear. The third programme was, in my view, by far the worst. Hardly a single critical voice was heard. Despite the odd word of reservation, the programme left the impression of being an advertisement for the herbal medicine industry. Did the BBC not think of asking a pharmacologist? In my view, this programme was a disservice to human knowledge. Let’s look at some of the details.
The programme once again starts with dramatic testimonials from satisfied customers. No hint is given to the viewer of the total unreliability of such testimonials. References, in awed voice, are mad to “a vast body of ancient knowledge that herbalists draw on”. No mention of the superb track record that ‘ancient knowledge’ has for turning out to being dead wrong. It was 11 minutes into the programme before the question of evidence was even mentioned and then we had a herbalist wandering through a field. At 13 minutes, the herbalist, Simon Mills, was interviewed -he rattled on about dampness. marshy conditions. “There
are herbs for heating and drying”. Sheer gobbledygook. And still no discussion of evidence.
Sutherlandia At 18 minutes “To get another view I’m going to a country where herbs are claimed to have dramatic effects”. Off to Africa to spend a good 10 minutes on Sutherlandia, a totally unverified treatment for AIDS. Why spend all this time (and licence-payers money) to end up with the conclusion that clinical trials have
not been done yet, and we have no real idea whether it works or not? A search of Pubmed for Sutherlandia and AIDS produces a mere five papers. Mills et al. Nutrition Journal 2005, 4:19 write as follows.
“Despite the popularity of their use and the support of Ministries of Health and NGOs in some African countries, no clinical trials of efficacy exist, and low-level evidence of harm identifies the potential for drug interactions with antiretroviral drugs.”
(and one of the authors on that paper is from the Canadian College of Naturopathic Medicine: hardly likely to have a bias against herbs). The comments made in the programme about AIDS were irresponsible and potentially dangerous: they could kill people..
It took until almost half way through the programme, before we got round to the question of whether any of these claims are true. Very impressive to learn that the Nazis pushed herbal medicine, but totally uninformative (or does it mean that herbalism appeals to nutters?). We are shown the German herbal bible, but again it is pointed out that it contains no evidence about their efficacy. So no further forward yet. Then we are introduced to chromatography: very pretty, but still no evidence about whether herbs help people.
At 9.34 pm we are last get round to some evidence. Or do we? Not yet, just another personal testimonial about the the wonders of St John’s Wort. St John’s Wort (Hypericum) is an interesting case, because there is at least some evidence that it works, though certainly not enough for it to be described as a “superherb”, as Sykes did. Of course depression (like knee surgery -above) makes a pretty good case for herbalists, because conventional antidepressants are so very unsatisfactory themselves. It doesn’t take much to do better than Seroxat (Paxil, paroxetine). At 9.38 pm we get the first actual numbers. And very selective numbers they are
too. The view presented in the programme was desperately over-optimistic about the wondrous effects of St John’s wort. Consider the recent review by Linde et al. (2005 Brit J. Psychiatry, 186, 99-107) (read
it yourself –download pdf file). The conclusion was as follows.
“Current evidence regarding Hypericum extracts inconsistent and confusing. In patients who meet criteria for major depression, several recent placebo-controlled trials suggest that Hypericum has minimal beneficial effects while other trials suggest that Hypericum and standard antidepressants have similar beneficial effects. ”
And another trial, again not mentioned in the programme, was published in Journal of the American Medical Association, 2002, 287, 1807 – 1814) [download the pdf file]. This paper was interesting because it compared placebo, St John’s Wort and sertraline (Zoloft), a drug of the same class as Seroxat). All three were indistinguishable (on the two primary outcome measures). So St John’s Wort was as good as Zoloft, but only because Zoloft was no better than placebo either. The paper concluded thus.
“This study fails to support the efficacy the efficacy of H. perforatum [St John’s Wort] in moderately severe major depression. The result may be due to low assay sensitivity of the trial, but the complete absence of trends suggestive of efficacy for H. perforatum is noteworthy.”
Why were we not told about trials like these?
At 9.43 pm, almost three quarters of the way through the programme, we are eventually told that ginseng, echinacea and evening primrose oil do not work. What took so long?
9.46 pm. Off to South Africa to look at research in Johannesburg on Sutherlandia by Carl Albrecht (more of him below). Some impressive stuff about flavonoids but no results. Flavonoids can’t be absorbed, but, aha, it contains saponins too. Perhaps they allow the flavonoids into cells. Well perhaps.
But this is not information, it is idle speculation.
At 9.51, we get back to brain imaging, this time at Imperial College. Professor Sykes seems to be excessively impressed by brain imaging. We are then treated to more idle speculation about how ginko might help in Alzheimer’s disease. Dr Warner is running a clinical trial to find out whether ginko really helps. But there were no results yet. In that case why not wait until there is a result, before telling us all about it?
We are told that herbs now “have to go through rigorous quality standards”. It was NOT made clear that the standards don’t include anything about the herb actually doing anything useful. The standards may give some protection against your being poisoned. They do nothing at all to guarantee you’ll be helped.
The conclusions
“What’s really impressed me is the way that different ingredients from particular herb can combine together and have really powerful effects on us humans. So I believe that herbs are going to play a key role in medicines of the future”
“What started as an ancient wisdom may just might provide new medicines that will help us all live longer, fuller lives”
These statements are quite outrageous! The first statement has no basis whatsoever. It is sheer idle speculation. It could be true, but there is no reason to believe it is.
The second statement is content-free. Yes, it “may just” do that. On the other hand it may not.
The web site for the third programme. (7th February, 2006, 2100-2200) concludes thus.
“So, what’s their secret? Working with fellow scientists, Kathy discovers that plants contain much more than a single – or even two or three – active ingredients. They are enormously complex
Chemical cocktails that have medicinal properties modern pharmaceuticals simply can’t reproduce.”
Just one snag (apart from the misleading implication the Sykes was doing pharmacological experiments), There is not the slightest reason, thus far, to think there is any advantage in using an “enormously complex chemical cocktail”.
Stop press: on Saw palmetto (one of the “superherbs” of the TV series)
The New England Journal of Medicine, for February 9th 2006 (354, 557 – 566), reports a clinical trial of “Saw Palmetto for Benign Prostatic Hyperplasia”. This is what they say.
“Saw palmetto is used by over 2 million men in the United States for the treatment of benign prostatic hyperplasia and is commonly recommended as an alternative to drugs approved by the Food and Drug Administration.”
“In this double-blind trial, we randomly assigned 225 men over the age of 49 years who had moderate-to-severe symptoms of benign prostatic hyperplasia to one year of treatment with saw palmetto extract (160 mg twice a day) or placebo.”
“Conclusions. In this study, saw palmetto did not improve symptoms or objective measures of benign prostatic hyperplasia.”
I hope that the BBC, the Open University and Prof Sykes now appreciate the folly of judging treatments before the results are in.
Postscript. Some reviews of the TV programmes
-
There has been some lively discussion of the BBC2 series on a forum of the James Randi Educational Foundation, on the BBC2/Open University site, on Ben Goldacres’s Badscience site, and at ebm-first.com.
- The Times TV critic was unenthusiastic.
“So having started out as a sceptic, Sykes ended the programme chirruping, like a born-again Christian, about how herbs contain complex combinations of chemicals that scientists cannot yet reproduce&”;
-
Simon Singh writes in the Daily Telegraph (14 Feb., 2006): "Did we really witness the ‘amazing power’ of acupuncture?
“A BBC series on unorthodox therapies was devoid of scepticism and rigour, says Simon Singh.”
“Although the second programme was indeed a rational look at the placebo effect, the other two episodes were little more than rose-tinted adverts for the alternative medicine industry.”
“For example, the scene showing a patient punctured with needles and undergoing open heart surgery left viewers with the strong impression that acupuncture was providing immense pain relief. In fact, in addition to acupuncture, the patient had a combination of three very powerful sedatives (midazolam, droperidol, fentanyl) and large volumes of local anaesthetic injected into the tissues on the front of the chest.
With such a cocktail of chemicals, the acupuncture needles were apparently cosmetic. In short, this memorable bit of telly was emotionally powerful, but scientifically meaningless in building a case for acupuncture. ”“This TV series pretended to be scientific and had the chance to set the record straight, but instead it chickened out of confronting the widespread failure of alternative medicine. ”
- Advertisers cash in. Sadly, but predictably, the programme on herbalism has
already been exploited by vendors of unproven treatments. While it is true that the programme did not actually assert that this herb cured AIDS, it certainly left the impression that it was good stuff. Here is an example: “As seen on BBC2”“In South Africa, BBC 2 TV presenter, Professor Kathy Sykes learnt of the herb Sutherlandia, which is being touted as a new weapon in the fight against HIV and ”
“It is with thanks to programmes such as Alternative Medicine shown on BBC 2 on Tuesday 7th February, and the work carried out by Professor Kathy Sykes that medicinal herbs can receive the acknowledgement they truly deserve, and this knowledge be passed on to the general public.”
“Bioharmony Sutherlandia is available from Revital Ltd in 60 x 300mg tablets for £19.99rrp. ”
- A groundbreaking experiment … or a sensationalised TV stunt?
Simon Singh, in The Guardian (25 March 2006) followed through with some more details on the BBC2 series. It’s not only pharmacologists who were unhappy about it. So were several of the people who advised the BBC and/or appeared on the programme.
“But this week scientists involved in the series have complained that elements of the programmes were misleading, the production team was uninformed, and scientists were used as “marionettes” ”
At the end of the first programme a “hugely ambitious” imaging experiment was shown with an enormous flourish. The outcome was, roughly speaking, that pushing needles into yourself produces a signal in the brain. Good heavens! Who’d have thought it? Even George Lewith, normally an apologist for CAM, was critical.
“The interpretation of the science in this particular programme was not good and was inappropriately sensationalised by the production team. I think all of us on the experiment felt like that.”
“The experiment was not groundbreaking, its results were sensationalised and there was insufficient time to analyse the data properly and so draw any sound conclusions. It was oversold and over-interpreted. We were encouraged to over-interpret, and proper scientific qualifications that might suggest alternative interpretations of the data appear to have been edited out of the programme.”
Edzard Ernst, professor of complementary medicine at Exeter University, and the main consultant for the series says:
“The BBC decided to do disturbingly simple storylines with disturbingly happy endings. But none of these stories is as simple as they presented, nor do they have such happy endings. Even when the evidence was outright negative, they somehow bent over backwards to create another happy ending.
“I feel that they abused me in a way. It was as if they had instructions from higher up that this had to be a happy story about complementary medicine without any complexity, and they used me to give a veneer of respectability.”The BBC, thus far, remain unapologetic
“We take these allegations very seriously and we strongly refute them.We used two scientific consultants for the series, Prof Ernst and Dr Jack Tinker, dean emeritus of the Royal Society of Medicine, both of whom signed off the programme scripts.”
This is the same Jack Tinker who, as Chairman of the Ethics Committee of the Dr Foster organisation, also approved their “COMPLEMENTARY therapists Guide 2004”, and the utterly uncritical complementary practioner directory. The ‘Dr Foster’ organisation is a commercial business that supplies "management information", "market research services", "marketing services" and "information for the public". Let’s hope their services in conventional health care are a bit more critical than their evaluation of CAM. Their “Guide to [CAM] therapies” repeats all the usual pseudo-scientific gobbledygook in a totally uncritical way.
Singh’s article ended with some quotations from this site, concerning Sutherlandia and AIDS, with the remark made above, highlighted: "Comments about Aids were irresponsible and potentially dangerous".
- Science accuses BBC of medical quackery
Lois Rogers, in the Sunday Times for 26th March, reports on the same topic.
“Ernst yesterday released the contents of a letter that he has written to Martin Wilson, the series producer, criticising him for promoting “US-style anti-science”.
He said he felt “abused” by the programme makers: “It was as if they had instructions from higher up that this had to be a happy story about complementary medicine without any complexity, and they used me to give a veneer of respectability.” “
“This is no longer a fringe game played by new age people,” said Colquhoun. “It is beginning to erode intellectual standards at real universities.”
Later a letter appeared in defence of the programmes. Investigation showed that this letter had actually been written by the BBC and not all of the ‘signatories’ had seen it.This is dealt with in a separate post, Alternative Medicine series: dirty tricks at the BBC?