Warning: fopen(/home/dcscience/public_html/wp-content/wflogs/rules.php): failed to open stream: No such file or directory in /home/dcscience/public_html/wp-content/plugins/wordfence/vendor/wordfence/wf-waf/src/lib/waf.php on line 325

Warning: flock() expects parameter 1 to be resource, boolean given in /home/dcscience/public_html/wp-content/plugins/wordfence/vendor/wordfence/wf-waf/src/lib/waf.php on line 326

Warning: include(/home/dcscience/public_html/wp-content/wflogs/rules.php): failed to open stream: No such file or directory in /home/dcscience/public_html/wp-content/plugins/wordfence/vendor/wordfence/wf-waf/src/lib/waf.php on line 328

Warning: include(): Failed opening '/home/dcscience/public_html/wp-content/wflogs/rules.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /home/dcscience/public_html/wp-content/plugins/wordfence/vendor/wordfence/wf-waf/src/lib/waf.php on line 328

Warning: flock() expects parameter 1 to be resource, boolean given in /home/dcscience/public_html/wp-content/plugins/wordfence/vendor/wordfence/wf-waf/src/lib/waf.php on line 329

Warning: fclose() expects parameter 1 to be resource, boolean given in /home/dcscience/public_html/wp-content/plugins/wordfence/vendor/wordfence/wf-waf/src/lib/waf.php on line 330
assessment – DC's Improbable Science

LOB-vs
Download Lectures on Biostatistics (1971).
Corrected and searchable version of Google books edition

Download review of Lectures on Biostatistics (THES, 1973).

Latest Tweets
Categories
Archives

assessment

The last email of Stephan Grimm has had more views than any other on this blog. “Publish and perish at Imperial College London: the death of Stefan Grimm“. Since then it’s been viewed more than 210,000 times. The day after it was posted, the server failed under the load.

Since than, I posted two follow-up pieces. On December 23, 2014 “Some experiences of life at Imperial College London. An external inquiry is needed after the death of Stefan Grimm“. Of course there was no external inquiry.

And on April 9, 2015, after the coroner’s report, and after Imperial’s internal inquiry, “The death of Stefan Grimm was “needless”. And Imperial has done nothing to prevent it happening again“.

On September 24th 2015, I posted a memorial on the first anniversary of his death. It included some of Grimm’s drawings that his mother and sister sent to me.

That tragedy led to two actions by Imperial, the metrics report (2015) and the bullying report (2016).

Let’s look at the outcomes.

The 2015 metrics report

In February 2015 and investigation was set up into the use of metrics to evaluate people, In December 2015 a report was produced: Application and Consistency of Approach in the Use of Performance Metrics. This was an internal enquiry so one didn’t expect very much from it. Out of 1338 academic staff surveyed at the College, 309 (23% of the total) responded
another 217 started the survey but did not submit anything). One can only speculate about the low return. It could be that 87% of staff were happy, or it could be that 87% of staff were frightened to give their opinions. It’s true that some departments use few if any metrics to assess people so one wouldn’t expect strong responses from them.

My position is clear: metrics don’t measure the quality of science, in fact they corrupt science.

This is not Imperial’s view though. The report says:

5.1 In seeking to form a view on performance metrics, we started from the premise that, whatever their benefits or deficiencies, performance metrics pervade UK universities. From REF to NSS via the THE and their attendant league tables, universities are measured and ranked in many dimensions and any view of performance metrics has to be formed in this context.

In other words, they simply acquiesce in the use of measures that demonstrably don’t do what’s claimed for them.

Furthermore the statement that “performance metrics pervade UK universities” is not entirely true. At UCL we were told in 2015.

“We will evaluate the quality of staff contributions appropriately, focusing on the quality of individual research outputs and their impact rather than quantity or journal-level metrics.” .

And one of the comments quoted in Imperial’s report says

“All my colleagues at MIT and Harvard etc tell me they reject metrics because they lead to mediocre candidates. If Imperial really wants to be a leader, it has to be bold enough to judge based on quality.”

It is rather shameful that only five UK universities (out of 114 or so) have signed the San Francisco Declaration on Research Assessment (DORA). I’m very happy that UCL is one of them, along with Sussex and Manchester, Birmingham and Liverpool. Imperial has not signed.

Imperial’s report concludes

“each department should develop profiles of its academic staff based on a series of published (ie open and transparent [perhaps on the College intranet]:”

There seems to be a word missing here. Presumably this means “open and transparent metrics“.

The gist of the report seems to be that departments can carry on doing what they want, as long as they say what it is. That’s not good enough, in my opinion.

A review of Imperial College’s institutional culture and its impact on gender equality

Unlike the metrics report, this one was external: that’s good. But, unlike the metrics report, it is secret: that’s bad.

The report was written by Alison Phipps (Director of Gender Studies and Reader in Sociology University of Sussex). But all that’s been released is an 11 page summary, written by Imperial, not by the authors of the report. When I asked Phipps for a copy of the whole report I was told

“Unfortunately we cannot share the full report – this is an internal document to Imperial, and we have to protect our research participants who told us their stories on this basis.”

It’s not surprising that the people who told their stories are afraid of repercussions. But it’s odd that their stories are concealed from everyone but the people who are in a position to punish them.

The report seems to have been commissioned because of this incident.

“The university apologised to the women’s rugby team after they were left playing to an empty stadium when the coaches ferrying spectators back to campus were allowed to leave early.”

“a member of staff was overheard saying that they did not care “how those fat girls” got home,”

But the report wasn’t restricted to sexism. It covered the whole culture at Imperial. One problem was that only 127 staff
and 85 students participated. There is no way to tell whether those who didn’t respond were happy or whether they were scared.

Here are some quotations from Imperial’s own summary of the secret report.

“For most, the meaning was restricted to excellence in research despite the fact that the College’s publicised mission statement gives equal prominence to research and education in the excellence context”

“Participants saw research excellence in metricised terms, positioning the College as a top-level player within the UK and in the world.”

Words used by those critical of Imperial’s culture included ” ‘cutthroat’, ‘intimidating’, ‘blaming’ and ‘arrogant’ “.

“Many participants in the survey and other methods felt that the external focus on excellence had emphasised internal competition rather than collaboration. This competition was noted as often being individualistic and adversarial. ”

“It was felt that there was an all-consuming focus on academic performance, and negative attitudes towards those who did not do well or who were not as driven as others. There was a reported lack of community spirit in the College’s culture including departments being ‘played off against each other’”

“The research findings noted comments that the lack of communal space on the campus had contributed to a lack of a community spirit. It was suggested that the College had ‘an impersonal culture’ and groups could therefore self-segregate in the absence of mechanisms for them to connect. ”

“There were many examples given to the researchers of bullying and discriminatory behaviour towards staff and students. These examples predominantly reflected hierarchies in work or study arrangements. ”

“The researchers reported that many of the participants linked it with the ‘elite’ white masculinity of the majority population, although a few examples of unacceptable behaviour by female staff and students were also cited. Examples of misogynistic and homophobic conduct were given and one interviewee expressed concern that the ‘ingrained misogyny’ at Imperial was so deep that it had become normal.”

“Although the College describes itself as a supportive environment, and many positive examples of that support were cited, a number of participants felt that senior management would turn a blind eye to poor behaviour if the individual involved was of value to the College.”

“Despite Imperial’s ‘no tolerance’ stance on harassment and bullying and initiatives such as ‘Have Your Say’, the researchers heard that people did not ‘speak up’ about many issues, ranging from discrimination and abuse to more subtle practices that leave people feeling vulnerable, unheard or undermined.”

“Relations between PIs and contract researchers were especially difficult, and often gendered as the PI was very often a man and the researcher a woman.”

“It was reported that there was also a clear sense of staff and students feeling afraid to speak up about issues and not receiving clear information or answers due to unclear institutional processes and one-way communication channels.”

“This representation of Imperial College as machine rather than organism resonated with observations on a culture of fear and silence, and the lack of empathy and community spirit at the College.”

“Some of the participants identified a surface commitment to diversity and representation but a lack of substantive system processes to support this. The obstacles to participation in the way of doing things at Imperial, and the associated issues of fear and insecurity, were reported as leading to feelings of hopelessness, demotivation, and low morale among some staff and students.”

“Some participants felt that Athena SWAN had merely scratched the surface of issues or had just provided a veneer which concealed continuing inequalities and that events such as the annual Athena SWAN lecture were little more than a ‘box ticking exercise.’”

The conclusions are pretty weak: e.g.

“They [the report’s authors] urged the College to implement changes that would ensure that its excellence in research is matched by excellence in other areas.”

Of course, Imperial College says that it will fix the problems. “Imperial’s provost, James Stirling, said that the institution must do better and was committed to gender equality”.

But that is exactly what they said in 2003

“The rector [then Richard Sykes] acknowledged the findings that came out of the staff audit – Imperial College – A Good Place to Work? – undertaken in August 2002.”

“He reinforced the message that harassment or bullying would not be tolerated in the College, and promised commitment from Council members and the Executive Committee for their continuing support to equal opportunities.”

This was eleven years before the pressure applied to Stefan Grimm caused him to take his own life. As always, it sounds good. But it seems that, thirteen years later, Imperial is going through exactly the same exercise.

It would be interesting to know whether Imperial’s Department of Medicine is still adopting the same cruel assessment methods as it was in 2007. Other departments at Imperial have never used such methods. It’s a continual source of bafflement to me that medicine, the caring profession, seems to care less for its employees that most other departments.

Other universities

Imperial is certainly not unique in having these problems. They are endemic. For example, Queen Mary, Kings College London and Warwick University have had similar problems, among many others.

Managers must learn that organisations function better when employees have good morale and are happy to work. Once again, I quote Scott Burkun (The myths of Innovation, 2007).

“Creation is sloppy; discovery is messy; exploration is dangerous. What’s a manager to do? The answer in general is to encourage curiosity and accept failure. Lots of failure.”

All big organisations are much the same -dissent is squashed and punished. Committees are set up. Fine-sounding statements are issued. But nothing much changes.

It should not be so.

Follow-up

Jump to follow-up

Stop press. Financial report casts doubt on Trainor’s claims

Science has a big problem. Most jobs are desperately insecure. It’s hard to do long term thorough work when you don’t know whether you’ll be able to pay your mortgage in a year’s time. The appalling career structure for young scientists has been the subject of much writing by the young (e.g. Jenny Rohn) and the old, e.g Bruce Alberts. Peter Lawrence (see also Real Lives and White Lies in the Funding of Scientific Research, and by me.

Until recently, this problem was largely restricted to post-doctoral fellows (postdocs). They already have PhDs and they are the people who do most of the experiments. Often large numbers of them work for a single principle investigator (PI). The PI spends most of his her time writing grant applications and traveling the world to hawk the wares of his lab. They also (to variable extents) teach students and deal with endless hassle from HR.

The salaries of most postdocs are paid from grants that last for three or sometimes five years. If that grant doesn’t get renewed. they are on the streets.

Universities have come to exploit their employees almost as badly as Amazon does.

The periodical research assessments not only waste large amounts of time and money, but they have distorted behaviour. In the hope of scoring highly, they recruit a lot of people before the submission, but as soon as that’s done with, they find that they can’t afford all of them, so some get cast aside like worn out old boots. Universities have allowed themselves to become dependent on "soft money" from grant-giving bodies. That strikes me as bad management.

The situation is even worse in the USA where most teaching staff rely on research grants to pay their salaries.

I have written three times about the insane methods that are being used to fire staff at Queen Mary College London (QMUL).
Is Queen Mary University of London trying to commit scientific suicide? (June 2012)
Queen Mary, University of London in The Times. Does Simon Gaskell care? (July 2012) and a version of it appeared th The Times (Thunderer column)
In which Simon Gaskell, of Queen Mary, University of London, makes a cock-up (August 2012)

The ostensible reason given there was to boost its ratings in university rankings. Their vice-chancellor, Simon Gaskell, seems to think that by firing people he can produce a university that’s full of Nobel prize-winners. The effect, of course, is just the opposite. Treating people like pawns in a game makes the good people leave and only those who can’t get a job with a better employer remain. That’s what I call bad management.

At QMUL people were chosen to be fired on the basis of a plain silly measure of their publication record, and by their grant income. That was combined with terrorisation of any staff who spoke out about the process (more on that coming soon).

Kings College London is now doing the same sort of thing. They have announced that they’ll fire 120 of the 777 staff in the schools of medicine and biomedical sciences, and the Institute of Psychiatry. These are humans, with children and mortgages to pay. One might ask why they were taken on the first place, if the university can’t afford them. That’s simply bad financial planning (or was it done in order to boost their Research Excellence submission?).

Surely it’s been obvious, at least since 2007, that hard financial times were coming, but that didn’t dent the hubris of the people who took an so many staff. HEFCE has failed to find a sensible way to fund universities. The attempt to separate the funding of teaching and research has just led to corruption.

The way in which people are to be chosen for the firing squad at Kings is crude in the extreme. If you are a professor at the Institute of Psychiatry then, unless you do a lot of teaching, you must have a grant income of at least £200,000 per year. You can read all the details in the Kings’ “Consultation document” that was sent to all employees. It’s headed "CONFIDENTIAL – Not for further circulation". Vice-chancellors still don’t seem to have realised that it’s no longer possible to keep things like this secret. In releasing it, I take ny cue from George Orwell.

"Journalism is printing what someone else does not want printed: everything else is public relations.”

There is no mention of the quality of your research, just income. Since in most sorts of research, the major cost is salaries, this rewards people who take on too many employees. Only too frequently, large groups are the ones in which students and research staff get the least supervision, and which bangs per buck are lowest. The university should be rewarding people who are deeply involved in research themselves -those with small groups. Instead, they are doing exactly the opposite.

Women are, I’d guess, less susceptible to the grandiosity of the enormous research group, so no doubt they will suffer disproportionately. PhD students will also suffer if their supervisor is fired while they are halfway through their projects.

An article in Times Higher Education pointed out

"According to the Royal Society’s 2010 report The Scientific Century: Securing our Future Prosperity, in the UK, 30 per cent of science PhD graduates go on to postdoctoral positions, but only around 4 per cent find permanent academic research posts. Less than half of 1 per cent of those with science doctorates end up as professors."

The panel that decides whether you’ll be fired consists of Professor Sir Robert Lechler, Professor Anne Greenough, Professor Simon Howell, Professor Shitij Kapur, Professor Karen O’Brien, Chris Mottershead, Rachel Parr & Carol Ford. If they had the slightest integrity, they’d refuse to implement such obviously silly criteria.

Universities in general. not only Kings and QMUL have become over-reliant on research funders to enhance their own reputations. PhD students and research staff are employed for the benefit of the university (and of the principle investigator), not for the benefit of the students or research staff, who are treated as expendable cost units, not as humans.

One thing that we expect of vice-chancellors is sensible financial planning. That seems to have failed at Kings. One would also hope that they would understand how to get good science. My only previous encounter with Kings’ vice chancellor, Rick Trainor, suggests that this is not where his talents lie. While he was president of the Universities UK (UUK), I suggested to him that degrees in homeopathy were not a good idea. His response was that of the true apparatchik.

“. . . degree courses change over time, are independently assessed for academic rigour and quality and provide a wider education than the simple description of the course might suggest”

That is hardly a response that suggests high academic integrity.

The students’ petition is on Change.org.

Follow-up

The problems that are faced in the UK are very similar to those in the USA. They have been described with superb clarity in “Rescuing US biomedical research from its systemic flaws“, This article, by Bruce Alberts, Marc W. Kirschner, Shirley Tilghman, and Harold Varmus, should be read by everyone. They observe that ” . . . little has been done to reform the system, primarily because it continues to benefit more established and hence more influential scientists”. I’d be more impressed by the senior people at Kings if they spent time trying to improve the system rather than firing people because their research is not sufficiently expensive.

10 June 2014

Progress on the cull, according to an anonymous correspondent

“The omnishambles that is KCL management

1) We were told we would receive our orange (at risk) or green letters (not at risk, this time) on Thursday PM 5th June as HR said that it’s not good to get bad news on a Friday!

2) We all got a letter on Friday that we would not be receiving our letters until Monday, so we all had a tense weekend

3) I finally got my letter on Monday, in my case it was “green” however a number of staff who work very hard at KCL doing teaching and research are “orange”, un bloody believable

As you can imagine the moral at King’s has dropped through the floor”

18 June 2014

Dorothy Bishop has written about the Trainor problem. Her post ends “One feels that if KCL were falling behind in a boat race, they’d respond by throwing out some of the rowers”.

The students’ petition can be found on the #KCLHealthSOS site. There is a reply to the petition, from Professor Sir Robert Lechler, and a rather better written response to it from students. Lechler’s response merely repeats the weasel words, and it attacks a few straw men without providing the slightest justification for the criteria that are being used to fire people. One can’t help noticing how often knighthoods go too the best apparatchiks rather than the best scientists.

14 July 2014

A 2013 report on Kings from Standard & Poor’s casts doubt on Trainor’s claims

Download the report from Standard and Poor’s Rating Service

A few things stand out.

  • KCL is in a strong financial position with lower debt than other similar Universities and cash reserves of £194 million.
  • The report says that KCL does carry some risk into the future especially that related to its large capital expansion program.
  • The report specifically warns KCL over the consequences of any staff cuts. Particularly relevant are the following quotations
  • Page p3 “Further staff-cost curtailment will be quite difficult …pressure to maintain its academic and non-academic service standards will weigh on its ability to cut costs further.”
  • page 4 The report goes on to say (see the section headed outlook, especially the final paragraph) that any decrease in KCL’s academic reputation (e.g. consequent on staff cuts) would be likely to impair its ability to attract overseas students and therefore adversely affect its financial position.
  • page 10 makes clear that KCL managers are privately aiming at 10% surplus, above the 6% operating surplus they talk about with us. However, S&P considers that ‘ambitious’. In other words KCL are shooting for double what a credit rating agency considers realistic.

One can infer from this that

  1. what staff have been told about the cuts being an immediate necessity is absolute nonsense
  2. KCL was warned against staff cuts by a credit agency
  3. the main problem KCL has is its overambitious building policy
  4. KCL is implementing a policy (staff cuts) which S & P warned against as they predict it may result in diminishing income.

What on earth is going on?

16 July 2014

I’ve been sent yet another damning document. The BMA’s response to Kings contains some numbers that seem to have escaped the attention of managers at Kings.

10 April 2015

King’s draft performance management plan for 2015

This document has just come to light (the highlighting is mine).

It’s labelled as "released for internal consultation". It seems that managers are slow to realise that it’s futile to try to keep secrets.

The document applies only to Institute of Psychiatry, Psychology and Neuroscience at King’s College London: "one of the global leaders in the fields" -the usual tedious blah that prefaces every document from every university.

It’s fascinating to me that the most cruel treatment of staff so often seems to arise in medical-related areas. I thought psychiatrists, of all people, were meant to understand people, not to kill them.

This document is not quite as crude as Imperial’s assessment, but it’s quite bad enough. Like other such documents, it pretends that it’s for the benefit of its victims. In fact it’s for the benefit of willy-waving managers who are obsessed by silly rankings.

Here are some of the sillier bits.

"The Head of Department is also responsible for ensuring that aspects of reward/recognition and additional support that are identified are appropriately followed through"

And, presumably, for firing people, but let’s not mention that.

"Academics are expected to produce original scientific publications of the highest quality that will significantly advance their field."

That’s what everyone has always tried to do. It can’t be compelled by performance managers. A large element of success is pure luck. That’s why they’re called experiments.

" However, it may take publications 12-18 months to reach a stable trajectory of citations, therefore, the quality of a journal (impact factor) and the judgment of knowledgeable peers can be alternative indicators of excellence."

It can also take 40 years for work to be cited. And there is little reason to believe that citations, especially those within 12-18 months, measure quality. And it is known for sure that "the quality of a journal (impact factor)" does not correlate with quality (or indeed with citations).

Later we read

"H Index and Citation Impact: These are good objective measures of the scientific impact of
publications"

NO, they are simply not a measure of quality (though this time they say “impact” rather than “excellence”).

The people who wrote that seem to be unaware of the most basic facts about science.

Then

"Carrying out high quality scientific work requires research teams"

Sometimes it does, sometimes it doesn’t. In the past the best work has been done by one or two people. In my field, think of Hodgkin & Huxley, Katz & Miledi or Neher & Sakmann. All got Nobel prizes. All did the work themselves. Performance managers might well have fired them before they got started.

By specifying minimum acceptable group sizes, King’s are really specifying minimum acceptable grant income, just like Imperial and Warwick. Nobody will be taken in by the thin attempt to disguise it.

The specification that a professor should have "Primary supervision of three or more PhD students, with additional secondary supervision." is particularly iniquitous. Everyone knows that far too many PhDs are being produced for the number of jobs that are available. This stipulation is not for the benefit of the young. It’s to ensure a supply of cheap labour to churn out more papers and help to lift the university’s ranking.

The document is not signed, but the document properties name its author. But she’s not a scientist and is presumably acting under orders, so please don’t blame her for this dire document. Blame the vice-chancellor.

Performance management is a direct incentive to do shoddy short-cut science.

No wonder that The Economist says "scientists are doing too much trusting and not enough verifying—to the detriment of the whole of science, and of humanity".

Feel ashamed.

Jump to follow-up

Academic staff are going to be fired at Queen Mary University of London (QMUL). It’s possible that universities may have to contract a bit in hard times, so what’s wrong?

What’s wrong is that the victims are being selected in a way that I can describe only as insane. The criteria they use are guaranteed to produce a generation of second-rate spiv scientists, with a consequent progressive decline in QMUL’s reputation.

The firings, it seems, are nothing to do with hard financial times, but are a result of QMUL’s aim to raise its ranking in university league tables.

In the UK university league table, a university’s position is directly related to its government research funding. So they need to do well in the 2014 ‘Research Excellence Framework’ (REF). To achieve that they plan to recruit new staff with high research profiles, take on more PhD students and post-docs, obtain more research funding from grants, and get rid of staff who are not doing ‘good’ enough research.

So far, that’s exactly what every other university is trying to do. This sort of distortion is one of the harmful side-effects of the REF. But what’s particularly stupid about QMUL’s behaviour is the way they are going about it. You can assess your own chances of survival at QMUL’s School of Biological and Chemical Sciences from the following table, which is taken from an article by Jeremy Garwood (Lab Times Online. July 4, 2012). The numbers refer to the four year period from 2008 to 2011.

Category of staff

Research Output ­ Quantity
(No. of  Papers)

Research Output ­
Quality
(No. of  high quality papers)

Research Income (£)
(Total)

Research Income (£)
As Principal Investigator

Professor

11

2

400,000

 at least 200,000

Reader

9

2

320,000

at least 150,000

Senior Lecturer

7

1

260,000

 at least 120,000

Lecturer

5

1

200,000

 at least 100,000

In addition to the three criteria, ‘Research Output ‐ quality’, ‘Research Output – quantity’, and ‘Research Income’, there is a minimum threshold of 1 PhD completion for staff at each academic level. All this data is “evidenced by objective metrics; publications cited in Web of Science, plus official QMUL metrics on grant income and PhD completion.”

To survive, staff must meet the minimum threshold in three out of the four categories, except as follows:

Demonstration of activity at an exceptional level in either ‘research outputs’ or ‘research income’, termed an ‘enhanced threshold’, is “sufficient” to justify selection regardless of levels of activity in the other two categories. And what are these enhanced thresholds?
For research quantity: a mere 26 published items with at least 11 as significant author (no distinction between academic level); research quality: a modest 6 items published in numerically-favoured journals (e.g. impact factor > 7). Alternatively you can buy your survival with a total ‘Research Income’ of £1,000,000 as PI.

The university notes that the above criteria “are useful as entry standards into the new school, but they fall short of the levels of activity that will be expected from staff in the future. These metrics should not, therefore, be regarded as targets for future performance.”

This means that those who survived the redundancy criteria will simply have to do better. But what is to reassure them that it won’t be their turn next time should they fail to match the numbers? 

To help them, Queen Mary is proposing to introduce ‘D3’ performance management (www.unions.qmul.ac.uk/ucu/docs/d3-part-one.doc). Based on more ‘administrative physics’, D3 is shorthand for ‘Direction × Delivery × Development.’ Apparently “all three are essential to a successful team or organisation. The multiplication indicates that where one is absent/zero, then the sum is zero!”

D3 is based on principles of accountability: “A sign of a mature organisation is where its members acknowledge that they face choices, they make commitments and are ready to be held to account for discharging these commitments, accepting the consequences rather than seeking to pass responsibility.” Inspired?

I presume the D3 document must have been written by an HR person. It has all the incoherent use of buzzwords so typical of HR. And it says "sum" when it means "product" (oh dear, innumeracy is rife).

The criteria are utterly brainless. The use of impact factors for assessing people has been discredited at least since Seglen (1997) showed that the number of citations that a paper gets is not perceptibly correlated with the impact factor of the journal in which it’s published. The reason for this is the distribution of the number of citations for papers in a particular journal is enormously skewed. This means that high-impact journals get most of their citations from a few articles.

The distribution for Nature is shown in Fig. 1. Far from being gaussian, it is even more skewed than a geometric distribution; the mean number of citations is 114, but 69% of papers have fewer than the mean, and 24% have fewer than 30 citations. One paper has 2,364 citations but 35 have 10 or fewer. ISI data for citations in 2001 of the 858 papers published in Nature in 1999 show that the 80 most-cited papers (16% of all papers) account for half of all the citations (from Colquhoun, 2003)

impact

The Institute of Scientific Information, ISI, is guilty of the unsound statistical practice of characterizing a distribution by its mean only, with no indication of its shape or even its spread. School of Biological and Chemical Sciences-QMUL is expecting everyone has to be above average in the new regime. Anomalously, the thresholds for psychologists are lower because it is said that it’s more difficult for them to get grants. This undermines even the twisted logic applied at the outset.  

All this stuff about skewed distributions is, no doubt, a bit too technical for HR people to understand. Which, of course, is precisely why they should have nothing to do with assessing people.

At a time when so may PhDs fail to get academic jobs we should be limiting the numbers. But QMUL requires everyone to have a PhD student, not for the benefit of the student, but to increase its standing in league tables. That is deeply unethical.

The demand to have two papers in journals with impact factor greater than seven is nonsense. In physiology, for example, there are only four journals with an impact factor greater that seven and three of them are review journals that don’t publish original research. The two best journals for electrophysiology are Journal of Physiology (impact factor 4.98, in 2010) and Journal of General Physiology (IF 4.71). These are the journals that publish papers that get you into the Royal Society or even Nobel prizes. But for QMUL, they don’t count.

I have been lucky to know well three Nobel prize winners. Andrew Huxley. Bernard Katz, and Bert Sakmann. I doubt that any of them would pass the criteria laid down for a professor by QMUL. They would have been fired.

The case of Sakmann is analysed in How to Get Good Science, [pdf version]. In the 10 years from 1976 to 1985, when Sakmann rose to fame, he published an average of 2.6 papers per year (range 0 to 6). In two of these 10 years he had no publications at all. In the 4 year period (1976 – 1979 ) that started with the paper that brought him to fame (Neher & Sakmann, 1976) he published 9 papers, just enough for the Reader grade, but in the four years from 1979 – 1982 he had 6 papers, in 2 of which he was neither first nor last author. His job would have been in danger if he’d worked at QMUL. In 1991 Sakmann, with Erwin Neher, got the Nobel Prize for Physiology or Medicine.

The most offensive thing of the lot is the way you can buy yourself out if you publish 26 papers in the 4 year period. Sakmann came nowhere near this. And my own total, for the entire time from my first paper (1963) until I was elected to the Royal Society (May 1985) was 27 papers (and 7 book chapters). I would have been fired.

Peter Higgs had no papers at all from the time he moved to Edinburgh in 1960, until 1964 when his two paper’s on what’s now called the Higgs’ Boson were published in Physics Letters. That journal now has an impact factor less than 7 so Queen Mary would not have counted them as “high quality” papers, and he would not have been returnable for the REF. He too would have been fired.

The encouragement to publish large numbers of papers is daft. I have seen people rejected from the Royal Society for publishing too much. If you are publishing a paper every six weeks, you certainly aren’t writing them, and possibly not even reading them. Most likely you are appending your name to somebody else’s work with little or no checking of the data. Such numbers can be reached only by unethical behaviour, as described by Peter Lawrence in The Mismeasurement of Science. Like so much managerialism, the rules provide an active encouragement to dishonesty.

In the face of such a boneheaded approach to assessment of your worth, it’s the duty of any responsible academic to point out the harm that’s being done to the College. Richard Horton, in the Lancet, did so in Bullying at Barts. There followed quickly letters from Stuart McDonald and Nick Wright, who used the Nuremburg defence, pointing out that the Dean (Tom Macdonald) was just obeying orders from above. That has never been as acceptable defence. If Macdonald agreed with the procedure, he should be fired for incompetence. If he did not agree with it he should have resigned.

It’s a pity, because Tom Macdonald was one of the people with whom I corresponded in support of Barts’ students who, very reasonably, objected to having course work marked by homeopaths (see St Bartholomew’s teaches antiscience, but students revolt, and, later, Bad medicine. Barts sinks further into the endarkenment). In that case he was not unreasonable, and, a mere two years later I heard that he’d taken action.

To cap it all, two academics did their job by applying a critical eye to what’s going on at Queen Mary. They wrote to the Lancet under the title Queen Mary: nobody expects the Spanish Inquisition

"For example, one of the “metrics” for research output at professorial level is to have published at least two papers in journals with impact factors of 7 or more. This is ludicrous, of course—a triumph of vanity as sensible as selecting athletes on the basis of their brand of track suit. But let us follow this “metric” for a moment. How does the Head of School fair? Zero, actually. He fails. Just consult Web of Science. Take care though, the result is classified information. HR’s “data” are marked Private and Confidential. Some things must be believed. To question them is heresy."

Astoundingly, the people who wrote this piece are now under investigation for “gross misconduct”. This is behaviour worthy of the University of Poppleton, as pointed out by the inimitable Laurie Taylor, in Times Higher Education (June 7)

The rustle of censorship

It appears that last week’s edition of our sister paper, The Poppleton Evening News, carried a letter from Dr Gene Ohm of our Biology Department criticising this university’s metrics-based redundancy programme.

We now learn that, following the precedent set by Queen Mary, University of London, Dr Ohm could be found guilty of “gross misconduct” and face “disciplinary proceedings leading to dismissal” for having the effrontery to raise such issues in a public place.

Louise Bimpson, the corporate director of our ever-expanding human resources team, admitted that this response might appear “severe” but pointed out that Poppleton was eager to follow the disciplinary practices set by such soon-to-be members of the prestigious Russell Group as Queen Mary. Thus it was only to be expected that we would seek to emulate its espousal of draconian censorship. She hoped this clarified the situation.

David Bignell, emeritus professor of zoology at Queen Mary hit the nail on the head.

"These managers worry me. Too many are modest achievers, retired from their own studies, intoxicated with jargon, delusional about corporate status and forever banging the metrics gong. Crucially, they don’t lead by example."

What the managers at Queen Mary have failed to notice is that the best academics can choose where to go.

People are being told to pack their bags and move out with one day’s notice. Access to journals stopped, email address removed, and you may need to be accompanied to your (ex)-office. Good scientists are being treated like criminals.

What scientist in their right mind would want to work at QMUL, now that their dimwitted assessment methods, and their bullying tactics, are public knowledge?

The responsibility must lie with the principal, Simon Gaskell. And we know what the punishment is for bringing your university into disrepute.

Follow-up

Send an email. You may want to join the many people who have already written to QMUL’s principal, Simon Gaskell (principal@qmul.ac.uk), and/or to Sir Nicholas Montagu, Chairman of Council, n.montagu@qmul.ac.uk.

Sunday 1 July 2012. Since this blog was posted after lunch on Friday 29th June, it has had around 9000 visits from 72 countries. Here is one of 17 maps showing the origins of 200 of the hits in the last two days

map

The tweets about QMUL are collected in a Storify timeline.

I’m reminded of a 2008 comment, on a post about the problems imposed by HR, In-human resources, science and pizza.

Thanks for that – I LOVED IT. It’s fantastic that the truth of HR (I truly hate that phrase) has been so ruthlessly exposed. Should be part of the School Handbook. Any VC who stripped out all the BS would immediately retain and attract good people and see their productivity soar.

That’s advice that Queen Mary should heed.

Part of the reason for that popularity was Ben Goldacre’s tweet, to his 201,000 followers

“destructive, unethical and crude metric incentives in academia (spotlight QMUL) bit.ly/MFHk2H by @david_colquhoun

3 July 2012. I have come by a copy of this email, which was sent to Queen Mary by a senior professor from the USA (word travels fast on the web). It shows just how easy it is to destroy the reputation of an institution.

Sir Nicholas Montagu, Chairman of Council, and Principal Gaskell,

I was appalled to read the criteria devised by your University to evaluate its faculty.   There are so flawed it is hard to know where to begin. 

Your criteria are antithetical to good scientific research.  The journals are littered with weak publications, which are generated mainly by scientists who feel the pressure to publish, no matter whether the results are interesting, valid, or meaningful.  The literature is flooded by sheer volume of these publications.

Your attempt to require “quality” research is provided by the requirement for publications in “high Impact Factor” journals.  IF has been discredited among scientists for many reasons:  it is inaccurate in not actually reflecting the merit of the specific paper, it is biased toward fields with lots of scientists, etc.  The demand for publications in absurdly high IF journals encourages, and practically enforces scientific fraud.  I have personally experienced those reviews from Nature demanding one or two more “final” experiments that will clinch the publication.  The authors KNOW how these experiments MUST turn out.  If they want their Nature paper (and their very academic survival if they are  at a brutal, anti-scientific university like QMUL), they must get the “right” answer.  The temptation to fudge the data to get this answer is extreme.  Some scientists may even be able to convince themselves that each contrary piece of data that they discard to ensure the “correct” answer is being discarded for a valid reason.  But the result is that scientific misconduct occurs.  I did not see in your criteria for “success” at QMUL whether you discount retracted papers from the tally of high IF publications, or perhaps the retraction itself counts as yet another high IF publication!

Your requirement for each faculty to have one or more postdocs or students promotes the abusive exploitation of these individuals for their cheap labor, and ignores the fact that they are being “trained” for jobs that do not exist. 

The “standards” you set are fantastically unrealistic.  For example, funding is not graded, but a sharp step function – we have 1 or 2 or 0 grants and even if the average is above your limits, no one could sustain this continuously. Once you have fired every one of your faculty, which will almost certainly happen within 1-2 rounds of pogroms, where will you find legitimate scientists who are willing to join such a ludicrous University?

4 July 2012.

Professor John F. Allen is Professor of Biochemistry at Queen Mary, University of London, and distinguished in the fields of Photosynthesis, Chloroplasts, Mitochondria, Genome function and evolution and Redox signalling. He, with a younger colleague, wrote a letter to the Lancet, Queen Mary: nobody expects the Spanish Inquisition. It is an admirable letter, the sort of thing any self-respecting academic should write. But not according to HR. On 14 May, Allen got a letter from HR, which starts thus.

14th May 2012

Dear Professor Allen

I am writing to inform you that the College had decided to commence a factfinding investigation into the below allegation: That in writing and/or signing your name to a letter entitled "Queen Mary: nobody expects the Spanish Inquisition," (enclosed) which was published in the Lancet online on 4th May 2012, you sought to bring the Head of School of Biological and Chemical Sciences and the Dean for Research in the School of Medicine and Dentistry into disrepute.

. . . .

Sam Holborn
HR Consultant- Science & Engineering

Download the entire letter. It is utterly disgraceful bullying. If anyone is bringing Queen Mary into disrepute, it is Sam Holborn and the principal, Simon Gaskell.

Here’s another letter, from the many that have been sent. This is from a researcher in the Netherlands.

Dear Sir Nicholas,

I am addressing this to you in the hope that you were not directly involved in creating this extremely stupid set of measures that have been thought up, not to improve the conduct of science at QMUL, but to cheat QMUL’s way up the league tables over the heads of the existing academic staff.

Others have written more succinctly about the crass stupidity of your Human Resources department than I could, and their apparent ignorance of how science actually works. As your principal must bear full responsibility for the introduction of these measures, I am not sending him a copy of this mail. I am pretty sure that his “principal” mail address will no longer be operative.

We have had a recent scandal in the Netherlands where a social psychology professor, who even won a national “Man of the Year” award, as well as as a very large amount of research money, was recently exposed as having faked all the data that went into a total number of articles running into three figures. This is not the sort of thing one wants to happen to one’s own university. He would have done well according to your REF .. before he was found out.

Human Resources departments have gained too much power, and are completely incompetent when it comes to judging academic standards. Let them get on with the old dull, and gobbledigook-free, tasks that personnel departments should be carrying out.


Yours,

5 July 2012.

Here’s another letter. It’s from a member of academic staff at QMUL, someone who is not himself threatened with being fired. It certainly shows that I’m not making a fuss about nothing. Rather, I’m the only person old enough to say what needs to be said without fear of losing my job and my house.

Dear Prof. Colquhoun,

I am an academic staff member in SBCS, QMUL.  I am writing from my personal email account because the risks of using my work account to send this email are too great.

I would like to thank you for highlighting our problems and how we have been treated by our employer (Queen Mary University of London), in your blog. I would please urge you to continue to tweet and blog about our plight, and staff in other universities experiencing similarly horrific working conditions.

I am not threatened with redundancy by QMUL, and in fact my research is quite successful. Nevertheless, the last nine months have been the most stressful of all my years of academic life. The best of my colleagues in SBCS, QMUL are leaving already and I hope to leave, if I can find another job in London. 

Staff do indeed feel very unfairly treated, intimidated and bullied. I never thought a job at a university could come to this.
Thank you again for your support. It really does matter to the many of us who cannot really speak out openly at present.
Best regards,

 

In a later letter, the same person pointed out

"There are many of us who would like to speak more openly, but we simply cannot."

"I have mortgage . . . . Losing my job would probably mean losing my home too at this point."

"The plight of our female staff has not even been mentioned. We already had very few female staff. And with restructuring, female staff are more likely to be forced into teaching-only contracts or indeed fired"."

"total madness in the current climate – who would want to join us unless desperate for a job!"

“fuss about nothing” – absolutely not. It is potentially a perfect storm leading to teaching and research disaster for a university!  Already the reputation of our university has been greatly damaged. And senior staff keep blaming and targeting the “messengers"."

6 July 2012.

Througn the miracle of WiFi, this is coming from Newton, MA. The Lancet today has another editorial on the Queen Mary scandal.

"As hopeful scientists prepare their applications to QMUL, they should be aware that, behind the glossy advertising, a sometimes harsh, at times repressive, and disturbingly unforgiving culture awaits them."

That sums it up nicely.

24 July 2012. I’m reminded by Nature writer, Richard van Noorden (@Richvn) that Nature itself has wriiten at least twice about the iniquity of judging people by impact factors. In 2005 Not-so-deep impact said

"Only 50 out of the roughly 1,800 citable items published in those two years received more than 100 citations in 2004. The great majority of our papers received fewer than 20 citations."

"None of this would really matter very much, were it not for the unhealthy reliance on impact factors by administrators and researchers’ employers worldwide to assess the scientific quality of nations and institutions, and often even to judge individuals."

And, more recently, in Assessing assessment” (2010).

29 July 2012. Jonathan L Rees. of the University of Edinburgh, ends his blog:

"I wonder what career advice I should offer to a young doctor circa 2012. Apart from not taking a job at Queen Mary of course. "

How to select candidates

I have, at various times, been asked how I would select candidates for a job, if not by counting papers and impact factors. This is a slightly modified version of a comment that I left on a blog, which describes roughly what I’d advocate

After a pilot study the entire Research Excellence Framework (which attempts to assess the quality of research in every UK university) made the following statement.

“No sub-panel will make any use of journal impact factors, rankings, lists or the perceived standing of publishers in assessing the quality of research outputs”

It seems that the REF is paying attention to the science not to bibliometricians.

It has been the practice at UCL to ask people to nominate their best papers (2 -4 papers depending on age). We then read the papers and asked candidates hard questions about them (not least about the methods section). It’s a method that I learned a long time ago from Stephen Heinemann, a senior scientist at the Salk Institute. It’s often been surprising to learn how little some candidates know about the contents of papers which they themselves select as their best. One aim of this is to find out how much the candidate understands the principles of what they are doing, as opposed to following a recipe.

Of course we also seek the opinions of people who know the work, and preferably know the person. Written references have suffered so much from ‘grade inflation’ that they are often worthless, but a talk on the telephone to someone that knows both the work, and the candidate, can be useful, That, however, is now banned by HR who seem to feel that any knowledge of the candidate’s ability would lead to bias.

It is not true that use of metrics is universal and thank heavens for that. There are alternatives and we use them.

Incidentally, the reason that I have described the Queen Mary procedures as insane, brainless and dimwitted is because their aim to increase their ratings is likely to be frustrated. No person in their right mind would want to work for a place that treats its employees like that, if they had any other option. And it is very odd that their attempt to improve their REF rating uses criteria that have been explicitly ruled out by the REF. You can’t get more brainless than that.

This discussion has been interesting to me, if only because it shows how little bibliometricians understand how to get good science.

Two weeks left to stop the Department of Health making a fool of itself. Email your response to tne Pittilo consultation to this email address HRDListening@dh.gsi.gov.uk

I’ve had permission to post a submission that has been sent to the Pittilo consultation. The whole document can be downloaded here. I have removed the name of the author. It is written by the person who has made some excellent contributions to this blog under the pseudonym "Allo V Psycho".

The document is a model of clarity, and it ends with constructive suggestions for forms of regulation that will, unlike the Pittilo proposals, really protect patients

Here is the summary. The full document explains each point in detail.

Executive Summary
Statutory regulation lends prestige, but needs to be balanced by a requirement for practitioners to be competent, as is the case for doctors and nurses. Regulation almost exclusively deals with conduct, but the unique risks posed by alternative medicine are not addressed by this. The harms which will arise from licensing practitioners who are not required to show evidence of competence and efficacy are:

  • Harm 1. Misdiagnosis of serious conditions.  Alternative practitioners offer to diagnose illnesses without proper training. This can lead to avoidable death, such as treating an ectopic pregnancy with ginger.
  • Harm 2. Withdrawal from treatment. Clients of alternative practitioners risk being encouraged to withdraw from life saving treatments in favours of treatments without evidence, as in the death of baby Gloria Thomas.
  • Harm 3. Harms arising from the nature of the alternative practice, but not covered by the regulatory framework, such as adulterated herbal remedies.
  • Harm 4.  Lack of informed consent. If alternative practitioners are not required to study or show evidence of efficacy, how can they inform patients of their options?
  • Harm 5. Equity. Doctors and nurses have to use evidence based methods, but it is proposed that alternative practitioners are not held to this standard. Is this fair? Health Minsters should ask themselves if they advocate withdrawing the requirement for evidence based treatment from doctors and nurses. If not, why not? And if not, why should alternative practitioners be treated differently?
  • Harm 6. Promotion of irrationality. If no evidence of efficacy is required, where do you draw the line? Witch doctoring is a ‘traditional practice’ in communities in the UK, and astrology is used by some herbal healers.
  • Harm 7. Opportunity Costs. If no evidence of efficacy is required of alternative medicine, significant sums will be wasted by individuals and by the NHS.
  • Harm 8. Reputational harms for UK Higher Education. UK Honours Degrees are based on the ability to think critically and to assess evidence. Alternative medicine Degree programmes do not require this. These positions are not compatible.
  • Harm 9.  Health care futures. We are making slow but steady progress on health indicators through the use of evidence based methods. Why should the requirement for evidence be abandoned now?

Instead, safe regulation of alternative practitioners should be through:

  • The Medicines and Healthcare Products Regulatory Agency
  • The Office of Trading Standards via the Unfair Trading Consumer Protection Regulations,
  • A new Health Advertising Standards Authority, modelled on the successful Cancer Act 1939.

The first two recommendations for effective regulation are much the same as mine, but the the third one is interesting. The problem with the Cancer Act (1939), and with the Unfair Trading regulations, is that they are applied very erratically. They are the responsibility of local Trading Standards offices, who have, as a rule, neither the expertise nor the time to enforce them effectively. A Health Advertising Standards Authority could perhaps take over the role of enforcing existing laws. But it should be an authority with teeth. It should have the ability to prosecute. The existing Advertising Standards Authority produces, on the whole, excellent judgements but it is quite ineffective because it can do very little.

A letter from an acupuncturist

I had a remarkable letter recently from someone who actually practises acupuncture. Here are some extracts.

“I very much enjoy reading your Improbable Science blog. It’s great to see good old-fashioned logic being applied incisively to the murk and spin that passes for government “thinking” these days.”

“It’s interesting that the British Acupuncture Council are in favour of statutory regulation. The reason is, as you have pointed out, that this will confer a respectability on them, and will be used as a lever to try to get NHS funding for acupuncture. Indeed, the BAcC’s mission statement includes a line “To contribute to the development of healthcare policy both now and in the future”, which is a huge joke when they clearly haven’t got the remotest idea about the issues involved.”

“Before anything is decided on statutory regulation, the British Acupuncture Council is trying to get a Royal Charter. If this is achieved, it will be seen as a significant boost to their respectability and, by implication, the validity of state-funded acupuncture. The argument will be that if Physios and O.T.s are Chartered and safe to work in the NHS, then why should Chartered Acupuncturists be treated differently? A postal vote of 2,700 BAcC members is under-way now and they are being urged to vote “yes”. The fact that the Privy Council are even considering it, is surprising when the BAcC does not even meet the requirement that the institution should have a minimum of 5000 members (http://www.privy-council.org.uk/output/Page45.asp). Chartered status is seen as a significant stepping-stone in strengthening their negotiating hand in the run-up to statutory regulation.”

“Whatever the efficacy of acupuncture, I would hate to see scarce NHS resources spent on well-meaning, but frequently gormless acupuncturists when there’s no money for the increasing costs of medical technology or proven life-saving pharmaceuticals.”

“The fact that universities are handing out a science degree in acupuncture is a testament to how devalued tertiary education has become since my day. An acupuncture degree cannot be called “scientific” in any normal sense of the term. The truth is that most acupuncturists have a poor understanding of the form of TCM taught in P.R.China, and hang on to a confused grasp of oriental concepts mixed in with a bit of New Age philosophy and trendy nutritional/life-coach advice that you see trotted out by journalists in the women’s weeklies. This casual eclectic approach is accompanied by a complete lack of intellectual rigour.

My view is that acupuncturists might help people who have not been helped by NHS interventions, but, in my experience, it has very little to do with the application of a proven set of clinical principles (alternative or otherwise). Some patients experience remission of symptoms and I’m sure that is, in part, bound up with the psychosomatic effects of good listening, and non-judgemental kindness. In that respect, the woolly-minded thinking of most traditional acupuncturists doesn’t really matter, they’re relatively harmless and well-meaning, a bit like hair-dressers. But just because you trust your hairdresser, it doesn’t mean hairdressers deserve the Privy Council’s Royal Charter or that they need to be regulated by the government because their clients are somehow supposedly “vulnerable”.”

Earlier postings on the Pittilo recommendations

A very bad report: gamma minus for the vice-chancellor http://www.dcscience.net/?p=235

Article in The Times (blame subeditor for the horrid title)
http://www.timesonline.co.uk/tol/comment/columnists/guest_contributors/article4628938.ece

Some follow up on The Times piece
http://www.dcscience.net/?p=251

The Health Professions Council breaks its own rules: the result is nonsense
http://www.dcscience.net/?p=1284

Chinese medicine -acupuncture gobbledygook revealed
http://www.dcscience.net/?p=1950

Consultation opens on the Pittilo report: help top stop the Department of Health making a fool of itself  http://www.dcscience.net/?p=2007

Why degrees in Chinese medicine are a danger to patients  http://www.dcscience.net/?p=2043

One month to stop the Department of Health endorsing quackery.  The Pittilo questionnaire, http://www.dcscience.net/?p=2310
 

Follow-up

Jump to follow-up

The first post was NICE falls for Bait and Switch by acupuncturists and chiropractors: it has let down the public and itself.

That was followed by NICE fiasco, part 2. Rawlins should withdraw guidance and start again.

Since then, something of a maelstrom has engulfed NICE, so it’s time for an update.

It isn’t only those who are appalled that NHS should endorse voodoo medicine on the basis of very slim evidence who are asking NICE to rethink their guidance on low back pain. Pain specialists are up in arms too, and have even started a blog, ‘Not Nearly as NICE as you think …‘, to express their views. Equally adverse opinions are being expressed in the Britsh Medical Journal. A letter there is signed by over 50 specialists in pain medicine. It ends thus

“Because of these new guidelines patients will continue to experience unnecessary pain and suffering and their rights to appropriately individually tailored treatment have been removed on the basis of a flawed analysis of available evidence. We believe the guidelines do not reflect best practice, remove patient choice and are not in our patients’ best interests.”

In a contribution headed “NICE misguidance”. Dr Michel Vagg ends

It seems to me that this guideline has been used as a propaganda vehicle to allow cherry-picked evidence to be enshrined as doctrine. This is an abuse of the guideline development process . . . ”

I have to say, though, that it seems to me that some of these people are promoting their own interests as much as chiropractors and acupuncturists. The evidence that spinal injections produce worthwhile benefits seems to be as thin as the evidence that chiropractic and acupuncture produce worthwhile benefits. But no doubt the injections are good for the budgets of PCTs or private practice doctors.. Could it perhaps be the case that some of the clinicians’ anger is being generated by doctors who are rushing to defend their own favourite ineffective treatment?

Why, oh why, can’t either NICE or the pain consultants bring themselves to state the obvious, that nothing works very well. The only thing that can be said for most of the regular treatments is that although they may not be much more effective than acupuncture or chiropractic, at least they don’t come with the intellectually-offensive hokum that accompanies the latter. Very sensible attempts have been made to identify the cause of low back pain [reviewed here], Occasionally they succeed. Mostly they don’t.

One clinician’s letter deserves special attention because it goes into the evidence, and the costs, in some detail. Its conclusions are very different from those in the NICE guidance.

The letter, a Review of NICE guidance, is from Dr C.J.D. Wells [download the whole letter].   He is a pain relief consultant from Liverpool.

Let’s look at some highlights.

Wells points out the absurdity of the cost estimates

“In the pricing section, they estimate that this will require an increase of facilities so that 3,500 patients can be treated instead of 1,000 at present (again see comments on pricing). This is not many treatments for the 20 million sufferers, of whom we can estimate that at least 2 million will have significant long-term disability and psychological distress”

And that is without even costing all the secondary costs of miseducating a new generation of students in fables about “Qi”, meridians, energy flow, subluxations and innate intelligence.

“The abysmal ignorance of the committee is reflected in the poor overall advice. So if you have a committee with special interests in Exercise, Manipulation, PMP’s, and Surgery, and you call an expert on Acupuncture, you get advice to use Exercise, Manipulation, Acupuncture, PMP’s and Surgery. Amazing.”

Another pain consutant, Charles Guaci, says in a comment in the Daily Mail.

I am a Pain Consultant of 30 years experience, have published two books (one translated into different languages).

NICE never asked me for my opinion.

This is the most ridicuculous pseudo-scientific document I have ever seen.

The panel consisted of a surgeon, psychologist, osteopath, acupuncturist a physiotherapist and an academic; not one pain consultant! The conclusions are simply a means of increasing the employment of their friends!

All evidence submitted to NICE was ignored.

It is almost certain than unless NICE rethink their ideas that Pain Consultants will be seeking a judicial review as well as full disclosure of how the panel arrived at their bizarre findings under the Freedom of information act.

Patients should realise that they are being taken for a ride.

Despite the outcry from opponents of magic medicine and from pain specialists, the assessment by the normally excellent NHS Choices site was disappointing. It made no mention at all of the secondary consequences of recommending CAM and described the assertions of the guidance group quite uncritically.

The reputation of NICE

NICE has been criticised before, though usually unjustly. In the past I have often supported them. For example. when NICE said that treatment of dementia with anticholinesterase drugs like galantamine was ineffective, there was a great outcry, but NICE were quite right. There is little or no rationale for such treatments, and more importantly, very little evidence that they work. But patients, especially when they are desperate, have greater faith in drug treatments than most pharmacologists, They want to clutch at straws. A bit like the NICE guidance committee, faced with a bunch of treatments most of which are almost ineffective, clutched at the straws of acupuncture and chiropractic. But this time it isn’t only the patients who are cross. It is most of the medical and scientific world too.

One interpretation of these bizarre events is that they represent a case of medical/scientific arrogance. Ben Goldacre wrote of another aspect of the same problem thus week, in Dodgy academic PR [download the paper on which this is based].

The first job of a scientist is to say openly when the answer to a question is not known. But scientists are under constant pressure to exaggerate the importance of their results. Last year we published an article which I feel may, if verified, turn out to be the second most important that I have ever been an author on. Because it happened to be published in Nature (not because of its quality), a press release was written (by an arts graduate!). It took some argument to prevent the distorted and exaggerated account being imposed on the public. This is typical of the sort of thing reported in Goldacre’s column. I reported a similar case a while ago, Why honey isn’t a wonder cough cure: more academic spin.

If NICE does not reconsider this guidance, it is hard to see how it can be taken seriously in the future. I hope that when NICE’s director, Professor Sir Michael Rawlins, returns from his trips abroad, he will find time to look at the case himself.

Indirectly, then, it can be argued that NICE’s bizarre guidance is just another manifestation of the management of science being passed from the hands of scientists into the hands of administrators and spin experts. It is yet another example of DC’s rule

Never trust anyone who uses the word ‘stakeholder’

Some bone-headed bureaucrat decides that any charlatan or quack is a ‘stakeholder’ in the provision of NHS care and gives them a quite disproportionate say in how taxpayers’ money is spent. The bureaucrats are so busy following processes and procedures, ticking boxes, and so deficient in scientific education, that they failed to notice that they’ve been caught out by the old trick of used car salesmen, bait and switch.

The consequences

The expected consequences have already started to materialise. The Prince of Wales’ Foundation for Magic Medicine is jubilant about having been endorsed by NICE. And I’m told that “The chiropractors have now just written letters to all health boards in Scotland asking for contracts for their services to deal with back pain”.

There could hardly have been a worse time for NICE to endorse chiropractic. We are in the middle of a storm about free speech because of the disgraceful action of the British Chiropractic Associaton in suing one of our best science writers, Simon Singh, for defamation because he had the temerity to express an opinion, Of course, even if the BCA wins in court, it will be the overall loser, because chiropractic claims are now being scrutinised as never before (just look at what they told me).

Follow-up

A much-cited paper. The paper that is most often cited by chiropractors who claim to be able to cure colic by spinal manipulation is Klougart N, Nilsson N and Jacobsen J (1989) Infantile Colic Treated by Chiropractors: A Prospective Study of 316 Cases, J Manip Physiol Ther,12:281-288. This is not easy to get hold of but Steve Vogel has sent me s scanned copy which you can download here. As evidence it is about as useless as the infamous Spence study so beloved of homeopaths. There was no control group at all. It simply follows 316 babies and found that most of them eventually got better. Well, they do, don’t they? It is a sign of the pathetic standard of reaearch in chiropractic that anyone should think this paper worth mentioning at all.

June 6 2009. More flak for NICE from the Royal College of Anaesthetists, and more adverse comment in the BMJ. And of course the blogs. for example, “If this is “evidence based medicine” I want my old job back“.

“Acupuncture on the NHS: a dangerous precedent”: a good analysis at counterknowledge.com.

June 6 2009, Comment sent to the BMJ. The comment was submitted, as below, early on Friday 5th June. The BMJ said it was a “sensitive issue” and for the next five days lawyers pondered over it.

Underwood and Littlejohns describe their guidance as being a “landmark”. I can only agree with that description. It is the first time that NICE has ever endorsed alternative medicine in the face of all the evidence. The guidance group could hardly have picked a worse moment to endorse chiropractic. Chiropractors find it so hard to find evidence for their practices that, when one of our finest science writers, Simon Singh, asked to see the evidence they sued him for defamation. I suggest that the guidance group should look at the formidable list of people who are supporting Singh, after his brave decision to appeal against this iniquitous persecution.

Of course I’m sure this bizarre decision has nothing to do with the presence on the guidance group of Peter Dixon, chair of the General Chiropractic Council. Nevertheless, I am curious to know why it is that when I telephoned two of the practices belonging to Peter Dixon Associates, I was told that they could probably treat infantile colic and asthma. Such claims have just been condemned by the Advertising Standards Authority.

The low back pain guidance stands a good chance of destroying NICE’s previously excellent reputation for dispassionate assessment of benefits and costs. Yes, that is indeed a landmark of sorts.

If NICE is ever to recover its reputation, I think that it will have to start again. Next time it will have to admit openly that none of the treatments work very well in most cases. And it will have to recognise properly the disastrous cultural consequences of giving endorsement to people who, when asked to produce evidence, resort to legal intimidation.

Eventually, on Wednesday 10 June the comment appeared in the BMJ, and it wasn’t greatly changed. Nevertheless if is yet another example of legal chill. This is the final version.

Underwood and Littlejohns describe their guidance as being a “landmark”. I can only agree with that description. It is the first time that NICE has ever endorsed alternative medicine in the face of all the evidence. The guidance group could hardly have picked a worse moment to endorse chiropractic. Chiropractors are so sensitive about criticisms of their practices that, when one of our finest science writers, Simon Singh, queried the evidence-base for their therapeutic claims they sued him for defamation. I suggest that the guidance group should look at the formidable list of people who are supporting Singh, after his brave decision to appeal against an illiberal court ruling in this iniquitous persecution.

One wonders whether this bizarre decision by NICE has anything to do with the presence on the guidance group of Peter Dixon, chair of the General Chiropractic Council. I am also curious to know why it is that when I telephoned two of the practices belonging to Peter Dixon Associates, I was told that chiropractic could be effective in the treatment of infantile colic and asthma. Similar claims about treating colic have just been condemned by the Advertising Standards Authority.

The low back pain guidance stands a good chance of destroying NICE’s previously excellent reputation for dispassionate assessment of benefits and costs. Yes, that is indeed a landmark of sorts.

If NICE is ever to recover its reputation, I think that it will have to start again. Next time it will have to admit openly that none of the treatments works very well in most cases. And it will have to recognise properly the disastrous cultural consequences of giving endorsement to people who, instead of engaging in scientific debate, resort to legal intimidation.

Bait and switch. Oh dear, oh dear. Just look at this. British Chiropractic Association tell their members to hide their sins from prying eyes.

Excellent round-up of the recent outburst of writing about “chiroquacktic” (Tut, tut, is there no respect?).

Dr Crippen writes “NICE recommends a cure for all known disease” [Ed some exaggeration, surely]

Jump to follow-up

It may be only post-1992 universities that run degrees in nonsense, but you can find plenty even in the highest places. Like St Bartholomew’s (founded in 1123).  That well known source of misleading medical advice, The Prince’s Foundation for Integrated Health (FiH), published last March, “Teaching integrated health at Barts and the London“. This consists of an interview with two members of staff from what is now known as the Barts and The London School of Medicine and Dentistry (SMD)..

Dr Mark Carroll BSc (Hons), PhD, FHEA is Associate Dean (Education Quality) in the Centre for Medical Education (SMD), specialising in all aspects of quality assurance in the SMD
Prof Chris Fowler BSc MA MS FRCP FRCS(Urol) FEBU is Dean of Education

They say they are dubious about alternative medicine, but rather keen on integrated medicine.  Seems odd, since the latter is really just a euphemism for the former.

After seeing the FiH posting, I wrote to Carroll and Fowler to get more information.

From Carroll 20 March

We are at an early stage in the planning process for the “Integrated Health & Wellbeing” strand in our new MBBS curriculum.  I can send you our ideas (attached).  Much will depend on whether we can make a new appointment of someone who can lead on the planning

From Fowler 25 March 2008

Our discussions with the PFIH [Prince’s Foundation for Integrated Health] have only progressed to the stage of indicating an indicative curriculum for integrated health.  We don’t have the sort of detail that you are asking for at present.  We are hoping that they will work with us to get someone to champion the development.

.  .  .

Your views would be welcome

So I sent them some views on 26 March (read them here). I also said   “I find it quite astonishing that a respectable medical school should feel it appropriate to have parts of its curriculum in the hands of the Prince of Wales.”.  It seems, though, that only one sort of view was wanted. On 27 March. Fowler wrote

“I find your insinuation unnecessary and insulting.  We have been working on a serious response to the GMC’s requirement that we teach medical students about the range of options available to patients. It is fatuous to suggest that the Prince of Wales is personally involved in any practical sense.  The Prince’s Foundation for Integrated Health is an important stakeholder and I think that it entirely reasonable both to talk to them and to seek funding to help us to develop an area that is deficient in our current provision.”

Uhuh, not a very nice response to a rather moderate letter.  Lesson 1: never trust anyone who uses the word “stakeholder”.

It does seem very odd that a medical school like Barts should turn to the Prince of Wales’ Foundation for advice on medicine.  After all, the bad advice given by the “Patients Guide” is rather well documented (see also here). If messrs Fowler and Carroll were really unaware of that, I’d argue that they aren’t doing their job properly.

It seems that Barts, like Edinburgh, has over-reacted to pressure from the General Medical Council (GMC).  Actually all that the GMC require is that

“They must be aware that many patients are interested in and choose to use a range of alternative and complementary therapies. Graduates must be aware of the existence and range of such therapies, why some patients use them, and how these might affect other types of treatment that patients are receiving.” (from Tomorrow’s Doctors, GMC)

There is nothing there about saying that they work.  Certainly medical students need to be familiar with alternative medicine, given the number of theit patients that use it. That is a job I have done myself, both at UCL and at Kings College London. I’d argue that I am marginally better qualified to assess the evidence than the Prince of Wales.

Oddly enough, the bad education in Edinburgh came also from a Professor of Medical Education and Director of Undergraduate Learning and Teaching,

The result is manifested in two ways.  Barts has a “Science in Medicine” course that has resulted in medical students being placed with homeopaths.  And it has a Special Studies Module in Ayurvedic Medicine.  Let’s take a look at them.

An Introduction to Ayurvedic Medicine

The aims of this Special Study Module are “To introduce the concepts and principles which underpin Ayurvedic medicine. To introduce Year 1 and 2 medical students to the Ayurvedic approach to patient assessment, diagnosis and treatment”, and to “Critically evaluate the evidence base for Ayurvedic treatments and yoga therapy”. Just one small snag there. There is next to no evidence base to be assessed.
Click to enlarge

The module is given by Professor Shrikala Warrier, who is Dean of MAYUR: The Ayurvedic University of Europe. That sounds quite grand.  But the web site of The Ayurvedic University of Europe is rather unusual for a university. It lists two courses but has no list of staff. Could it be that Professor Shrikala Warrier is the staff? Neither is it clear where Professor Warrier’s professorial title comes from. Her own private university perhaps?

The two courses it offers are B.Sc.(Hons) Ayurveda and B.Sc.(Hons) Yoga.  It says that the course the “BA(Hons)Ayurvedic Studies is a three year programme of study developed in collaboration with Thames Valley University in London”. That’s odd too, because there is no mention of it on the Thames Valley University web site (and TVU is not in London, it’s in Slough). Elsewhere it is stated that the “programme has been validated by MAHE, which is also the degree awarding body”. MAHE is not explained but it appears to refer to the Manipal Academy of Higher Education,  in Goa, India. That looks like a pretty good place. It does not offer degrees in Ayurveda, though there is a small Department of Ayurvedic medicine within the otherwise entirely conventional Kasturba Medical College-Manipal. Their first year physiology exam would tax our students.

Elsewhere we see the same address, 81 Wimpole Street, listed as The Manipal Ayurvedic University of Europe (a joint venture between The Manipal University and the Ayurvedic Company of Great Britain) Prof. S. Warrier, B.A.(Hons), M.A., Ph.D., MILT, Dean of Academic Planning.

If one checks Mayur Ltd at Companies House, one finds that it has two directors, Lady Sarah Morritt and Professor Shrikala Warrier. The company report shows that no accounts have been filed up to now and their 2008 accounts are overdue.

The business history of ayurveda is nothing if not tortuous.  The London Gazette (May 2008) notifies us that

AYURVEDA HOLDINGS LIMITED (chairman Lady Sarah Morritt) was passed a Special Resolution: “That the company be wound up voluntarily.”

If you email the Ayurvedic University of Europe, the reply comes not from a University address but from unififiedherbal.com. That seems to be some sort of marketing company, at the same address, 81 Wimpole Street. But efforts to find out more about it from Companies House show that UnifiedHerbal.com was dissolved on 3 October 2006.

Several of the links are broken on the web site of Ayurvedic University of Europe, but one that does work is ‘products’. That takes you to the sales pages of http://www.drwarrier.co.uk/.  That doesn’t look much like a university, but no prizes for guessing the address.  Yes, it’s 81 Wimpole Street again. They will sell you all sorts of cosmetics, though Companies House lists Dr Warrier Limited, and tells us

Last Accounts Made Up To : 31/08/2007 (DORMANT) and

Next Return Due : 26/09/2008 OVERDUE.  Their registered office is
at Harold House, Waltham Cross EN8 7AF.

From drwarrier.co.uk you can buy, for example,

Triphala

The most commonly prescribed Ayurvedic formula. Triphala is an effective blood purifier that detoxifies the liver, helps digestion and assimilation, and reduces serum cholesterol and lipid levels.

Blood purifier? Detox? Where have we seen this sort of utter gobbledygook before? Or perhaps she can sell you some

Guggul (Commiphora
mukul)

Traditionally used for obesity and overweight, and reducing and preventing accumulation of cholesterol (LDL). Its anti-inflammatory and detoxifying actions help reduce arthritic pain and swelling.

There isn’t the slightest evidence for these effects in man. Hence, no doubt, the usual weasel words. “traditionally used for . . . ”

The sales department alone casts rather a large doubt on Prof Warrier’s ability to teach medical students how to “critically evaluate the evidence base for Ayurvedic treatments”.

It does seem a bit surprising that a top flight medical school should think that this is an appropriate place to educate its students.

Medicine in Society

The syllabus at Barts includes something called Medicine and Society. Page 5 of the second year Tutor Guide mentions “complementary therapies” as part of the course. There is little hint about what that means in practice.

It turns out that the alt med placements are at the Greenwich Natural Health Centre. Nothing is too barmy for them Acupuncture. Cranial Osteopathy, Craniosacral Therapy, Herbal Therapy, Homeopathy, Hot Stone Therapy and Nutritional therapy to name but a few of the preposterous make-believe stuff that is on offer.

Medical students are having to spend their time listing to stuff like this, on ‘hot stone therapy’.

“Hot stone therapy / massage is a kind of massage that uses treated volcanic rocks such as basalt and basinite that are believed to promote relaxation as well as eliminating negative energy within the client’s body, mind and soul.”

“These stones are carefully gathered and handcrafted for various sizes, shapes and weights according to what part of the body it will be use on.”

Or this, on ‘nutritional therapy’.

“Many of us lack the basic raw materials (from food and drink) to function at our best. Intensive farming, pollution, stress, stimulants and an over-reliance on processed foods are just some of the reasons for us being deficient in vital nutrients. As a result, we may develop serious degenerative diseases like cancer or arthritis.”

“Some clients may experience reactions like headaches, skin eruptions or bad breath during the first stage of treatment. These are quite normal and are due to detoxification, which is usually followed by a sense of well-being and increased energy.”

That must be about as close as you can get to claiming you can prevent cancer by taking vitamin pills. It is wrong and it is dangerous,

Sigh. What century are we living in?

According to Barts’ second year Tutor Guide, “Placement tutors are responsible for student assessment”.

What sort of grade will the student get if they tell their tutor in homeopathy or ‘Nutritional therapy’ that they are talking nonsense?

What do the students think?

Could the Ayurvedic course be the very same course that is referred to by a second year medical student on the Unprotected Text blog?

“When I found out my friend had been attached to a “doctor” in Ayurvedic “medicine” for the year I was horrified, as was she, and the school would not allow her to change claiming that the point is not to learn the medicine but its role in the multidisciplinary healthcare team.

I don’t believe that there is such a role.

The very fact that a student is forced to put up with this as a part of their education is appalling.”

A comment left on Unprotected Text by someone writing as ‘Barts Medic’ said

“I was HORRIFIED to hear that some of my friends have their medsoc placements
at such RUBBISH places too!

last week, they were forced into a room one by room to be touched up (‘massaged’)
by the ‘doctor-person’ to HEAL them. she rubbed them all over, and CHANTED! WTF.

if i was given a CRAPPY placement like that, i wouldnt turn up either”.

And there is an excellent statement about “holistic medicine” on Unprotected Text. Better, perhaps, than you’d get from the GMC.

“Holistic medicine is in fact a world away from homeopathy, although the two are often confused predominantly by homeopaths trying to validate their branch of “medicine”. Much of what is taught in medical school is in fact, holistic, and so it should be. The importance of mentality, or spirituality in medicine should not be used to excuse homeopathy.”

“That doesn’t seem to add up” is another blog that relates the experiences of another student who has been exposed to “Enforced quackery Day 1, “, “Enforced quackery Day 2“, “Enforced quackery Days 3 and 4“. He says

“. . .by the end of day 3 the students *still* hadn’t seen any patients and that, when confronted with this fact the person in charge is reported to have said that this was because she was scared of what the students might say to the patients… Apart from this being a massive insult to the professionalism of the students, it is at least an encouraging sign that they have not been very effectively indoctrinated.”

It seems that we shall soon have some more documentary evidence.  It is truly impressive to find that Barts’ medical students are so bright and that they have the courage to speak up about it.

So there is one good thing. We have some very perspicacious medical students in London.

Pity that one can’t say the same thing of their teachers.

Follow-up

I have it on good authority that the unhappy students who were placed at the Greenwich Natural Health Centre were presented with one of the more absurd documents ever to be produced by homeopaths, “An Overview of positive homeopathy research and surveys“.

There is no need to argue about whether homeopaths cherry-pick the evidence.  The selective use of evidence is announced proudly, right there in the title.

What excuse can Bart’s have for exposing medical students to such profoundly anti-educational stuff as this?

Later there appeared on the That doesn’t seem to add up blog, Enforced
Quackery – the literature
. The unfortunate students who were pushed into a homeopathic placement were give a print out of a page from Sue Young’s homeopathic web site.  It is merely a bit of phony history that attempts to link Pasteur with homeopathy.

Sue Young, incidentally, is a homeopath who has consistently breached the Code of Ethicsof the Society of Homeopaths by claiming to treat serious diseases, though needless to say the Society did nothing about it.  She is also the person who wrote a wholly inaccurate account of the reasons why my blog left the UCL server (see alse here and here). She didn’t, needless to say, ask me, but luckily she was soon corrected on quackometer and in the Guardian.

Incidentally, the Unprotected Text blog continues to provide a fascinating student view on medical education.  Students show more sense than their teachers not only about alternative nonsense but also about other gimmicks like ‘problem based learning’.

Jump to follow-up

It’s hard enough to communicate basic ideas about how to assess evidence to adults without having the effort hindered by schools.

The teaching of quackery to 16 year-olds has been approved by a maze of quangos, none  of which will take responsibility, or justify their actions. So far I’ve located no fewer than eight of them.

[For non-UK readers, quango = Quasi-Autonomous Non-Governmental Organisation].


A lot of odd qualifications are accredited by OfQual (see here).  Consider, for example, Edexcel Level 3 BTEC Nationals in Health and Social Care (these exams are described here), Download the specifications here and check page 309.

Unit 23: Complementary Therapies for Health and Social Care
NQF Level 3: BTEC National
Guided learning hours: 60

Unit abstract

“In order to be able to take a holistic view towards medicine and health care, health and social care professionals need to understand the potential range of complementary therapies available and how they may be used in the support of conventional medicine.”

Well, Goldacre has always said that homeopathy makes the perfect vehicle for teaching how easy it is to be deceived by bad science, so what’s wrong?  But wait

“Learners will consider the benefits of complementary therapies to health and wellbeing, as well as identifying any contraindications and health and safety issues in relation to their use.”

Then later

“The holistic approach to illnesses such as cancer could be used as a focus here. For example, there could be some tutor input to introduce ideas about the role of complementary therapies in the treatment and management of cancer, this being followed up by individual or small group research by learners using both the internet and the services available locally/regionally. If available, a local homeopathic hospital, for example, would be an interesting place to visit.”

It’s true that to get a distinction, you have to “evaluate the evidence relating to the use of complementary therapies in contemporary society”, but it isn’t at all clear that this refers to evidence about whether the treatment works.

The really revealing bit comes when you get to the

“Indicative reading for learners
There are many resources available to support this unit.

Websites
www.acupuncture.org.uk British Acupuncture Council
www.bant.org.uk British Association for Nutritional Therapy
www.exeter.ac.uk/sshs/compmed Exeter University’s academic department of Complementary medicine
www.gcc-uk.org General Chiropractic Council
www.nimh.org.uk National Institute of Medical Herbalists
www.nursingtimes.net The Nursing Times
www.osteopathy.org.uk General Osteopathic Council
www.the-cma.org.uk The Complementary Medical Association”

This list is truly astonishing. Almost every one of them can be relied on to produce self-serving inaccurate information about the form of “therapy” it exists to promote. The one obvious exception is the reference to Exeter University’s academic department of complementary medicine (and the link to that one is wrong). The Nursing Times should be an exception too, but their articles about CAM are just about always written by people who are committed to it.

It is no consolation that the 2005 version was even worse.  In its classification of ‘therapies’ it said “Pharmaceutically mediated: eg herbalism, homeopathy “. Grotesque! And this is the examinng body!

The Teacher

This particular educational disaster came to my attention when I had a letter from a teacher.  She had been asked to teach this unit, and wanted to know if I could provide any resources for it.  She said that Edexcel hadn’t done so. She asked ” Do you know of any universities that teach CT’s [sic] so I could contact them about useful teaching resources?.” She seemed to think that reliable information about homeopathy could be found from a ‘university’ homeopathy teacher.  Not a good sign. It soon emerged why.
She said.

“My students are studying BTEC National Health Studies and the link is Edexcel BTEC National Complimentary [sic] studies.”

“I am a psychotherapist with an MA in Education and Psychology. I am also trained in massage and shiatsu and have plenty of personal experience of alternative therapy”

Shiatsu uh? It seems the teacher is already committed to placebo medicine.  Nevertheless I spent some time looking for some better teaching material for 16 year-old children.  There is good stuff at Planet
Science
, and in some of the pamphlets from Sense about Science, not least their latest, I’ve got nothing to lose by trying it – A guide to weighing up claims about cures and treatments.  I sent all this stuff to her, and prefaced the material by saying

“First of all, I should put my cards on the table and say that I am quite appalled by the specification of Unit 23. In particular, it has almost no emphasis at all on the one thing that you want to know about any therapy, namely does it work?  The reference list for reading consists almost entirely of organisations that are trying to sell you various sorts of quackery, There is no hint of balance; furthermore it is all quite incompatible with unit 22, which IS concerned with evidence.”

At this point the teacher the teacher came clean too, As always, anyone who disagrees with the assessment (if any) of the evidence by a true believer is unmeasured and inflammatory.

“I have found your responses very unmeasured and inflammatory and I am sorry to say that this prejudicial attitude has meant that I have not found your comments useful.”

shortly followed by

“I am not coming from a scientific background, neither is the course claiming to be scientific.”

That will teach me to spend a couple of hours trying to help a teacher.

What does Edexcel say?

I wrote to Edexcel’s science subject advisors with some questions about what was being taught. The response that I got was not from the science subject advisors but from the Head of Customer support, presumably a PR person.

From: (Bola Arabome) 12/11/2008 04.31 PM

Dear Professor Colquhoun

Thank you for email communication concerning the complementary therapies unit which is available in our BTEC National in Health and BTEC National in Health and Social Care qualifications. I have replied on behalf of Stephen Nugus, our science subject advisor, because your questions do not refer to a science qualification. I would like to answer your questions as directly as possible and then provide some background information relating to the qualifications.

The units and whole qualifications for all awarding bodies are accredited by the regulator, the Qualifications and Curriculum Authority. The resource reading list is also produced by us to help teachers and learners. The qualification as a whole is related to the National Occupational Standards for the vocational sectors of Health and Health and social care with consultation taken from the relevant sector skills councils . As you will be aware many of these complementary therapies are available in care centres and health centres under the NHS and in the private sector. The aim of BTEC qualifications is to prepare people for work in these particular sectors. Clearly a critical awareness is encouraged with reference to health and safety and regulation. There are other units, in some cases compulsory, within the qualification with a scientific approach.

‘  ‘  ‘  ‘  ‘

Stephen Harris

Head of Customer Support



Aha, so it seems that teaching people to treat sick patients is “not a science qualification”.  Just a business qualification perhaps?.  I haven’t yet managed to reach the people who make these decisions, so I persisted with the PR man. Here is part of the next letter (Edexcel’s reply in italic).

19 November

I find it quite fascinating that Edexcel regards the treatment of sick patients as not being part of science (“do not refer to a science qualification”).

Does that mean Edexcel regard the “Health” part of “Health and Social Care” as being nothing to do with science, and that it therefore doesn’t matter if Health Care is unscientific, or even actively anti-scientific?

I am sorry if my answer lacked clarity. My comment, that I had taken your enquiry on behalf of our Science Advisor because  this was not a science qualification, was intended to explain why I was replying. It was not intended as a comment on the relationship between Health and Social  Care and science. At Edexcel we use bureaucratic categories where we align our management of qualifications with officially recognised occupational sectors. Often we rely on sector bodies such as Sector Skills Councils to endorse or even approve the qualifications we offer. Those involved in production of our Science qualifications and our
Health and Social care qualifications are, as far as I can ascertain, neither anti-scientific nor non-scientific in their approach

(4) You say “The qualification as a whole is related to the National Occupational Standards for the vocational sectors of Health and Health and social care with consultation taken from the relevant sector skills councils”. Are you aware that the Skills for Health specifications for Alternative medicine were written essentially by the Prince of Wales Foundation?
When I asked them if they would be writing a competence in talking to trees, they took the question totally seriously!! (You can see the transcript of the conversation at http://dcscience.net/?p=215 ).

The qualification was approved by both ‘Skills for Health’ and ‘Skills for Care and Development’ prior to being accredited by QCA. It uses the NOS in Health and Social Care as the basis for many of the mandatory units. The ‘Complementary Therapies’ NOS were not used. This was not a requirement of a ‘Health and Social Care’ qualification.

“Are the NOS in Health and Social Care that you mention the ones listed here? http://www.ukstandards.org/Find_Occupational_Standards.aspx?NosFindID=1&ClassificationItemId=174 If so, I can see nothing there about ‘complementary therapies’. if I have missed it, I’d be very grateful if you could let me know where it is. If it is not there, I remain very puzzled about the provenance of Unit 23, since you say it is not based on Skills for Health.”



Now we are immediately at sea, struggling under a tidal wave of acronyms for endless overlapping quangos.  In this one short paragraph we have no fewer than four of them. ‘Skills for Health’, ‘Skills for Care and Development’ , ‘Quality and Curriculum Authority (QCA) and NOS.

It seems that the specification of unit 23 was written by Edexcel, but Harris (25 Nov) declines to name those responsible

“When I refer to our “Health and Social care team” I mean the mix of Edexcel Staff and the associates we employ on a contract basis as writers, examiners and  external verifiers.   The writers are generally recruited from those who are involved in teaching and assessment the subjects in schools and colleges. The editorial responsibility lies with the Edexcel Staff. I do not have access to the names of the writers and in any case would not be able to pass on this information. Specifications indicate the managers responsible for authorising publication”

“Edexcel takes full responsibility for its ethical position on this and other issues. However we can not accept responsibility for the opinions expressed in third party materials. There is a disclaimer to this effect at the beginning of the specification. ”
” You have the correct link to the Health NOS . These are the standards, which where appropriate, influence our qualifications. However in the case of Unit 23 I understand that there is no link with the Health NOS. I don’t know if the NOS cover the unit 23 content.”

So, contrary to what I was told at first, neither Skills for Health, nor NOS were involved  Or were they (see below)?

So who does take responsibility?  Aha that is secret.  And the approval by the QCA is also secret.

“I cannot provide you with copies of any correspondence between Skills for Health and  Edexcel. We regard this as confidential. “

What does the QCA say?

The strapline of the QCA is

“We are committed to building a world-class education and training framework. We develop and modernise the curriculum, assessments, examinations and qualifications.”

Referring school children to the Society of Homeopaths for advice seems to be world-class bollocks rather than world-class education.

When this matter was brought to light by Graeme Paton in the Daily Telegraph, he quoted Kathleen Tattersall, CEO of the QCA. She said

“The design of these diplomas has met Ofqual’s high standards. We will monitor them closely as they are delivered to make sure that learners get a fair deal and that standards are set appropriately.”

Just the usual vacuous bureaucratic defensive sound-bite there. So I wrote to Kathleen Tattersall  myself with some specific questions. The letter went on 2nd September 2008.  Up to today, 26 November, I had only letters saying

“Thank you for your email of 12 November addressed to Kathleen Tattersall, a response is being prepared which will be forwarded to you shortly.”

“Thank you for your email of 25th November addressed to Kathleen Tattersall. A more detailed response is being prepared which will be sent to you shortly.”

Here are some of the questions that I asked.

I wrote to Edexcel’s subject advisors about unit 23 and I was told “your questions do not refer to a science qualification”. This seems to mean that if it comes under the name “Health Care” then the care of sick patients is treated as though it were nothing to do with science, That seems to me to be both wrong and dangerous, and I should like to hear your view about that question.

Clearly the fundamental problem here is that the BTEC is intended as a vocational training for careers in alternative medicine, As a body concerned with education, surely you cannot ignore the view of 99% of scientists and doctors that almost all alternative medicine is fraud. That doesn’t mean that you can’t make a living from it, but it surely does create a dilemma for an educational organisation. What is your view of that dilemma?



Eventually, on 27th November, I get a reply (of sorts)  It came not from the Kathleen Tattersall of the QCA but from yet another regulatory body, OfQual, the office of the Qualifications and Examinations Regulator.  You’d think that they’d know the answers, but if they do they aren’t telling, [download whole letter.  It is very short.  The “more detailed response” says nothing.

Ofqual does not take a view on the detailed content of vocational qualifications as that responsibility sits with the relevant Sector Skills Council which represents employers and others involved in the sector. Ofqual accredits the specifications, submitted by sector-skilled professionals, after ensuring they meet National Occupational Standards.  Ofqual relies on the professional judgement of these sector-skilled professionals to include relevant subjects and develop and enhance the occupational standards in their profession.

The accreditation of this BTEC qualification was supported by both Skills for Health, and Skills for Care and Development, organisations which represent the emerging Sector Qualifications Strategies and comply with the relevant National Occupational Standards

Isabel Nisbet

Acting Chief Executive



So no further forward. Every time I ask a question, the buck gets passed to another quango (or two, or three). This letter, in any case, seems to contradict what Edexcel said about the involvement of Skills for Health (that’s the talking to trees outfit),

A nightmare maze of quangos

You may well be wondering what the relationship is between Ofqual and the QCA.  There is an ‘explanation’ here.

Ofqual will take over the regulatory responsibilities of the Qualifications and Curriculum Authority (QCA), with stronger powers in relation to safeguarding the standards of qualifications and assessment and an explicit remit as a market regulator. The QCA will evolve into the Qualifications and Curriculum Development Agency (QCDA): supporting Ministers with advice and undertaking certain design and delivery support functions in relation to the curriculum, qualifications, learning and development in the Early Years Foundation Stage, and National Curriculum and Early Years Foundation Stage assessments.

Notice tha QCA won’t be abolished. There will be yet another quango.

The result of all this regulatory bureaucracy seems to be worse regulation, Exactly the same thing happens with accreditiation of dodgy degrees in universities.

At one time, a proposal for something like Unit 23 would have been shown to any competent science teacher, who would have said”you must be joking” and binned it.  Now a few hundred bureaucrats tick their boxes and rubbish gets approved.

There seems to be nobody in any of these quangos with the education to realise that if you want to know the truth about homeopathy, the last person you ask is the Society of Homeopaths or the Prince of Wales.

What next?

So the mystery remains. I can’t find out who is responsible for the provenance of the appallingly anti-science Unit 23, and I can’t find out how it got approved.  Neither can I get a straight answer to the obvious question about whether it is OK to encourage vocational qualifications for jobs that are bordering on being fraudulent.

.All I can get is platitudes and bland assurances.  Everything that might be informative is clouded in secrecy.

The Freedom of Information requests are in.  Watch this space. But don’t hold your breath.

Follow-up

Here are some attempts to break through the wall of silence.

Edexcel. I sent them this request.

Freedom of Information Act

Hello

I should like to see please all documents from Edexcel and OfQual or QCA (and communications between then) that concern the formulation and approval of Unit 23 (Complementary Therapies) in the level3 BTEC (page 309 in attached document). In vew of the contentious nature of the subject matter, I believe that is is in the public interest that this information be provided

David Colquhoun

The answer was quite fast, and quite unequivocal, Buzz off.

Dear Mr Colquhoun,

Thank you of your e-mail of today’s date. I note your request for information pursuant to The Freedom of Information Act. As you may know this Act only applies to public bodies and not to the private sector. Edexcel Limited is privately owned and therefore not subject to this Act. Edexcel is therefore not obliged to provide information to you and is not prepared to give you the information you seek.

Please do not hesitate to contact me again if you have any further queries.

Kate Gregory
Director of Legal Services
Pearson Assessments & Testing
One90 High Holborn, London, WC1V 7BH
T: +44 (0)20 7190 5157 / F: +44 (0)207 190 5478
Email: kate.gregory@pearson. com



This lack of public accountability just compounds their appalling inability to distinguish education from miseducation.

International Therapy Examination Council (ITEC)

Mojo’s comment, below, draws attention to the Foundation degree in Complementary Therapies offered by Cornwall College, Camborne, Cornwall (as well as to the fact that the Royal National Lifeboat Institution has been wasting money on ‘research’ on homeopathy –write to them).

At least the courses are held on the Camborne campus of Cornwall College, not on the Duchy campus (do we detect the hand of the Quacktitioner Royal in all this nonsense?).

Cornwall College descends to a new level of barminess in its course Crystal Healing VTCT Level 3

“Who is this course for?

This course is designed to enhance the skills of the Holistic Therapist. Crystals may be used on their own in conjunction with other therapies such as Indian Head Massage, Aromatherapy and Reflexology. Due to the nature of the demands of the holistic programme this course is only suitable for students over the age of 18.”

“What will I be doing on the course?

Students will study the art of Crystal healing which is an energy based treatment where crystals and gemstones are used to channel and focus various energy frequencies.”

.VTCT stands for the Vocational Training Charitable Trust.

It is yet another organisation that runs vocational exams, and it is responsible for this particular horror

The crystals are here. I quote.

Objectives

  • the use of interpersonal skills with client
  • how to complement other therapies with crystals
  • the types and effects of different crystals
  • uses of crystals including cleansing, energising, configurations
  • concepts of auras and chakras

This is, of course, pure meaningless nonsense. Utter bollocks being offered as further education

Cornwall College has many courses run by ITEC.

The College says

“You will become a professional practitioner with the International Therapy Examination Council (ITEC), study a number of essential modules to give a vocational direction to your study that include: Homeopathy and its application,”

Who on earth, I hear you cry, are ITEC? That brings us to the seventh organisation in the maze of quangos and private companies involved in the miseducation of young people about science and medicine. It appears, like Edexcel, to be a private company though its web site is very coy about that.

After the foundation degree you can go on to “a brand new innovative BSc in Complementary Health Studies (from Sept 2009)”

The ITEC web site says

Oddly enough, there is no mention of accreditation by a University (not that that is worth much).  So a few more Freedom of Information requests are going off, in an attempt to find out why are kids are being miseducated about science and medicine.

Meanwhile you can judge the effect of all that education in physiology by one of the sample questions for ITEC Unit 4, reflexology.

The pancreas reflex:

A Extends across both feet
B Is on the right foot only
C Is on the left foot only
D Is between the toes on both feet

Uhuh, they seem to have forgotten the option ‘none of the above’.

Or how about a sample question from ITEC Unit 47 – Stone Therapy Massage

Which organ of the body is associated with the element fire?

A Heart
B Liver
C Spleen
D Pancreas


Or perhaps this?

Which incantation makes hot stones work best?

A Incarcerous
B Avada Kedavra,
C Dissendium
D Expelliarmus.

(OK I made the last one up, with help from Harry Potter, but it makes just about as much sense as the real ones).

And guess what? You can’t use the Freedom of Information Act to find out how this preposterous rubbish got into the educational system because ” ITEC is a private organisation therefore does not come under this legislation”. The ability to conduct business in secret is a side effect of the privatisation of public education is another reason why it’s a bad idea.

Ofsted

Ofsted has inspected Cornwall College. They say “We inspect and regulate to achieve excellence in the care of children and young people, and in education and skills for learners of all ages.”. I can find no mention of this nonsense in their report, so I’ve asked them.

Ofsted has admitted a spectacular failure in its inspection of child care in the London Borough of Haringey. Polly Curtis wrote in the Guardian (6 Dec 2008) “We failed over Haringey – Ofsted head”. It was the front page story. But of course Ofsted don’t take the blame, they say they were supplied with false information,

That is precisely what happens whenever a committee or quango endorses rubbish. They look only at the documents sent to them and they don’t investigate, don’t engage their brains.

In the case of these courses in utter preposterous rubbish, it seems rather likely that the ultimate source of the misinformation is the Princes’ Foundation for Integrated Health. Tha views of the Prince of Wales get passed on to the ludicrous Skills for Health and used as a criterion by all the other organisations, without a moment of critical appraisal intervening at any point.

2 December 2008 A link from James Randi has sent the hit rate for this post soaring.  Someone there left are rather nice comment.

“A quango seems to be a kind of job creation for the otherwise unemployable ‘educated ‘( degree in alternative navel contemplation) middle classes who can’t be expected to do anything useful like cleaning latines ( the only other thing they seem qualified for ). I really hate to think of my taxes paying for this codswollop.”


Jump to follow-up

During the last year, there has been a very welcome flurry of good and informative books about alternative medicine. They are all written in a style that requires little scientific background, even the one that is intended for medical students.

CAM, Cumming | Trick or Treatment | Snake Oil Science |
Testing treatments | Suckers | Healing, Hype or Harm

I’ll start with the bad one, which has not been mentioned on this blog before.

Complementary and Alternative medicine. An illustrated text.

by Allan D. Cumming, Karen R. Simpson and David Brown (and 12 others). 94 pages, Churchill Livingstone; 1 edition (8 Dec 2006).

The authors of this book sound impressive

Allan Cumming, BSc(Hons), MBChB, MD, FRCP(E), Professor of Medical Education and Director of Undergraduate Learning and Teaching, and Honorary Consultant Physician, College of Medicine and Veterinary Medicine, University of Edinburgh, Edinburgh, UK;

Karen Simpson, BA(Hons), RN, RNT, Fellow in Medical Education, College of Medicine and Veterinary Medicine

David Brown, MBChB, DRCOG, General Practitioner, The Murrayfield Medical Centre, and Honorary Clinical Tutor, University of Edinburgh

Sadly, this is a book so utterly stifled by political correctness that it ends up saying nothing useful at all. The slim volume is, I have to say, quite remarkably devoid of useful information. Partly that is a result of out-of-date and selective references (specially in the chapters written by alternative practitioners),

But the lack of information goes beyond the usual distortions and wishful thinking. I get the strong impression is that it results not so much for a strong commitment to alternative medicine (at least by Cumming) as from the fact that the first two authors are involved with medical education. It seems that they belong to that singularly barmy fringe of educationalists who hold that the teacher must not give information to s student for fear of imparting bias. Rather the student must be told how to find out the information themselves. There is just one little problem with this view. It would take about 200 years to graduate in medicine.

There is something that worries me about medical education specialists. Just look at the welcome given by Yale’s Dean of Medical Education, Richard Belitsky, to Yale’s own division of “fluid concepts of evidence”, as described at Integrative baloney @ Yale, and as featured on YouTube. There are a lot of cryptic allusions to alternative forms of evidence in Cumming’s book too, but nothing in enough detail to be useful to the reader.

What should a book about Alternative medicine tell you? My list would look something like this.

  • Why people are so keen to deceive themselves about the efficacy of a treatment
  • Why it is that are so often deceived into thinking that something works when it doesn’t
  • How to tell whether a medicine works better than placebo or not,
  • Summaries of the evidence concerning the efficacy and safety of the main types of alternative treatments.

The Cumming book contains chapters with titles like these. It asks most of the right questions, but fails to answer any of them. There is, time and time again, the usual pious talk about the importance of evidence, but then very little attempt to tell you what the evidence says. When an attempt is made to mention evidence, it is usually partial and out of date. Nowhere are you told clearly about the hazards that will be encountered when trying to find out whether a treatment works.

The usual silly reflexology diagram is reproduced in Cumming’s introductory chapter, but with no comment at all, The fact that it is obviously total baloney is carefully hidden from the reader.. What is the poor medical student meant to think when they perceive that it is totally incompatible with all the physiology they have learned? No guidance is offered.
You will look in vain for a decent account of how to do a good randomised controlled trial, though you do get a rather puerile cartoon, The chapter about evidence is written by a librarian. Since the question of evidence is crucial, this is a fatal omission.

Despite the lack of presentation of evidence that any of it works, there seems to be an assumption throughout the book that is is desirable to integrate alternative medicine into clinical practice. In Cumming’s chapter (page 6) we see

Since it would not be in the interests of patients to integrate treatments that don’t work with treatments that do work, I see only two ways to explain this attitude. Either the authors have assumed than most alternative methods work (in which case they haven’t read the evidence), or they think integration is a good idea even if the treatment doesn’t work. Neither case strikes me as good medical education.

The early chapters are merely vague and uninformative. Some of the later chapters are simply a disgrace.

Most obviously the chapter on homeopathy is highly selective and inaccurate, That is hardly surprising because it is was written by Thomas Whitmarsh, a consultant physician at Glasgow Homeopathic Hospital (one that has still survived). It has all the usual religious zeal of the homeopath. I honestly don’t know whether people like Whitmarsh are incapable of understanding what constitutes evidence, or are simply too blinded by faith to even try. Since the only other possibility is that they are dishonest, I suppose it must be one of the former.

The chapter on “Nutritional therapy” is also written by a convert and is equally misleading piece of special pleading.

The same is true of the chapter on Prayer and Faith Healing. This chapter reproduces the header of the Cochrane Review on “Intercessory prayer for the alleviation of Ill Health”, but then proceeds to ignore entirely its conclusion “Most of the studies show no real differences”).

If you want to know about alternative medicine, don’t buy this book. Although this book was written for medical students, you will learn a great deal more from any of the following books, all of which were written for the general public.


Trick or Treatment

by Simon Singh and Edzard Ernst, Bantam Press, 2008

Simon Singh is the author of many well-known science books, like Fermat’s Last Theorem. Edzard Ernst is the UK’s first professor of complementary and alternative medicine.

Ernst, unlike Cumming et. al is a real expert in alternative medicine. He practised it at an early stage in his career and has now devoted all his efforts to careful, fair and honest assessment of the evidence. That is what this book is about. It is a very good account of the subject and it should be read by everyone, and certainly by every medical student.

Singh and Ernst follow the sensible pattern laid out above, The first chapter goes in detail into how you distinguish truth from fiction (a little detail often forgotten in this area).

The authors argue, very convincingly, that the development of medicine during the 19th and 20th century depended very clearly on the acceptance of evidence not anecdote. There is a fascinating history of clinical trials, from James Lind (lemons and scurvy), John Snow and the Broad Street pump, Florence Nightingale’s contribution not just to hygiene, but also to the statistical analysis that was needed to demonstrate the strength of her conclusions (she became the first female member of the Royal Statistical Society, and had studied under Cayley and Sylvester, pioneers of matrix algebra).

There are detailed assessments of the evidence for acupuncture, homeopathy, chiropractic and herbalism, and shorter synopses for dozens of others. The assessments are fair, even generous in marginal cases.

Acupunture. Like the other good books (but not Cumming’s), it is pointed out that acupuncture in the West is not so much the product of ancient wisdom (which is usually wrong anyway), but rather a product of Chinese nationalist propaganda engineered by Mao Tse-tung after 1949. It spread to the West after Nixon’s visit Their fabricated demonstrations of open heart surgery under acupuncture have been known since the 70s but quite recently they managed again to deceive the BBC It was Singh who revealed the deception. The conclusion is ” . . . this chapter demonstrates that acupuncture is very likely to be acting as nothing more than a placebo . . . ”

Homeopathy. “hundreds of trials have failed to deliver significant or convincing evidence to support the use of homeopathy for the treatment of any particular ailment. On the contrary, it would be to say that there is a mountain of evidence to suggest that homeopathic remedies simply do not work”.

Chiropractic. Like the other good books (but not Cumming’s) there is a good account of the origins of chiropractic (see, especially, Suckers). D.D. Palmer, grocer, spiritual healer, magnetic therapist and fairground quack, finally found a way to get rich by removing entirely imaginary ‘subluxations’. They point out the dangers of chiropractic (the subject of court action), and they point out that physiotherapy is just as effective and safer.

Herbalism. There is a useful table that summarises the evidence. They conclude that a few work and most don’t Unlike homeopathy, there is nothing absurd about herbalism, but the evidence that most of them do any good is very thin indeed.

“We argue that it is now the time for the tricks to stop, and for the real treatments to take priority. In the name of honesty, progress and good healthcare, we call for scientific standards, evaluation and regulation to be applied to all types of medicine, so that patients can be confident that they are receiving treatments that demonstrably generate more harm than good.”

Snake Oil Science, The Truth about Complementary and Alternative Medicine.

R. Barker Bausell, Oxford University Press, 2007

Another wonderful book from someone who has been involved himself in acupuncture research, Bausell is a statistician and experimental designer who was Research Director of a Complementary and Alternative Medicine Specialised Research Center at the University of Maryland.

This book gives a superb account of how you find out the truth about medicines, and of how easy it is to be deceived about their efficacy.

I can’t do better than quote the review by Robert Park of the American Physical Society (his own book, Voodoo Science, is also excellent)

“Hang up your lantern, Diogenes, an honest man has been found. Barker Bausell, a biostatistician, has stepped out of the shadows to give us an insider’s look at how clinical evidence is manipulated to package and market the placebo effect. Labeled as ‘Complementary and Alternative Medicine’, the placebo effect is being sold, not just to a gullible public, but to an increasing number of health professionals as well. Bausell knows every trick and explains each one in clear language”

Bausell’s conclusion is stronger than that of Singh and Ernst.

“There is no compelling, credible scientific evidence to suggest that any CAM therapy benefits any medical condition or reduces any medical symptom (pain or otherwise) better than a placebo”.

Here are two quotations from Bausell that I love.

[Page 22] ” seriously doubt, however, that there is a traditional Chinese medicine practitioner anywhere who ever stopped performing acupuncture on an afflicted body in the presence of similarly definitive negative evidence. CAM therapists simply do not value (and most cases, in my experience, do not understand) the scientific process”

And even better,

[Page39] “But why should nonscientists care one iota about something as esoteric as causal inference? I believe that the answer to this question is because the making of causal inferences is part of our job description as Homo Sapiens.”


Testing Treatments: Better Research for Better Healthcare

by Imogen Evans, Hazel Thornton, Iain Chalmers, British Library, 15 May 2006

You don’t even need to pay for this excellent book (but buy it anyway, eg from Amazon). If you can’t afford, £15 then download it from the James Lind Library.

This book is a unlike all the others, because it is barely mentions alternative medicine. What it does, and does very well, is to describe he harm that can be done to patients when they are treated on the basis of guesswork or ideology, rather than on the basis of proper tests. This, of course, is true whether or not the treatment is labelled ‘alternative’.

It is worth noting that one of the authors of this book is someone who has devoted much of his life to the honest assessement of evidence, Sir Iain Chalmers, one of the founders of the Cochrane Collaboration , and Editor of the James Lind Library .

A central theme is that randomised double blind trial are essentially the only way to be sure you have the right answer. One of the examples that the authors use to illustrate this is Hormone Replacement Therapy (HRT). For over 20 years, women were told that HRT would reduce their risk of heart attacks and strokes. But when, eventually, proper randomised trials were done, it was found that precisely the opposite was true. The lives of many women were cut short because the RCT had not been done,

The reason why the observational studies gave the wrong answer is pretty obvious. HRT was used predominantly by the wealthier and better-educated women. Income is just about the best predictor of longevity. The samples were biassed, and when a proper RCT was done it was revealed that the people who used HRT voluntarily lived longer despite the HRT, not because of it. It is worth remembering that there are very few RCTs that test the effects of diet. And diet differs a lot between rich and poor people. That, no doubt, is why there are so many conflicting recommendations about diet. And that is why “nutritional therapy” is little more than quackery. Sadly, the media just love crap epidemiology. One of the best discussions of this topics was in Radio 4 Programme. “The Rise of the Lifestyle Nutritionists“, by Ben Goldacre.

One of the big problems in all assessment is the influence of money, in other words corruption, The alternative industry is entirely corrupt of course, but the pharmaceutical industry has been increasingly bad. Testing Treatments reproduces this trenchant comment.


Suckers. How Alternative Medicine Makes Fools of Us All

Rose Shapiro, Random House, London 2008

I love this book. It is well-researched, feisty and a thoroughly good read.

It was put well in the review by George Monbiot.

“A fascinating and excoriating book; witty, shocking and utterly convincing”

The chapters on osteopathy and chiropractic are particularly fascinating.

This passage describes the founder of the chiropractic religion.

“By the 1890s Palmer had established a magnetic healing practice in Davenport, Iowa, and was styling himself ‘doctor’. Not everyone was convinced as a piece about him in an 1894 edition of the local paper, the Davenport Leader, shows.”

A crank on magnetism has a crazy notion hat he can cure the sick and crippled with his magnetic hands. His victims are the eak-minded, ignorant and superstitious, those foolish people who have been sick for years and have become tired of the regular physician and want health by the short-cut method . . . he has certainly profited by the ignorance of his victims . . . His increase in business shows what can be done in Davenport, even by a quack”

Over 100 years later, it seems that the “weak-minded, ignorant and superstitious” include the UK’s Department of Health, who have given these quacks a similar status to the General Medical Council.

The intellectual standards of a 19th Century mid-western provincial newspaper leader writer are rather better than the intellectual standards of the Department of Health, and of several university vice-chancellors in 2007.


Healing Hype or Harm

Edited by Imprint Academic (1 Jun 2008)
Download the contents page

My own chapter in this compilation of essays, “Alternative medicine in UK Universities” is an extended version of what was published in Nature last year (I don’t use the term CAM because I don’t believe anything can be labelled ‘complementary’ until it has been shown to work). Download a copy if the corrected proof of this chapter (pdf).

Perhaps the best two chapters, though, are “CAM and Politics” by Rose and Ernst, and “CAM in Court” by John Garrow.

CAM and politics gives us some horrifiying examples of the total ignorance of almost all politicians and civil servants about the scientific method (and their refusal to listen to anyone who does understand it).

CAM in Court has some fascinating examples of prosecutions for defrauding the public. Recent changes in the law mean we may be seeing a lot more of these soon. Rational argument doesn’t work well very well with irrational people. But a few homeopaths in jail for killing people with malaria would probably be rather effective.

Follow-up

Healing, Hype or Harm has had some nice reviews,  That isn’t so surprising from the excellent Harriet Hall at Science-Based Medicine. The introduction to my chapter was a fable about the replacemment of the Department of Physics and Astronomy by the new Department of Alternative Physics and Astrology. It was an unashamedly based on Laurie Taylor’e University of Poppleton column. Hall refers to it as “Crislip-style”, a new term to me. I guess the incomparable Laurie Taylor is not well-known in the USA, Luckily Hall gives a link to Mark Crislip’s lovely article, Alternative Flight,

“Americans want choice. Americans are increasingly using alternative aviation. A recent government study suggests that 75% of Americans have attempted some form of alternative flight, which includes everything from ultralights to falling, tripping and use of bungee cords.”

“Current airplane design is based upon a white male Western European model of what powered flight should look like. Long metal tubes with wings are a phallic design that insults the sensibilities of women, who have an alternative, more natural, emotional, way of understanding airplane design. In the one size fits all design of allopathic airlines, alternative designs are ignored and airplane design utilizing the ideas and esthetics of indigenous peoples and ancient flying traditions are derided as primitive and unscientific, despite centuries of successful use.”

Metapsychology Online Reviews doesn’t sound like a promising title for a good review of Healing, Hype or Harm, but in fact their review by Kevin Purday is very sympathetic. I like the ending.

“One may not agree with everything that is written in this book but it is wonderful that academic honesty is still alive and well.”


Jump to follow up

A report has appeared on Regulation of Practitioners of Acupuncture, Herbal Medicine, Traditional Chinese Medicine. The report is written by people all of whom have vested interests in spreading quackery. It shows an execrable ability to assess evidence, and it advocates degrees in antiscience It would fail any examination. Sorry, Prof Pittilo, but it’s gamma minus.[Download the report]

Alice Miles put it well in The Times, today.

“This week came the publication of the “Report to Ministers from the Department of Health Steering Group on the Statutory Regulation of Practitioners of Acupuncture, Herbal Medicine, Traditional Chinese Medicine and Other Traditional Medicine Systems Practised in the UK”. Otherwise known as twaddle.” . . .

“Regulate the practitioners – for safety, note, not for efficacy, as that is impossible to prove – and you give them official recognition. From recognition it is but a short hop to demand and then prescription: packet of Prozac, bit of yoga and a bag of dodgy herbs for you, sir.” . . .

“The Government responded on Monday – with a three-month consultation. So join in. Write to the Health Minister Ben Bradshaw at Richmond House, 79 Whitehall, SW1A 2NS. Write, on behalf of the NHS: “What I want for my 60th birthday is… the chance to provide medical, dental, and nursing care to all. And absolutely nothing else.”

Judging by Ben Bradshaw’s speech to the Prince’s Foundation, there may be a problem in conveying to him the evidence, but one can and must try.

Why is it that a health joutnalist can do so much better than a university head? Yes, the chair of the steering group is Professor R. Michael Pittilo BSc PhD CBiol FIBiol FIBMS FRSH FLS FRSA, Principal and Vice-Chancellor of The Robert Gordon University, Aberdeen. Despite all those impressive-lookin initials after his name, I believe that this is a very bad report.

Here is something about Prof Pittilo from his university’s web site (the emphasis is mine).

Professor Michael Pittilo joined The Robert Gordon University, Aberdeen, as Principal and Vice-Chancellor on 5th September, 2005.

After postdoctoral research on arterial disease at the University of London, he was appointed to Kingston University where he became Head of Life Sciences. In 1995 he became Foundation Dean of the Faculty of Health and Social Care Sciences at Kingston University and St George’s Medical School (University of London). He was appointed Pro Vice Chancellor at the University of Hertfordshire in 2001.

Professor Pittilo has held a number of additional roles, including chairing Department of Health working groups, and as a trustee for the Prince of Wales’s Foundation for Integrated Health.

Notice that Prof Pittilo is a Trustee of the Prince’s Foundation for Integrated Health, source of some of the least reliable information about alternative medicine to be found anywhere.

This steering group is, as so often, a nest of vested interests. It does not seem to have on it any regular medical or clinical scientist whatsoever. Why not? They just might produce some embarrassing facts perhaps? Like most government committees its members seem to have been chosen to produce the desired outcome.

For a start, the university run by Prof Pittilo, Robert Gordon’s University, is itself involved in a few antiscientific courses. Since his report recommends that degrees in quackery should become mandatory, I expect he’d welcome the chance to run more. Amazingly, Robert Gordon’s University runs an Introduction to Homeopathy, just about the daftest of all the common sorts of magic medicine.

Most of the the members of the steering group represent vested interests, though strangely this is not made clear in the list of members. An earlier report, in 2006, from the steering group was more open about this. Twelve of the members of the group represent Herbal Medicine, Chinese Medicine and Acupuncture (four from each). Most of the rest are lay members or bureaucrats. With membership like that it is, I suppose, not surprising that the assessment of evidence is, to put it kindly, grossly distorted and woefully inadequate.

The report starts badly by failing to mention that the House of Lords report (2000), and the government’s response to it, set the following priorities. Both state clearly

“… we recommend that three important questions should be addressed in the following order . .

  • (1) does the treatment offer therapeutic benefits greater than placebo?
  • (2) is the treatment safe?
  • (3) how does it compare, in medical outcome and cost-effectiveness, with other forms of treatment?

The word ‘placebo’ does not occur a single time in the main report (and only twice in the text of the seven appendices). But they do say (page 11):

“We recommend that public funding from the NHS should be used to fund CAM therapies where there is evidence of efficacy, safety and quality assurance.”

The evidence

The problem is that the assessment of the evidence for efficacy in the report is pathetically poor. The report, sad to say, consists essentially of 161 pages of special pleading by the alternative medicine industry, served up with the usual large dose of HR gobbledygook.

There is really no excuse for this utterly incompetent assessment. There have been plenty of books this year alone that make excellent summaries of the evidence, mostly written for the lay public. They should, therefore, be understandable by any university vice-chancellor (president). The one benefit of the upsurge in public interest in magic medicine is that there are now quite a lot of good clinical trials, and when the trials are done properly, they mostly confirm what we thought before: in most cases the effects are no more than placebo.

Here is one example. Annexe1 concerns “Developing Research and Providing an Evidence Base for Acupuncture and Herbal/Traditional Medicine Treatment”. The wording of the title itself suggests, rightly, that this evidence base does not exist, in which case why on earth are we talking about them as “professions”? The discussion of the evidence in Annexe 1 is nothing if not partial. But what do you expect if you ask herbalists to assess herbal medicine? An honest assessment would put them out of business. The eternal mantra of the alternative industry appears as usual, “Absence of evidence is, of course, not evidence of absence”. True of course, but utterly irrelevant. Annexe 1 says

“Acupuncture is a complex intervention and lack of a suitable placebo control has hindered efforts to evaluate efficacy”

This is simply untrue, In recent years enormous efforts have been put into devising controls for assessment of acupuncture, but they are entirely ignored here. One thing that has been established quite clearly is that it makes no difference where you put the needles, so all the talk of Qi and meridians is obvious mumbo-jumbo.

Have the authors of Annexe 1, and Professor Pittilo, not read the relevant studies? Two books this year have dealt with the question of evidence with great care. They are both by people who have been involved personally with acupuncture research, Prof Edzard Ernst and Dr Barker Bausell. Edzard Ernst is the UK’s first Professor of Complementary Medicine. Barker Bausell was research director of an NIH-funded Complementary and Alternative Medicine Specialized Research Center at the University of Maryland.

Singh and Ernst discuss thoroughly the question of controls and assess all the evidence carefully. Their conclusions include the following.

  • The traditional principles of acupuncture are deeply flawed, as there is no evidence at all to demonstrate the existence of Ch’i [Qi] or meridians.
  • By focussing on the increasing number of high-quality research papers, reliable conclusions from systematic reviews make it clear that acupuncture does not work for a whole range of conditions, except as a placebo.
  • In short, the evidence is neither consistent nor convincing. It is borderline.

Barker Bausell was himself involved in designing and analysing trialsof acupuncture. His conclusions are even less positive.

“There is no compelling, credible scientific evidence to suggest that any CAM therapy benefits any medical condition or reduces any medical symptom (pain or otherwise) better than a placebo”.

These are serious authors with direct experience in CAM research, which is more than can be said of anyone on the steering group. Why are their conclusions ignored entirely? That is sheer incompetence.

Degrees in anti-science

One conclusion of the report is that

“The threshold entry route to the register will normally be through a Bachelor degree with Honours”

This is utter nonsense. It is quite obvious surely that you can’t award honours degrees until after you have the evidence. You can read on page 55 of the report

3a: Registrant acupuncturists must:

understand the following aspects and concepts for traditional East-Asian acupuncture:

– yin/yang, /5 elements/phases, eight principles, cyclical rhythms, qi ,blood and body fluids, different levels of qi, pathogenic factors, 12 zang fu and 6 extraordinary fu, jing luo/ meridians, the major acupuncture points, East-Asian medicine disease categorisation, the three burners, the 4 stages/levels and 6 divisions

– causes of disharmony/disease causation

– the four traditional diagnostic methods: questioning, palpation, listening and observing”

This is utter baloney. Anyone who advocates giving honours degrees in such nonsense deserves to be fired for bringing his university into disrepute (and, in the process, bringing all universities and science itself into disrepute).

That includes also degrees that teach that “amethysts emit high yin energy“.

So what should be done?

If making peole do degrees in mumbo-jumbo is not the answer, what is? Clearly it would be far too draconian to try to ban quackery (and it would only increase its popularity anyway).

The answer seems to me to be quite simple. All that needs to done is to enforce existing laws. It is already illegal to sell contaminated and poisonous goods to the public. It is already illegal to make fraudulent advertisemants and to sell goods that are not as described on the label.

The only problem is that the agencies that enforce these rules are toothless and that there are a lot of loopholes and exceptions that work in favour of quackery. I have tried myself to complain about mislabelling of homeopathic pills to the Office of Fair Trading on the grounds that are labelled Arnica 30C but contain no Arnica. They solemnly bought a bottle and sent it to an analyst and of course they found no arnica, But nothing happened, because an exception to the usual law applies to homeopathic pills.

The Advertising Standards Authority is good as far as it goes. They quickly told Boots Pharmacies to withdraw advertisements that claimed CoQ10 “increased vitality”. But they can exact no penalties and they can’t deal with lies that are told to you orally, or with anything at all on the web.

The Health Professions Council (HPC) says that one of the criteria for registering new professions is aspirant groups must “Practise based on evidence of efficacy”. If that were actually applied, none of this process would occur anyway. No doubt the HPC will fail to apply its own criteria. On past form, it can be expected to adopt a “fluid concept of evidence“.,

One more thing, New European legislation was described recently in the BMJ

“Consumers in the United Kingdom are to receive stronger legal safeguards against products that claim, without any identifiable scientific evidence, to provide physical and mental health benefits such as tackling obesity or depression.”

“The scope of the legislation is deliberately wide and is the biggest shake up in consumer law for decades. It targets any unfair selling to consumers by any business.”

Politicians seem to be immune to rational argument when it comes to quackery. But a few legal actions under these laws could bring the house of cards tumbling so fast that this gamma-minus report would become rapidly irrelevant. There will be no shortage of people to bring the actions. I can’t wait.

Follow up

Dominic Lawson, 24 June 2008. An excellent column appeared today in the Independent. Dominic Lawson writes about the Pittilo report: “So now we will have degrees in quackery. What, really, is the difference between acupuncture and psychic surgery?“. The reference to that well known conjuring trick, “psychic surgery” as a “profession”, revealed here, causes Lawson to say

“It makes it clear that the lunatics have taken over the asylum. For a start, how could Philip Hunt, previously director of the National Association of Health Authorities and Trusts, possibly have thought that “psychic healing” constituted a “profession” – let alone one which would “develop its own system of voluntary self-regulation?”

“One can see how this might fit in with the Government’s “never mind the quality, feel the width” approach to university education. One can also see how established practitioners of such therapies might see this as a future source of income – how pleasant it might be to become Visiting Professor of Vibrational Medicine at the University of Westminster.

Thus garlanded with the laurels of academic pseudo-science, the newly professionalised practitioners of “alternative medicine” can look down on such riff-raff as the “psychic surgeons”

Once again I have to ask, how is it that we have to rely on journalists to prevent vice-chancellors eroding academic standards; indeed eroding simple common sense? I guess it is just another sign of the delusional thinking engendered by the culture of managerialism that grips universities.

Last year, Nature published a pretty forthright condemnation of the award of Bachelor of Science degrees in subjects that are not science: in fact positively anti-science. This topic has come up again in Times Higher Education (24 April 2008).

A league table shows that the largest number of anti-science courses is run by the University of Westminster [download paper version].



Vice chancellors have consistently refused to answer letters, from me, from the Times Higher Education or from the BBC, asking them to defend their practices.

The vice chancellors union, Universities UK, has simply refused to consider this very basic threat to academic standards.

It is particularly amazing that vice-chancellors continue to support courses in homeopathy when they have been condemned by no less a person than the head honcho of homeopathy in the UK, Dr Peter Fisher. He is clinical director of the Royal London Homeopathic Hospital and Homeopathic Physician to the Queen. Peter Fisher and I were interviewed on BBC London News after publication of the Nature article. At the end, Fisher was asked by the presenter, Riz Lateef, about whether homeopathy was a suitable subject for a science degree.[Watch the movie]

Riz Lateef (presenter): “Dr Fisher, could you ever see it
[homeopathy] as a science degree in the future?


Dr Peter Fisher:
“I would hope so. I wouldn’t deny that a lot of scientific research needs to be done, and I would hope that in the future it would have a scientific basis. I have to say that at the moment that basis isn’t comprehensive. To that extent I would agree with Professor Colquhoun.”


The one exception was a response, of sorts, that I got from Westminster University.

I can interpret this lack of response only as a sign of guilt on the part of the vice chancellors of the 16 or so universities who teach this stuff. That interpretation is reinforced by the refusal of two of them to release their teaching materials, despite requests under the Freedom of Information Act 2000. Both the University of Central Lancashire and the University of Westminster have turned down appeals, and refused to hand over anything. The former case has been with the Information Commissioner for some time now, and if the ruling goes as a hope, the taxpayer may soon be able to see how their money is being spent.

But the wonderful thing about the electronic age is that it has become really quite difficult to keep secrets. Last year I managed to find an exam paper set by the University of Westminster in Homeopathic Materia Medica, and a question from that paper has already appeared in Nature.

I recently acquired copies of a course handbook. and of the powerpoint slides used for the lecture on ‘Vibrational Medicine’ by the University of Westminster. This appears to be from a course in Complementary Therapies, part of “Health Sciences: Complementary Therapies BSc Honours”, according to Westminster’s web site. A lot of people have access to this first year course material, so Westminster needn’t bother trying to guess how I got hold of this interesting material

In the public interest, here are a few quotations. Taxpayers should know how their money is being spent.

According to the handbook

“Complementary Therapies is a core module for the Therapeutic Bodywork, Herbal Medicine, Homœopathy, Nutritional Therapy and Complementary Therapies courses. Therefore all students of these degree courses are required to take this module.”

The University of Central Lancashire also has “Vitalistic Medicine” as part of its BSc Homeopathy (but, like Westminster, has some excellent people too).

There is a rather good Wikipedia entry on Vitalism, a topic that is now largely the preserve of cranks.

The handbook is wonderful. The word ‘evidence’. in the context of ‘does it work?’, does not occur a single time. There is plenty of the usual edu-bollocks jargon that is so beloved by bureaucrats, but not the slightest hint of critical thinking about assessment of the ‘therapies’.

The course seems to be a romp through almost every form of battiness known to humankind. Not just homeopathy, traditional Chinese medicine and nutritional therapy, but also dowsing, crystal healing and other forms of advanced delusional thinking. Before somebody grumbles, let me emphasise that ‘nutrition’ is to be distinguished from ‘nutritional therapy’: the latter involves imaginative claims that buying expensive supplements can prevent or cure almost anything. There’s a lot more about that here, and here.

Here are just 5 days from the timetable.

9am-1.00pm : Homœopathy (group work and video)
9am-1.00pm : Traditional Chinese Medicine
9am-10.45pm : BODYWORK THERAPIES
11.15-1.00pm : Nutritional Therapy
9am-1.00pm : Vibrational Medicine/Energy Concepts (L&P)



All this can be yours -at a cost.
Full-time UK/EU fee – £3,145
Full-time Overseas fee – £9,450

The slides for the last of these lectures show some of the most glorious examples of the abuse of sciencey-sounding words that I’ve seen in a while.


Sigh. All this is sheer imagination. It is ancient vitalism dressed up pretentiously in sciencey words.Then a bit later we come to the general theory -“energy concepts”.


More plausible-sounding, but utterly meaningless words about vibrations. And then on to old superstitions about dowsing with rods and pendulums.

.

Not a single word of scepticism appears about any of this mumbo jumbo. Can it get worse? Yes it can. CRYSTAL HEALING comes next.


Are you having difficulty in understanding what all these words mean? I certainly hope so, because they have no meaning to understand. Don’t worry too much though, There are some helpful diagrams.


Aura photographs? They are just fairground conjuring tricks. Well, that is what you thought. But here we see them presented, apparently in all seriousness, as part of a vocational bachelor of science degree in a UK
university.Never mind, it is all assessed properly, with all the right box-ticking jargon. The course handbook says

Learning Outcomes

On successful completion of this module you will be able to:

• describe the theoretical basis and classification of a range of
complementary therapies



What theoretical basis? There isn’t any theoretical basis, just a meaningless jumble of words.


You just couldn’t make it up.


Westminster University is not all like this

This post is not intended as an attack on the University of Westminster as a whole. Last year I had an invitation from their biomedical people to give a talk there. They asked for a talk on “What is is the evidence for Alternative Medicine?”. But then I got an email from them saying

“I was surprised to be sat on heavily on return from said trip by the VC, Provosts and Deans (including Peter Davies the leader of the Alt Med School !) once news of your talk leaked out. Could you give a talk on your research instead- yep I know its pusillanimous of me and yep I know unis stand for freedom of speech and yep I know that fellow members of staff suggested you come and others were keen to listen to your views on quackery.”

So on November 2nd 2007 I gave a seminar about single ion channel work (our new ideas about partial agonists). Of course all the excellent staff whom I met agreed with me about the embarrassment that having degrees in homeopathy etc. The fault lies not with their academic staff, but with their administration. Freedom of speech does not seem to be high on their agenda.

Postscript I recently learned that when Times Higher Education asked Westminster about my seminar, they were given the following statement.

“Prof David Colquhoun was invited to take part in a research seminar series organised by the University’s School of Biosciences last year. As part of this series, on Friday 2 November 2007, he gave a talk on the agreed topic of “Single ion Channel studies suggest a new mechanism for partial agonism” – his area of research.”

Perhaps I am naive, but it truly shocks me that a university can issue such a dishonest account of what happened.

Jump to follow up

This is a fuller version, with links, of the comment piece published in Times Higher Education on 10 April 2008. Download newspaper version here.

If you still have any doubt about the problems of directed research, look at the trenchant editorial in Nature (3 April, 2008. Look also at the editorial in Science by Bruce Alberts. The UK’s establishment is busy pushing an agenda that is already fading in the USA.

Since this went to press, more sense about “Brain Gym” has appeared. First Jeremy Paxman had a good go on Newsnight. Skeptobot has posted links to the videos of the broadcast, which have now appeared on YouTube.

Then, in the Education Guardian, Charlie Brooker started his article about “Brain Gym” thus

“Man the lifeboats. The idiots are winning. Last week I watched, open-mouthed,
a Newsnight piece on the spread of “Brain Gym” in British schools “

Dr Aust’s cogent comments are at “Brain Gym” loses its trousers.

The Times Higher’s subeditor removed my snappy title and substituted this.


So here it is.



“HR is like many parts of modern businesses: a simple expense, and a burden on the backs of the productive workers”, “They don’t sell or produce: they consume. They are the amorphous support services” .

So wrote Luke Johnson recently in the Financial Times. He went on, “Training advisers are employed to distract everyone from doing their job with pointless courses”. Luke Johnson is no woolly-minded professor. He is in the Times’ Power 100 list, he organised the acquisition of PizzaExpress before he turned 30 and he now runs Channel 4 TV.

Why is it that Human Resources (you know, the folks we used to call Personnel) have acquired such a bad public image? It is not only in universities that this has happened. It seems to be universal, and worldwide. Well here are a few reasons.

Like most groups of people, HR is intent on expanding its power and status. That is precisely why they changed their name from Personnel to HR. As Personnel Managers they were seen as a service, and even, heaven forbid, on the side of the employees. As Human Resources they become part of the senior management team, and see themselves not as providing a service, but as managing people. My concern is the effect that change is having on science, but it seems that the effects on pizza sales are not greatly different.

The problem with having HR people (or lawyers, or any other non-scientists) managing science is simple. They have no idea how it works. They seem to think that every activity
can be run as though it was Wal-Mart That idea is old-fashioned even in management circles. Good employers have hit on the bright idea that people work best when they are not constantly harassed and when they feel that they are assessed fairly. If the best people don’t feel that, they just leave at the first opportunity. That is why the culture of managerialism and audit. though rampant, will do harm in the end to any university that embraces it.

As it happens, there was a good example this week of the damage that can be inflicted on intellectual standards by the HR mentality. As a research assistant, I was sent the Human Resources Division Staff Development and Training booklet. Some of the courses they run are quite reasonable. Others amount to little more than the promotion of quackery. Here are three examples. We are offered a courses in “Self-hypnosis”, in “Innovations for Researchers” and in “Communication and Learning: Recent Theories and Methodologies”. What’s wrong with them?

“Self-hypnosis” seems to be nothing more than a pretentious word for relaxation. The person who is teaching researchers to innovate left science straight after his PhD and then did courses in “neurolinguistic programming” and life-coaching (the Carole Caplin of academia perhaps?). How that qualifies him to teach scientists to be innovative in research may not be obvious.

The third course teaches, among other things, the “core principles” of neurolinguistic programming, the Sedona method (“Your key to lasting happiness, success, peace and well-being”), and, wait for it, Brain Gym. This booklet arrived within a day or two of Ben
Goldacre’s spectacular demolition of Brain Gym “Nonsense dressed up as neuroscience”

“Brain Gym is a set of perfectly good fun exercise break ideas for kids, which costs a packet and comes attached to a bizarre and entirely bogus pseudoscientific explanatory framework”

“This ridiculousness comes at very great cost, paid for by you, the taxpayer, in thousands of state schools. It is peddled directly to your children by their credulous and apparently moronic teachers”

And now, it seems, peddled to your researchers by your credulous and
moronic HR department.

Neurolinguistic programming is an equally discredited form of psycho-babble, the dubious status of which was highlighted in a Beyerstein’s 1995 review, from Simon Fraser University.

“ Pop-psychology. The human potential movement and the fringe areas of psychotherapy also harbor a number of other scientifically questionable panaceas. Among these are Scientology, Neurolinguistic Programming, Re-birthing and Primal Scream Therapy which have never provided a scientifically acceptable rationale or evidence to support their therapeutic claims.”

The intellectual standards for many of the training courses that are inflicted on young researchers seem to be roughly on a par with the self-help pages of a downmarket women’s magazine. It is the Norman Vincent Peale approach to education. Uhuh, sorry, not education, but training. Michael O’Donnell defined Education as “Elitist activity. Cost ineffective. Unpopular with Grey Suits . Now largely replaced by Training .”

In the UK most good universities have stayed fairly free of quackery (the exceptions being the sixteen post-1992 universities that give BSc degrees in things like homeopathy). But now it is creeping in though the back door of credulous HR departments. Admittedly UCL Hospitals Trust recently advertised for spiritual healers, but that is the NHS not a university. The job specification form for spiritual healers was, it’s true, a pretty good example of the HR box-ticking mentality. You are in as long as you could tick the box to say that you have a “Full National Federation of Spiritual Healer certificate. or a full Reiki Master qualification, and two years post certificate experience”. To the HR mentality, it doesn’t matter a damn if you have a certificate in balderdash, as long as you have the piece of paper. How would they know the difference?

A lot of the pressure for this sort of nonsense comes, sadly, from a government that is obsessed with measuring the unmeasurable. Again, real management people have already worked this out. The management editor of the Guardian, said

“What happens when bad measures drive out good is strikingly described in an article in the current Economic Journal. Investigating the effects of competition in the NHS, Carol Propper and her colleagues made an extraordinary discovery. Under competition, hospitals improved their patient waiting times. At the same time, the death-rate e emergency heart-attack admissions substantially increased.”

Two new government initiatives provide beautiful examples of the HR mentality in action, They are Skills for Health, and the recently-created Complementary and Natural Healthcare Council.(already dubbed OfQuack).

The purpose of the Natural Healthcare Council .seems to be to implement a box-ticking exercise that will have the effect of giving a government stamp of approval to treatments that don’t work. Polly Toynbee summed it up when she wrote about “ Quackery
and superstition – available soon on the NHS
“ . The advertisement for its CEO has already appeared, It says that main function of the new body will be to enhance public protection and confidence in the use of complementary therapists. Shouldn’t it be decreasing confidence in quacks, not increasing it? But, disgracefully, they will pay no attention at all to whether the treatments work. And the advertisement refers you to
the Prince of Wales’ Foundation for Integrated Health for more information (hang on, aren’t we supposed to have a constitutional monarchy?).

Skills for Health, or rather that unofficial branch of government, the Prince of Wales’ Foundation, had been busy making ‘competences’ for distant healing, with a helpful bulletted list.

“This workforce competence is applicable to:

  • healing in the presence of the client
  • distant healing in contact with the client
  • distant healing not in contact with the client”

And they have done the same for homeopathy and its kindred delusions. The one thing they never consider is whether they are writing ‘competences’ in talking gobbledygook. When I phoned them to try to find out who was writing this stuff (they wouldn’t say), I made a passing joke about writing competences in talking to trees. The answer came back, in all seriousness,

“You’d have to talk to LANTRA, the land-based organisation for that”,
“LANTRA which is the sector council for the land-based industries uh, sector, not with us sorry . . . areas such as horticulture etc.”.

Anyone for competences in sense of humour studies?

The “unrepentant capitalist” Luke Johnson, in the FT, said

“I have radically downsized HR in several companies I have run, and business has gone all the better for it.”

Now there’s a thought.

The follow-up

The provost’s newletter for 24th June 2008 could just be a delayed reaction to this piece? For no obvious reason, it starts thus.

“(1) what’s management about?
Human resources often gets a bad name in universities, because as academics we seem to sense instinctively that management isn’t for us. We are autonomous lone scholars who work hours well beyond those expected, inspired more by intellectual curiosity than by objectives and targets. Yet a world-class institution like UCL obviously requires high quality management, a theme that I reflect on whenever I chair the Human Resources Policy Committee, or speak at one of the regular meetings to welcome new staff to UCL. The competition is tough, and resources are scarce, so they need to be efficiently used. The drive for better management isn’t simply a preoccupation of some distant UCL bureaucracy, but an important responsibility for all of us. UCL is a single institution, not a series of fiefdoms; each of us contributes to the academic mission and good management permeates everything we do. I despair at times when quite unnecessary functional breakdowns are brought to my attention, sometimes even leading to proceedings in the Employment Tribunal, when it is clear that early and professional management could have stopped the rot from setting in years before. UCL has long been a leader in providing all newly appointed heads of department with special training in management, and the results have been impressive. There is, to say the least, a close correlation between high performing departments and the quality of their academic leadership. At its best, the ethos of UCL lies in working hard but also in working smart; in understanding that UCL is a world-class institution and not the place for a comfortable existence free from stretch and challenge; yet also a good place for highly-motivated people who are also smart about getting the work-life balance right.”

I don’t know quite what to make of this. Is it really a defence of the Brain Gym mentality?

Of course everyone wants good management. That’s obvious, and we really don’t need a condescending lecture about it. The interesting question is whether we are getting it.

There is nothing one can really object to in this lecture, apart from the stunning post hoc ergo propter hoc fallacy implicit in “UCL has long been a leader in providing all newly appointed heads of department with special training in management, and the results have been impressive.”. That’s worthy of a nutritional therapist.

Before I started writing this response at 08.25 I had already got an email from a talented and hard-working senior postdoc. “Let’s start our beautiful working day with this charging thought of the week:”.

He was obviously rather insulted at the suggestion that it was necessary to lecture academics with words like ” not the place for a comfortable existence free from stretch and challenge; yet also a good place for highly-motivated people who are also smart about getting the work-life balance right.”. I suppose nobody had thought of that until HR wrote it down in a “competence”?

To provoke this sort of reaction in our most talented young scientists could, arguably, be regarded as unfortunate.

I don’t blame the postdoc for feeling a bit insulted by this little homily.

So do I.

Now back to science.

This afternoon I went to the Coliseum to see a revival of Jonathan Miller’s 1986 production of the Mikado. It was beautifully staged. The well-known patter song of Ko-Ko, the Lord High Executioner of Japan, begged for a version that deals with anti-science (original here). The serious post will come later. Meanwhile here’s some late night rhyming.

Ko-Ko

As some day it may happen that a victim must be found,
I’ve got a little list — I’ve got a little list
Of society offenders who might well be underground,
And who never would be missed — who never would be missed!
There’s the pestilential bureaucrats who want to send you on a course —
The HR folks who treat you not as human but as mere resource
Skills specialists who think that education just means training
And all ex-scientists who used to work, but now are only feigning
They’d none of ’em be missed — they’d none of ’em be missed!

Chorus.

He’s got ’em on the list — he’s got ’em on the list;
And they’ll none of ’em be missed — they’ll none of ’em be missed.

Ko-Ko

And that deluded nuisance, whom no one understands
The homeopathist – I’ve got her on the list!
All Reiki folks, pill hucksters and layers on of hands
They’d none of ’em be missed — they’d none of ’em be missed.
And herbalists and Princes who like to talk to trees
Those phony nutritionists who’ll treat you for large fees
And that singular anomaly, the acupuncturist —
I don’t think they’d be missed — I’m sure they’d not be missed!

Chorus.

You may put ’em on the list — you may put ’em on the list;
And they’ll none of ’em be missed — they’ll none of ’em be missed!

Ko-Ko

There’s the vision statement writer, and others of that sort
And the crystal therapist — I’ve got him on the list!
And the people who think long words are a substitute for thought
They never would be missed — they never would be missed!
Then those whose knight starvation makes them crave the honours list
So all below must suffer in case their chance is missed
And those who think that science can be judged by its citations
And so kill creativity by funding only applications
But it really doesn’t matter whom you put upon the list,
For they’d none of ’em be missed — they’d none of ’em be missed!

Chorus

You may put ’em on the list — you may put ’em on the list;
And they’ll none of ’em be missed — they’ll none of ’em be missed!

Postscript

I replaced two lines after my pedantic sister pointed out their imperfect rhyme and scansion. Personally I’m with Charles Babbage.

Here is letter that Babbage is said to have written to Tennyson after reading “The vision of sin”.

In your otherwise beautiful poem there is a verse that reads:

“Every moment dies a man
Every moment one is born”

It must be manifest that, were this true, the population of the world would be at a standstill. In truth the rate of birth is slightly in excess of that of death. I would suggest that in the next edition of your poem you have it read:

“Every moment dies a man
Every moment 1 1/16 is born”

Strictly speaking this is not correct. The actual figure is a decimal so long that I cannot get it in the line, but I believe that 1 1/16 will be sufficiently accurate for poetry.

I am etc,

(The Mathematical Gazette, 1927, p270)

Jump to Times Higher Education coverage


This is a longer version of comments published in the Times Higher Education Supplement, June 1, 2007. This longer version has now been printed in full in Physiology News, 69, 12 – 14, 2007 [download the pdf version].

It has now been translated into Russian.

Download pdf version of this paper.

I should make it clear that the term ‘bean counter’ is not aimed at accountants (we need good honest accountants). Rather it is aimed at a small number of senior academics and HR people who do not understand how to assess people.

How to get good science

David Colquhoun, Department of Pharmacology, University College London (May 2007).
email me

The aim of this article is to consider how a university can achieve the best research and teaching, and the most efficient administration.

My aims, in other words, are exactly the same as every university vice-chancellor (president/rector/provost) in the country. Continue reading