I haven’t read much about this issue, so I cannot make any specific comments.

IMHO I feel like science can only give us a perspective. If we are really looking for highest proof or causality in everything, all we have is proof based on a narrow set of observations.

I think that hardest part is teach the public everything is not black and white as they think or they want to think.There are so many shades of gray. And these shades will hopefully move towards white or black as science develops.

I think perhaps that Wiseman is also exaggerating a bit the extent to which there is unanimity about the question of causality. If I may resort, for a moment, to an anecdote, my recent encounter with the Royal Marsden Hospital provides an example.  I noticed that the refreshment counter in their waiting room was selling ham sandwiches.  When I mentioned this to my rather distinguished oncologist he just laughed.  It is true that the authors of the report include some very distinguished people.  But however distinguished you may be, you can’t divine causality withour data to support your conclusion, and the data are really very thin. 

There are two opposing problems to dietary advice.  On one hand, every small association can be interpreted as causal and very detailed advice offered. This extreme precautionary approach is what the WCRF has adopted and it can certainly be argued that this is the responsible thing to do. The other side of the argument is that excessively detailed advice can be counterproductive.  It makes people laugh at science and runs the risk that really soundly-based advice will be dismisssed,  It is very common to hear people saying that "these people can’t make up their mind".  It happens every time a report on the beneficial/harmless/lethal effect of red wine appears in the press.  Still worse, the example of hormone replacement therapy shows that advice that is not based on sound evidence for causality may do harm rather than good.

I suppose that my thesis, at heart, is that scientists should say rather more often than they do "I don’t know".   Exaggeration about the strength of evidence, however well intentioned, is, in the end, counterproductive.

Where we agree with David Colquhoun is that it is very likely that the more that people eat processed meat, the higher their risk of bowel cancer. Of course we can never be absolutely certain – no studies can ever do that – but the question is whether we are sure enough to make a recommendation. This is where we differ from David.

The difference is in interpretation of the evidence. We think the evidence is convincing and should form the basis of public health advice. David thinks the small element of doubt means the evidence is not strong enough to warrant advising people about processed meat.

David also suggests we should not be giving advice on processed meat because there has not been a randomised control trial. While there is still an element of doubt in randomised trials, everyone agrees they are the most robust type of study.

But while this type of trial is relatively straightforward when measuring the effect of a pill for treating a condition, it is neither practical or realistic to do this for something like the impact of habitual lifelong consumption of processed meat on a condition such as colorectal cancer which has a latency of decades.

The trouble is – as David accepts – some scientific questions simply are not testable using double blind randomised trials, and the impact of food, nutrition and physical activity over a lifetime on cancer risk is one of those. David feels that therefore no recommendation should be made; but this is a vitally important question and we feel that people should be advised according to the best available evidence.

This is a question of judgement and people can make their own minds up about who is right and wrong. But the decision to describe the evidence as convincing comes from a panel of 21 world-renowned scientists.

But if I was about to eat something that someone was confident – even if not certain – was going to be harmful to me, I would want that person to warn me about it.

So in the absence of a randomised control trial, should you never give advice about diet and lifestyle? We would argue this would be an irresponsible approach and this is at the heart of where we disagree with David.

Professor Martin Wiseman
Project Director
Food, Nutrition, Physical Activity and the Prevention of Cancer: a Global Perspective

I did some quick searching and found a few reviews. Figure 1 in each of (Ness AR, Powles JW. Fruit and vegetables, and cardiovascular disease: a review. Int J Epidemiol 1997;26:1–13) and (Steinmetz KA, Potter JD. Vegetables, fruit, and cancer prevention: a review. J Am Diet Assoc 1996;96:1027–39) seem to indicate protective effects of fruit+veg against some cancers and against cardiovascular disease. Nice to know that there are some things we think we know…?

Probability Theory is no magic bullet for sure. It just elevates the discussion of evidential support from “he said/she said” to mathematics. That pushes the argument to either one of mathematical errors or incorrect inputs. Those inputs also can be extended beyond just the data, and include other information.

Consider the case of establishing causality in the face of a large number of potential confounders. The inputs are not just the data, but other potentially relevant information. For an observational study, such information is really your explicit specification of what you know you don’t know, e.g. “I’m measuring a bunch of variables, but I do not know the relationships amongst them”, in particular the arrow of causality. So when you test the hypothesis “eating bacon causes heart disease” vs. “heart disease makes you eat bacon”, you would quantitatively find no preference, as you included no information in the problem that indicates the arrow of causality.

So it’s an extension of your point: you can’t get more information from a problem than what you put in. Information extends beyond data, but also includes other pieces of knowledge. Probability Theory allows you to include this other knowledge (usually excluded from statistical analyses) and make rigorous and consistent assessments of how ALL of the included information supports hypotheses.

BTW, the first 3 chapters of Jaynes book are available free online:

http://bayes.wustl.edu/etj/prob/book.pdf

It might be fun, though, to do an addendum on the beauty of randomisation tests. They are surely on of the best ways of demonstrating the fundamental role of randomisation as the basis of significance testing. I’ve been enthusiastic about them for quite a long time now (DC, Lectures on Biostatistics, 1971, Clarendon Press) and used them for teaching.

The bacon problem illustrates that significance testing is only a small part of the problem, Causality is much harder to show. Having huge numbers of subjects helps a lot for getting “significant” differences but with the risk that you may end up simply detecting smaller and smaller associations that are ever more susceptible to misinterpration because of confounders.

You’d be surprised :-)

Probability Theory doesn’t really “solve” the problem of establishing causality. It does, however, provide a rigorous and internally consistent mathematical framework for reasoning with incomplete information. You could, for instance, quantitatively compare observational vs. randomized controlled studies, and numerically show that the former provides little support for hypotheses of causality. Also very useful for designing experiments, as you can quantify which design is likely to provide greater insight.

Probability Theory would also raise discussions of the relative merits of different study types above that of philosophy. It’s just math.

I suspect, thought, that if it had any solution to the huge problem of establishing causality, we’d have heard about it.

I agree entirely with what you say, and perhaps I should have spelled out in more detail the problem of confounders which is what makes randomisation so important.

In lab experiments, as well as diet studies, systematic errors are often much more important than random sampling errors. I recall that for years each estimate of the speed of light that was made was outside the error limits of the preceding study. Confounders of the sort you mention come into the same category -errors that are reproducible from trial to trial, so giving a spurious appearance of reproducibility.

It is very striking, once again, how very difficult it can be to get firm answers to what sound like simple questions.