Doctor Garry XIX: Nullius in Verba

Much like Bruce Forsyth, the Royal Society has a catchphrase: Nullius in verba or 'On the word of nobody’. Science isn’t about assertions on what is right, handed down from authority figures. It’s about clear descriptions of studies, and the results that came from them, followed by an explanation of why they support or refute a given idea.

Ben Goldacre:

I Think You’ll Find It’s A Bit More Complicated Than That, Fourth Estate (2014), page 17

How time flies. It’s a year to the day since Doctor Garry was first in. A year and a day since I Am Garry decided that

There are many studies showing positive effect of chess on delaying, improving dementia/Alzheimer’s. Also good results with Down (sic) Syndrome.

was a message that he wanted to send out into the world.

Alas for IAG - fortunately for the rest of us - science isn’t about assertions handed down from authority figures. I think you’ll find it’s a bit more complicated than that. If somebody - even an ex-World Chess Champion - says there are "studies" to support their argument it means precisely nothing unless we know what those studies are. Unless we can check for ourselves should we choose so to do.

This is fundamental. It’s a point that Goldacre repeats over and over again in the book of his collected writings.

Science has authority, not because of white coats or titles, but because of precision and transparency: you explain your theory, set out your evidence, and reference the studies that support your case. Other scientists can then read it, see if you’ve fairly represented the evidence, and decide whether the methods of the papers you’ve cited really do produce results that meaningfully support your hypothesis.

ITYFIABMCTT, pg 4

... science isn’t about authoritative utterances from men in white coats, it’s about showing your working.

ITYFIABMCTT, pg 116

Academic papers are filled with ideas and evidence to be read, weighed up, and critically appraised by people with the motivation and skills to do so, whoever they may be. Science is not, and should not be, about arguing from authority. The idea that conclusions of a published paper are automatically true was never helpful. 

ITYFIABMCTT, pg 146

As so often, this is about transparency, which is ultimately the only source of authority in science: we want the methods and result of scientific research to be formally presented, and accessible by all, so that we can see what was done, and what they found. If a government report on anything relies substantially on unpublished and inaccessible research, then we are correctly concerned. 

ITYFIABMCTT, pg 193

science is about clarity and transparency

ITYFIABMCTT, pg 199

It is possible in healthcare to do great harm, while intending to do good, and so medicine thrives on criticism: this is how ideas improve ....

ITYFIABMCTT, pg 251

If science has any credibility, it derives from transparency: when you make a claim about how something works, you provide references to experiments, which describe openly and in full what was done, in enough detail for the experiment to be replicated. You explain what was measured, and how. Then people can freely discuss what they think this all means in the real world. 

ITYFIABMCTT, pg 255

The entirety of science is built on transparency, on giving your evidence and engaging with legitimate criticism. If you hear of a company refusing to hand over the evidence it says it supports its claims - whether it is a drug company, or some dismal cosmetics firm - all you know is that you are being deprived of information, and that vital parts of the picture are missing.

ITYFIABMCTT, pg 256

This is not difficult to understand and conclusions are not difficult to draw. Anybody who doesn’t get it is simply not to be trusted.

twitter

And here we are again.  "It's been shown in studies ...." I Am Garry told us just a few weeks ago.

For what it’s worth, the 'potential for distraction is as disruptive as actual distraction' thesis strikes me as more than a little plausible. That I find it so is entirely besides the point. What if I wanted to check? What if I wanted to prove my gut feeling/suspicion/opinion based on personal experience of  two years trying to get some bloody work done at the god-awful LSE Library/etc was actually correct?

What could be less clear, less transparent, more inaccessible than "studies" that are not named? Since it’s impossible to know what research is being referred to here, there is no possible way to discover whether it exists and if it does whether it says what it’s claimed to say. We cannot look it up. We cannot check.

In truth there is nothing here of any substance at all. Nothing but, "believe on my word for I Am Garry."

That, a year on, is where we still are with Garry Kasparov. He implies 'scientific method' but scratch the surface and it turns out that what he’s up to is indistinguishable from the behaviour of a Chessbase bullshitter.

Chess and Dementia Index