Wednesday, May 28, 2014

Doctor Garry III: Dogs That Don't Bark

NEJM: Don’t mention the chess

Verghese. Joe Verghese. It only takes the briefest of internet searches combining 'chess' and 'dementia' to find the name.

Dr Verghese was the lead author of a journal article which is routinely cited amongst chessers on the internet. Leontxo Garcia recently called it "the best study" on the thesis that chess could, "prevent or delay Alzheimer’s". I don’t claim to be a "great expert", as Kasparov dubbed Garcia in that nonsense "many studies" tweet of his (Doctor Garry is In), but I have been looking into the subject for a couple of years off and on, and I’ve yet to discover anything better the research undertaken by Verghese and his colleagues either.

The New England Journal of Medicine
'Leisure Activities and the Risk of Dementia in the Elderly
2003 (348: 2508-2516)
Joe Verghese et al

If you’re interested, you can read the whole thing for free online. It’s here.

If you do, you’ll notice that there’s a problem. Actually, for an academic study which is supposed to be the proof that playing chess has a beneficial impact on your chances of avoiding dementia, it’s a very bizarre problem indeed.

Verghese’s article doesn’t actually mention chess. At all.

It’s a straightforward idea: higher rates of participation in leisure activities amongst older people are associated with a lower chance of a receiving a diagnosis of dementia. Not a difficult hypothesis to grasp, to be sure, and yet one that’s not as easy to test as you might imagine.

A formal diagnosis of dementia is typically preceded by a lengthy period of decline in brain functioning. This means that asking people what they like to do in their spare time and then waiting to see if they go on to get dementia isn’t a great research method because you can never know for sure whether the activity levels reported at the beginning are already being affected by a dementia which hasn’t yet been recognised1.

Verghese’s solution to this methodological difficulty was simplicity itself: collect the data over an extremely long period of time. 21 years to be exact. From 1980 to 2001, 4692 participants aged at least 75 and shown to be free of dementia were asked about their participation in various leisure activities and then given health assessments every year or so.

By the end of the study 124 of the participants had been given a diagnosis of dementia (approximately half of those being Alzheimer’s Disease). Verghese and his colleagues then crunched the numbers and concluded that certain leisure activities were indeed associated with a lower chance of going on to develop dementia.

It sounds good, doesn’t it? Which is probably why references to Verghese’s research crop up all over the shop - including this article that Susan Polgar felt the need to bring to a wider audience, for instance.

Given that our subject is bullshit you might feel that there’s a certain barrel like quality in taking aim at Polgie. True enough, I suppose. As it happens, she’s far from alone in referencing Verghese. You’ll find his study mentioned here, for example. And here. Not to mention Leontxo Garcia’s tweet and his article "Checkmate to Alzheimer’s" (which you can view here).

Follow the links and you’ll find phrases like, "Playing chess can lower the risk of dementia by 74%", "Recent research indicates that playing chess can reduce the risk of AD by up to 75%"3 and "Chess seems like a treatment that works"4. What none of these sources tells you is that the word "chess" doesn’t actually appear in the Verghese article. Not even once. The categories for what the authors call 'cognitive activities' are actually,

  • "reading books or newspapers";
  • "writing for pleasure";
  • "doing crossword puzzles";
  • "playing board games or cards";
  • "participating in organised group discussions";
  • "playing musical instruments".

The claim that Verghese’s research proves/suggests/implies that chess prevents/reduces the risk of/cures dementia rests on the reader deciding that the 'playing board games or cards' category means playing 'chess'. This despite the fact that cards are clearly very much not chess and that 'board games' could mean almost anything at all.

Actually that last statement isn’t quite true. Verghese and his colleagues don’t break down the definition of what 'board games' actually means, but we do have a clue from another article in the same edition of the NEJM.

You may recognise the reference ...

The New England Journal of Medicine
'Use it of Lose it - Do Effortful Mental Activities Protect against Dementia?’
2003 (348: 2489)
Joseph T. Coyle

... as the origin of our 'reasonable position' benchmark from Doctor Garry II. As far as I know the article is not available online, but I was able to get a copy via the Alzheimer’s Society library a year or two back5. We don't find "chess" in the main body of Coyle's article either, but there is one graph that’s of interest. Not the chart per se. The caption.

"Persons Who Frequently Played Board Games Such as Chess, Checkers, Backgammon or Cards  [with] Subjects Who Rarely Played Such Games."

So those internet chessers can assume that Verghese's research was about chess and Leonxto Garcia can state outright, that "When the study [i.e Verghese’s article - JMGB] says "board games" means chess and bridge" if they want. If they do, though, the most generous thing you can say about them is that they are misleading their readers6.

A chess set

If it doesn’t mention chess, how did the Verghese study come to be the rallying point for chess - dementia enthusiasts in the first place? It’s an interesting question. One that forms the subject of our next post (spoiler alert: for those who can’t wait).

In the meantime we have reached our bottom line for today. 'Leisure Activities and the Risk of Dementia in the Elderly': not a dog that didn’t bark in the night. Not actually a dog at all.

The curious absence is not the lack of any mention of chess in Verghese's study. It’s the fact that those who are keen to tell you that the research shows chess to be an effective intervention to tackle dementia manage to do so without letting you know that there is not actually any chess in the material that they reference7. Now that’s a mystery worthy of an explanation. Yet again, though, we’ll have to leave it for another time.

This day’s conclusion: whatever else we might say about Verghese’s research, we know that there’s nothing whatsoever in either of his or Coyle's articles that justifies singling out chess. Nothing that supports a claim that our particular board game is special or different to the others listed.

Which leaves us exactly where we were at the end of Doctor Garry II:

... seniors should be encouraged to read, play board games, and go ballroom dancing, because these activities, at the very least, enhance quality of life, and they just might do more than that.

That was Doctor Coyle’s opinion after reviewing Verghese’s work. I wonder why it isn’t ours.

Chess and Dementia Index

1 It is known that one symptom of dementia is a progressive withdrawal from social and leisure activities. The problem the experimenters faced was the possibility that they might see people with low rates of social activity going on to be diagnosed with dementia and conclude the former led to the latter when in fact, unbeknown to all concerned, the person already had dementia before the study began and the causal relationship was precisely the opposite of what it appeared to be.

2 Not counting 19 who dropped out.

3 More on these impressively large figures some other day. For now, suffice to say that not only will you not find 'chess' in the original article, you’ll also struggle to identify any claim for outcomes that’s bold as you see here.

4 Again, there's plenty more to say about this particular phrase, but will leave it for another time.

5 The British Library should also be able to help you if you want to track down a copy of the original text.

6 To be fair to Garcia, it is only reasonable to acknowledge that twitter is hardly a medium that's conducive to conveying complexity even when you’re tweeting in your native language.

7 To his credit, Garcia did acknowledge this point when pushed.


Anonymous said...

Perhaps you need to know how many chess players there were in the 469? On the definition of "once a year" used in some surveys, there might have been as many as 50, but it's somewhat stretching it to suggest a once a year activity has any medical effect. "Serious" players you might only have zero, one or two.


Jonathan B said...

A point well made Roger.

Of course that’s the very first thing you need to know if you’re claiming that the study in any way has anything to say about chess and dementia.

There’s quite a bit more to come on this theme but it will take a few weeks.

In the meantime, one of the other things you’ll need to know is how much chess playing was going on. The study takes 'frequently' to mean 'at least several times a week'. But it doesn’t say how long each of those 'several' chess sessions last.

Nor does it say how long the study participants have been playing chess. Only that they were playing at the start of the study.

And what do we mean by 'play chess' in this context. Serious players like us or recreational players who might play chess in the same way that they’d play monopoly or what have you?

Richard James said...

Excellent series, Jonathan. I trust you're going to follow it up with a series deconstructing the claims that chess makes you smarter.

Andrew Gelman said...

I don't know if chess makes me smarter, but playing it makes me feel pretty stupid most of the time!

Bertrand Russell said...

You seem to be attacking a straw man. Who said that chess was "special or different" to other board games in respect of reducing the risk of dementia? Does Kasparov say that playing chess reduces the incidence of dementia, but playing bridge, draughts or backgammon do not?

Verghese says that playing board games reduces the risk of dementia. Chess is a board game. So, playing chess reduces the risk of dementia. What's wrong with that?

ejh said...

The late Professor Russell:

It's worth reminding yourself of what exactly Kasparov did say, which I am sure, with a philopsopher's attachment to logic and exactitude, you will agree was something he couldn't justify.

Anonymous said...

"Bertrand Russell"'s logic is obviously flawed. Ludo, snakes and ladders are also board games and by that logic all reduce the risk of dementia. Only what has been studied can be concluded on.

I suppose we could apply the logic to anything. Smoking has been proven to cause lung cancer. Smoking fish is a form of smoking. Therefore the act of smoking some salmon predisposes to lung cancer... Even more fasciciously (if possible) - food is required for sustaining human life. McDonalds is a type of food (probably), therefore ipso facto McDonalds is required for life!

ejh said...

I think that at the point where the leading chess figures touting chess as a remedy for dementia

(a) are clear that chess hasn't been shown to be more effective than any other board or card game


(b) are clear about what board and card games have been shown to do and what they haven't

then we'll be in a position to say that yes, if chess is (as far as we are yet aware) no more effective than bridge or crosswords then at least it is not (as far as we are yet aware) any less.

That's a bit convoluted, but hopefully you see the point.

Jonathan B said...

@Richard: maybe i will do. I’ve got plenty to keep me going on the chess and dementia front for a good while yet though.

Obviously there are crossovers on the themes, though. (e.g. claims made that are not supported by any evidence) and things like the Leonxto Garcia article being available on the Chess in Schools Website.

Jonathan B said...


Anonymous has saved me the bother of a longer reply to you. I only need add that when you write, "Verghese says that playing board games reduces the risk of dementia." you are factually incorrect - as you would know if you’d troubled yourself to actually read his article.

I have to say that my intention for this series is to write about stuff that interests me on the chess-dementia theme. It is not to provide a rebuttal service for any ignorant, ill-informed or just plain wrong opinion that anybody might want to throw my way.

Feel free to not engage with what I have to say if you wish, but comments of the quality that you provided today will simply be binned in future.

In short: raise your game or take yourself off somewhere else. Like Susan Polgar’s website, for example.

That said, now we have demonstrated to our satisfaction that Verghese talked about 'board games and playing cards' and not 'chess' we can start to ask interesting questions like, is chess better than other board games (or worse or the same) and what evidence do we have to support the argument either way.

And we will ask those questions, we will. In due course. I’ve got some other stuff to get through yet.

Anonymous said...

And what do we mean by 'play chess' in this context. Serious players like us or recreational players who might play chess in the same way that they’d play monopoly or what have you?

In this context a loose definition would do. So someone with a published rating, including those on chess servers or someone who plays several times a week, even if casually against a chess engine.

Am I right to say there isn't currently a cure, but it did occur to me that if you wanted an early test, would a loss of performance in a mind sport such as chess be an early warning sign? That pre-supposes there's enough existing performance to be lost.


Jonathan B said...

"In this context a loose definition would do."

I don’t think so. In the same way that Pill X is beneficial for Condition Y, but only if it is taken at a certain dose at a certain frequency, simply 'playing chess' isn’t really good enough.

It *may* be that it doesn’t really matter whether you play for years or take up the game later in life. It may be that it doesn’t matter if you play seriously or casually, but we can’t assume that.

As for your later point, no there’s no cure for dementia just now. And no way to guarantee not getting it. There are certain lifestyles and activities that are associated with lower risks. Something we’ll explore later.

You’re right. There is the period of declining mental function precedes a formal diagnosis so it could be that loss of performance is an early warning sign. On the other hand, performance might decline anyway regardless of whether the person has dementia or a Mild Cognitive Impairment or not.

On the other side of the coin, it’s frequently the case that peoples cognitive functions decline in various areas but in something with which they are very familiar. It could possibly be that performance at chess is maintained and to some extents hides declines elsewhere.

It’s a complex issue. One to come back to.