Figure of fun

How many chess players are there in the world? The figure will surprise you. It surprised me all right.

It's true. I read it in the paper.

I say the figure surprised me, since the last time I was following this statistic it was only half a billion. Good Lord, another hundred million or so in less than nine months, you'd think the world would take more notice.

I mock, of course. But perhaps I mock without good reason, since this stat, it is claimed, derives from some actual research.

No, really. I asked the chap who wrote the Metro piece where he got his figure from.

Give him credit: this might be a straightforward case of churnalism, but the chap was decent enough to reply. He directed my attention to this press release, issued last August by AGON, published on the FIDE website and referring to research carried out by YouGov.

The press release claims:

605 million adults play chess regularly

among whom

over 6m, 35m, 16m, 50m and 85m people in the U.K., U.S., Germany, Russia, and India (ABC1), respectively, are playing chess regularly.

Uh huh. OK, those are some numbers, where do they come from? The press release goes on at some length about its methodology, and the number of individuals on YouGov's panels

The methodology for the UK, Germany, Russia and India surveys was the same as for the US, conducted using online interviews administered to members of the YouGov Plc UK panel of 350,000+ individuals who have agreed to take part in surveys, the YouGov Plc German panel of 120,000+ individuals, the YouGov Plc Russian panel of 77,000 individuals, and the YouGov Plc Indian panel of 8,000+ individuals

which sounds a lot - a "UK panel of 350,000+ individuals" - although the actual numbers involved in the survey turn out to be rather smaller

Total sample sizes were 4,161 adults (UK), 2,024 adults (US), 953 adults (Germany), 2,013 adults (Russia), and 1,000 adults (India).

Still, that's a sample, and as the final paragraph goes on to say:

All sets of figures for the UK, US, Germany and Russia have been weighted and are representative of each country’s adults (aged 18+)

albeit

the figures for India are representative of the online Indian adult population

which, one suspects, may not be entirely representative of the Indian population as a whole.

Also, from the information supplied, one doesn't - quite unusually for a survey of this kind, I think - know what questions were being asked, nor what answers were given, still less how exactly these answers were extrapolated to give us a figure of 605 million. Which are all things you kind of need to know, before you know how to evaluate the figure, before you can say how much credibility you think the figure actually possesses.

Never mind, though, because we are invited to contact a PR company if we wish to have "access to the raw YouGov survey results". Which we surely do.

Unfortunately, the individual named in the press release is no longer at the PR company. She appears to be working for the Telegraph (which in itself - going from a PR company to a daily paper - helps explain why there's so much churnalism about) and despite calling them up and being given another individual's email address, I was not favoured with any kind of reply. So I contacted YouGov directly.

Before I go any further, a note or three. The first is that one thing that made me suspicious about the 605 million figure, quite apart from its general improbability, was that AGON had been using it prior to the survey being completed, let alone published. A survey might well confirm one's general impression. It may confirm one's opinion. However, for it to confirm a figure like "605 million" - not more, not less - is pretty lucky, not to put it more strongly.

The second is that one should always treat with scepticism figures for numbers "worldwide" (such as the figures one often sees, for instance, for the television audience for important sporting events). There is, for example, a good cautionary tale here* involving a not dissimilar figure. If you don't know how the figure was arrived at, it's probably an exaggeration. If it relates to the audience for whatever product the people broadcasting the figure are selling, even more so.

The third is that I have been playing chess for around thirty-five years, and I am as sure as sure can be that there are no more 605 million chess players in the world than there are 605 million Euros in my bank account.

Indeed, Andrew Paulson claims of the figure, which he was apparently given by FIDE:

I didn't particularly believe this number – nobody would tell where did it come from. It was just like choosing the number, which would be high enough, so that people would buy it.

You think? What luck, then, that

in fact, what was strange, that based on all of the information that we assembled from the above mentioned sources...it actually seems that 600 million is entirely credible, maybe even low.

Phew! But you're not convinced? Then go check for yourselves, because

the raw data from the polls is available to any of you without any question.

Without any question, he says, but as I discovered, not entirely without difficulty. Still, YouGov were much more helpful than the PR company and were able to send me four charts. These relate to:

• Germany
• Moscow
• the UK
• the US.

This data does at least tell us what questions were asked, what answers were given and by how many people.

The data is incomplete and there are some inconsistencies with the commentary in the AGON press release. For instance, there are 2090 German respondents rather than only 953, we have 251 respondents from Moscow where there were apparently 2013 from Russia overall - and we have nothing at all from India.

But what we do have is interesting enough and merits more detailed consideration than, for reasons of time and space, it's going to get here. For now, let us look just at the numbers who answered the question

Which ONE, if any, of the following BEST describes the last time you played chess either against a person or a computer?

with one of the following

- I generally play chess at least once a week
- I generally play chess at least once a month; or
- I generally play chess at least once a year.

That would include all the people who say they generally play chess at least once a year or more, and hence can be described as playing chess regularly. We get these totals:

23% for Germany
55% for Moscow
12% for the UK
14% for the US.

These are higher numbers than one might have expected: if, for instance, we can reliably extrapolate from the UK figure to the UK adult population generally, we'd get, I think, close to six million people who, in their opinion, play at least one game a year. (I'd treat that claim with caution, but from this research at least, it can't be ruled out.)

I'm not sure how they get to this claim:

over 6m, 35m, 16m, 50m and 85m people in the U.K., U.S., Germany, Russia, and India (ABC1), respectively

but even so, we can take our perhaps-six-million in the UK, multiply that by one hundred, since about one in a hundred adults in the world are from the UK** and we're close to our figure. Hurrah!

Except of course, that's what we can't do. We can't assume that, even if the samples we have are representative of the populations of those countries, they're at all representative of the wider world: of the rest of Europe, of Africa, Latin America, the Middle East. Of Asia, outside the online section of India (not that we have seen the figures for that community).

We're talking, of course, largely of those large parts of the world which are likely to have a far weaker chess culture if they have one at all, where they are likely to be poorer, where they are less likely, often much less likely, to have access to computers, and therefore where we may reasonably conclude that chess is far less likely to be played.

Now presumably the 605 million figure wasn't reached by multiplying up from the UK data. But it is obvious that in order to arrive at a figure like 605 million, from the published data, you would have to do something like this, if you're going to get anywhere close to our 605 million. But that's precisely what you can't do, because there's no way that the samples in the research are at all representative of the world as a whole.

Seeing as they haven't actually sampled the world as a whole, it's up to the researchers to demonstrate how the figure was properly arrived at. Or rather, it's up to AGON. Because the figure wasn't, it seems, arrived at by YouGov. It's not their number.

I wrote to YouGov:

I do potentially have some questions about it, but in order to know what questions to ask, I need to know whether the figure of 605 million regular chess players, which has been associated with the research, actually appears in it anywhere. Does the figure, or anything comparable, appear in the YouGov research? [Email, 3 February.]

They wrote back to me:

I am afraid that we do not have any further information about this and about how that figure was arrived at. The client wasn't very good at sending through press releases for us to check before they were released into the public domain. Our only guess is that the 605 million figure was extrapolated from the findings about the number [of] people who say they have ever played chess. Unfortunately all we have is the tables that you've seen. [Email, 12 February.]

I rather like that: the client wasn't very good at sending through press releases for us to check before they were released into the public domain.

So, it's a crock. The research didn't say anything about 605 million regular chess players and the researchers themselves don't know where the number came from.

All of which enables me to announce the preliminary findings of my research, which can be summarised as follows:

• there are almost certainly not 605 million regular chess players in the world
• the YouGov research does not suggest any such figure
• the figure of 605 million was not derived from that research, but was linked to it without proper justification
• no basis for the 605 million figure has ever been presented.

Not that it matters. Who needs to justify what you can have recycled without question?

So no doubt we will see the 605 million figure again in the weeks to come. But how ever many times we see it, it will still be absolute cobblers. As far as we know, it was basically made up.

[*Or indeed this one, which appeared in between my writing this piece and this piece being published.]
[** Note however that both these sources use a definition for an adult as aged fifteen and over, which the AGON research does not. I don't think though that this should affect the very approximate 1:100 ratio too much. ]

[Thanks to Seán O'Keefe and others.]