I interviewed Nate Silver last month at the Commonwealth Club in San Francisco for an MSRI event. Video here.
Or so I argue in today’s Boston Globe, where I review Silver’s excellent new book. I considered trying to wedge a “The Signal and The Noise” / “The Colour and the Shape” joke in there too, but it was too labored.
Prediction is a fundamentally human activity. Just as a novel is no less an expression of human feeling for being composed on a laptop, the forecasts Silver studies — at least the good ones — are expressions of human thought and belief, no matter how many theorems and algorithms forecasters bring to their aid. The math serves as a check on our human biases, and our insight serves as a check on the computer’s bugs and blind spots. In Silver’s world, math can’t replace or supersede us. Quite the contrary: It is math that allows us to become our wiser selves.
In January, Timothy Gowers, a professor of mathematics at Cambridge and a holder of the Fields Medal, math’s highest honor, decided to see if the comment section of his blog could prove a theorem he could not.
It’s been years since we’ve been New York Times subscribers; looking at Sunday’s paper I was struck by how much math was in it. In the Year in Ideas section, besides my piece, there’s one about using random walks to identify species critical to the survival of an ecosystem, another about the differential equations governing zombie diffusion, and a third about Nate Silver’s detective work on the fishy final digits of poll results. (I blogged about DigitGate a few months back.) Elsewhere in the paper, John Allen Paulos writes about the expected value of early breast cancer screening, and the Book Review takes on Perfect Rigor, Masha Gessen’s new biography of Perelman. Personally, I think Gessen missed a huge commercial opportunity by not titling the book He’s Just Not That Into Yau.
Nate Silver at 538 looks at the trailing digits of about 5000 poll results from secretive polling outfit Strategic Vision, finds a badly non-uniform distribution, and says this strongly suggests that SV is making up numbers. I’m a fan of Nate’s stuff, both sabermetric and electoral, but I’m not so sure he’s right on this.
Nate’s argument is similar to that of Beber and Scacco’s article on the fraudulence of Iran’s election returns. Humans are bad at picking “random” numbers; so the last digits of human-chosen (i.e. fake) numbers will look less uniform than truly random digits would.
There are at least three ways Nate’s case is weaker than Beber and Scacco’s.
So I wouldn’t say, as Nate does, that the numbers compiled at 538 “suggest, perhaps strongly, the possibility of fraud.”
Update (27 Sep): More from Nate on the Strategic Vision digits. Here he directly compares the digits from Strategic Vision to digits gathered by the same protocol from Quinnipiac. To my eye, they certainly look different. I think this strengthens his case. If he ran the same procedure for five other national pollsters, and the other five all looked like Quinnipiac, I think we’d be in the position of saying “There is good evidence that there’s a methodological difference between SV and other pollsters which has an effect on the distribution of terminal digits.” But it’s a long way from there to “The methodological difference is that SV makes stuff up.”
On the other hand, Nate remarks that the deviation of the Quinnipiac digits from uniformity is consistent with Benford’s Law. This is a terrible thing to remark. Benford’s law applies to the leading digit, not the last one. The fact that Nate would even bring it up in this context makes me feel a little shaky about the rest of his computations.
Also, there’s a good post about this on Pollster by Mark Blumenthal, whose priors about polling firms are far more reliable than mine.