I really like John Ioannidis: his famous 2005 article “Why Most Published Research Findings are False” probably did more than any other paper to draw attention to the problems with blind use of p-value certification in medicine.
But he has a preprint up on medrxiv today that is really poorly done, so much so that it made me mad, and when I get mad, I blog.
Ioannidis has been saying for months that the COVID-19 pandemic, while bad, is not as bad as people think. Obviously this is true for some value of “people.” And I think he is right that the infection fatality rate, or IFR, is in most places not going to be as high as the 0.9% figure the March 16 Imperial College model used as an estimate. But Ioannidis has a much stronger claim; he thinks the IFR, in general, is going to be about 1 or 2 in a thousand, and in order to make that case, he has written a paper about twelve studies which show a high prevalence of antibodies in populations where not very many people have died. High prevalence of infection + few deaths = low IFR.
I think I am especially irritated with this paper because I agree that the IFR now looks lower than it looked two months ago, and I think it’s important to have good big-picture analysis to back that intuition up — and this isn’t it. There’s a lot wrong with this paper but I just want to focus on one thing that jumped out at me as especially wrong, and that is Ioannidis’s treatment of the Netherlands antibody study.
That study found that in blood donors, all ages 18-72 (Ioannidis says <70, not sure why), 2.7% showed immunity. Ioannidis reports this, then makes the following computation. About 15m of the 17m people in the Netherlands are under 70, so this suggests roughly 400,000 people in that age group had been infected, of whom only 344 had died at the time of the study, giving an IFR of a mere 0.09%. Some plague! Ioannidis puts this number in his table and counts it among those of which he writes “Seven of the 12 inferred IFRs are in the range 0.07 to 0.20 (corrected IFR of 0.06 to 0.16) which are similar to IFR values of seasonal influenza.”
But of course the one thing we really do know about COVID, in this sea of uncertainty, is that it’s much, much more deadly to old people. The IFR for people under 70 is not going to be a good estimate for the overall IFR.
I hashed out some numbers — it looks to me like, using the original March 16 Imperial College estimates, derived from Wuhan, you would derive an infection fatality rate of about 0.47% among people age 20-70. There are about 10.8m Dutch people in that range (I am taking all this from Wikipedia data on the age distribution of the Netherlands) so if 2.7% of those are infected, that’s about 300,000 infections, and 344 deaths in that group is about 0.11%. Lower than the Imperial estimate! But four times lower, not ten times lower.
What about the overall IFR? That, after all, is what Ioannidis’s paper is about. If you count the old people who died, the toll as of April 15 wasn’t 344, it was over 3100. If the 2.7% prevalence rate were accurate as a population-wide estimate, the total number of infected people would be about 460,000, for an IFR of 0.67%, more than seven times higher than the figure Ioannidis reports (though still a little lower than the 0.9% figure in the Imperial paper.) Now we definitely don’t know that the infection rate among old Dutch people is the same as it is in the overall population! But even if you suppose that every single person over 70 in the country is infected, that gets you to a little over 2 million infections, and an IFR of 0.15%. In other words, the number reported by Ioannidis is substantially lower than the theoretical minimum the IFR could actually be. And of course, it’s not the case that everybody over 70 already had COVID-19 in the middle of April. (For one thing, that would make the IFR for over-70s only slightly higher than the IFR overall, which contradicts the one thing about COVID we really know!)
There’s no fraud here, I hasten to say. Ioannidis tells you exactly what he’s doing. But he’s doing the wrong thing.
Quomodocumque 