Malcolm Gladwell wrote:
It doesn’t take much imagination to see how risk homeostasis applies to NASA and the space shuttle. In one frequently quoted phrase, Richard Feynman, the Nobel Prize- winning physicist who served on the Challenger commission, said that at NASA decision-making was “a kind of Russian roulette.” When the O-rings began to have problems and nothing happened, the agency began to believe that “the risk is no longer so high for the next flights,” Feynman said, and that “we can lower our standards a little bit because we got away with it last time.” But fixing the O-rings doesn’t mean that this kind of risk-taking stops. There are six whole volumes of shuttle components that are deemed by NASA to be as risky as O-rings. It is entirely possible that better O-rings just give NASA the confidence to play Russian roulette with something else.
If this is really what Feynman said, wasn’t he wrong? In Russian roulette, you know there’s one bullet in the gun. The chance of a catastrophe is just one in six the first time you put the gun to your head; but if you survive the first try, you know the round is in one of the remaining five chambers and the chance of death next time you pull the trigger climbs to 20%. The longer you play, the more likely disaster becomes.
But what if you don’t know how many chambers are loaded? Suppose you play “Bayes Roulette,” in which the number of bullets is equally likely to be anywhere from 1 to 6. Then the chance of survival on the first try is (5/6) if there’s 1 bullet in the cylinder, (4/6) if 2 bullets, and so on, for a total of
(1/6)(5/6) + (1/6)(4/6) + … (1/6)(0/6) = 5/12
which is about 41%. Pretty bad. But let’s say you pull the trigger once and live. Now by Bayes’ theorem, the chance that there’s 1 bullet in the cylinder is
Pr(1 bullet in cylinder and I survived the first try) / P(I survived the first try)
(5/36)/(5/12) = 1/3.
Similarly, the chance that there are 5 bullets in the cylinder is
(1/36)/(5/12) = 1/15.
And the chance that there were 6 bullets in the cylinder is 0, because if there had been, well, you would be a former Bayesian.
All in all, your chance of surviving the next shot is
(5/15)*(4/5) + (4/15)*(3/5) + (3/15)*(2/5) + (2/15)*(1/5) + (1/15)*0= 8/15.
In other words, once you survive the first try, you’re more likely, not less, to survive the next one; because you’ve increased the odds that the gun is mostly empty.
Or suppose the gun is either fully loaded or empty, but you don’t know which. The first time you pull the trigger, you have no idea what your odds of death are. But the second time, you know you’re completely safe.
I think the space shuttle are a lot more like Bayes Roulette than Russian Roulette. You don’t know how likely an O-ring failure is to cause a crash, just as you don’t know how many bullets are in the gun. And if the O-rings fail now and then, with no adverse consequences, you are in principle perfectly justified in worrying less about O-rings. If you shoot yourself four times and no bullet comes out, you ought to be getting more confident the gun is empty.