A few months ago I posted a puzzle about aggregating probability estimates from different sources, and in particular how to aggregate opinions about the independence of two events.
I think I now understand the story slightly better. I am essentially going to agree with what Terry T. said in the comments to the first post (this is my surprised face) but at the same time try to dissolve my initial resistance to talking about second-order probabilities (statements of the form “the probability that the probability is p is q….”)
To save you a click, the question amounts to: if half of your advisors tell you that X and Y are independent coins with probability .9 of landing heads, and the other half of your advisors agree the coins are independent but say that the probability of heads is .1 for each, what should your degree of belief in X, Y, and X&Y be? And should you believe that X and Y are independent events, a fact about which your advisors are unanimous?
The answer depends, at least in part, on what you mean by “probability” and “independence.”
On one account, probability is a number between 0 and 1 that represents your degree of belief in a hypothesis, and independence of X and Y means that Pr(X&Y) = Pr(X)Pr(Y). Both are assertions about your mental state. So there’s no reason that the unanimity of your advisors about the independence of X and Y should make you believe that X and Y are independent; why should this aspect of their mental state automatically be taken to be a guide to yours? Relevant comparison: what if each advisor said “I am really sure my belief about the coin is correct.” Since all your advisors agree that the nature of the coin is very strongly certain, should you agree about that too? No — given that half your advisors think the coin is very likely to fall heads and half that it is very likely to fall heads, you are reasonably pretty unsure about the nature of the coin. Moreover, if X falls heads, you should rationally increase your degree of belief that Y will fall heads too, because X falling heads is evidence that the 0.9 gang is correct in their beliefs. So (for you, even if not for your advisors) the two events are not independent.
There is another account, in which the probability is an intrinsic property of the coin. On this account, it makes sense to talk about second-order probabilities: to say, for instance, that the probability that “the probability that the coin falls heads is .9″ is 1/2. On this account, we can talk (as Terry does) about conditional independence; we say that there is an unknown parameter p which measures the propensity of the coin to fall heads, and that the condition Pr(X&Y) = P(X)P(Y) for independence only makes sense once P(X) and P(Y) are known.
In fact, I’ve come to favor the second view, at least as regards coins. Because here’s the thing. Let’s say I start with the first view. I have in mind a degree of belief that the first coin will fall heads, and I call this P(X). Given the evidence I have, probably P(X) should be 0.5. But once I’m forming degrees of belief, I must also have a degree of belief that a sequence of k tosses of the coin will all fall heads. And this should be the average of (0.9)^k and (0.1)^k, not (0.5)^k!
Having in mind the probability distributions on “number of heads in k tosses” for all k is, by De Finetti’s theorem, more or less the same as having in mind a probability distribution on the propensity of the coin to fall heads. That is, if a binary event is one we can imagine repeating, then our subjective degrees of belief about the event automatically have the structure of a second-order probability distribution on (Bernoulli) probability distributions. In fact, I think this was why De Finetti proved De Finetti’s theorem. In this context, independence is an intrinsic fact about the coins, not about our knowledge, and we should agree with our advisors that the coins are independent.
I’m less sure this story applies to uncertain events which are, by their nature, unrepeatable. What do we mean when we talk about the probability that Ankylosaurus had feathers? Is it meaningful in this context to say “I think there’s a 50% chance that there’s a 90% chance Ankylosaurus had feathers, and a 50% chance that there’s only a 10% chance” or is this exactly the same as saying you think there’s a 50% chance?