John Quiggin had a good post yesterday on Crooked Timber about the various flavors of the Efficient Market Hypothesis, which according to Quiggin is a piece of shuffling zombie social science that won’t die no matter how many flaming sticks you jam through its skull.

The weakest form of the EMH is the so-called “random walk hypothesis” that the future behavior of a stock price is independent from its prior behavior. If that’s the case, no amount of staring at charts is going to help you beat the market. The random walk hypothesis is pretty well-supported by the data we have; a really nice popular account is Burton Malkiel’s A Random Walk Down Wall Street, one of the mathiest bestsellers I know of. It makes a great present for anyone in need of a good rationale for not paying attention to their investments.

Quiggin writes

The success of the random walk hypothesis showed that the existence of predictable price patterns in markets with rational and well-informed traders was logically self-contradictory.

But this doesn’t seem quite right. The EMF, even in its weakest form, holds that the current price of a stock is a best estimate arrived at by an aggregate of profit-maximizing investors with knowledge of the stock’s previous price: if there were a reliable way to use previous prices to determine that tomorrow’s price would be Y, then the investors would figure that out, and today’s going price would be Y as well.

But the random walk hypothesis is much weaker. It makes no claim about the etiology of stock prices. It’s compatible with EMF, but it’s compatible with plenty of other models too — for instance, the one in which stock prices *really are* a random walk, where the price at time t+1 is the price at time t plus, let’s say, a normally distributed random variable X. Such a market would be as impossible to beat as a roulette wheel — but evidently *not* because stock prices are best estimates of the future price of the stock, or of the underlying value of the company, or of anything at all. (Fellow mathematician David Speyer makes a similar point in the comments at CT.)

You can dress this up a bit: suppose price(t+1) – price(t) is the random variable X + [(.001) x t’th digit of pi] – (.045). Unless something very weird is going on with the digits of pi, this version of the stock market would also satisfy the random walk hypothesis. But unlike the pure random walk it’s a market where you can make some money; if I’m lucky enough to have access to the “digits of pi” rule, I can make a small average profit. So the random walk hypothesis can hold for markets that are neither efficient nor unbeatable.

Stronger versions of the EMH hold that the market price already takes into account, not only the previous prices of the stock, but also publicly available information about the stock. So what would happen if I let my secret investing strategy slip out? This isn’t a rhetorical question; I’m authentically curious about what the rational-investor model would say about a market in which prices are publically revealed to have been governed, up until now, by some completely deterministic but “random-looking” sequence like the digits of pi.

I’m authentically curious about what the rational-investor model would say about a market in which prices are publically revealed to have been governed, up until now, by some completely deterministic but “random-looking” sequence like the digits of pi.IANA economist of any sort, but my “gut reaction” is to say that the hypothesis is unrealistic — market behavior simply can’t be described by something that simple. That’s sort of dodging the question, but a slightly less slippery way of putting it might be to conjecture that predicting stock prices is “Wall Street-complete,” i.e. that it is computationally as hard to forecast the market as it is to actually simulate the actions of traders, etc. (I don’t actually believe that this is true in theory, but it seems to be close to true in practice.)

In any case, it’s probably possible to make somewhat rigorous (using Kolmogorov complexity, maybe?) the idea that such a simple formula is (at least “usually”) incompatible with the rational-investor model in the first place.

Lots of things about this post confuse me :) Apologies in advance if I’m missing the point in an epic way!:

The weakest form of the EMH is the so-called “random walk hypothesis” that the future behavior of a stock price is independent from its prior behavior. If that’s the case, no amount of staring at charts is going to help you beat the market.

Nonetheless, it may be possible to beat the market under this independence hypothesis, if future increments in the stock price don’t have mean zero. For example, suppose the stock price just increases by 1 cent every day, deterministically. Then trivially independence holds (any deterministic random variable is independent of anything else) but nonetheless it’s easy to make a profit. (Though of course it’s still true that staring at charts doesn’t help you!)

The random walk hypothesis is pretty well-supported by the data we have

Presumably over long time-intervals? Over short times, it seems clearly false. If I know that a share has lost 50% of its value in the last hour, I will certainly expect some further wild swings, up or down or both, in the next hour too.

But this doesn’t seem quite right. The EMF, even in its weakest form,

Above you said that the weakest form of the EMF _was_ the random walk hypothesis? (Or is there a difference between EMF and EMH?)

holds that the current price of a stock is a best estimate arrived at by an aggregate of profit-maximizing investors with knowledge of the stock’s previous price: if there were a reliable way to use previous prices to determine that tomorrow’s price would be Y, then the investors would figure that out, and today’s going price would be Y as well.

But the random walk hypothesis is much weaker. It makes no claim about the etiology of stock prices. It’s compatible with EMF, but it’s compatible with plenty of other models too — for instance, the one in which stock prices really are a random walk, where the price at time t+1 is the price at time t plus, let’s say, a normally distributed random variable X. Such a market would be as impossible to beat as a roulette wheel — but evidently not because stock prices are best estimates of the future price of the stock, or of the underlying value of the company, or of anything at all.

Why do you say that the stock price is not the best estimate of future price in this case? If X has mean 0, and we don’t yet have any information about X, then surely the current price _is_ the best estimate of the future price? (and if X does not have mean 0, then the market is not impossible to beat).

You can dress this up a bit: suppose price(t+1) – price(t) is the random variable X + [(.001) x t’th digit of pi] – (.045). Unless something very weird is going on with the digits of pi, this version of the stock market would also satisfy the random walk hypothesis. But unlike the pure random walk it’s a market where you can make some money; if I’m lucky enough to have access to the “digits of pi” rule, I can make a small average profit. So the random walk hypothesis can hold for markets that are neither efficient nor unbeatable.

Sure, in this case future increments don’t have mean zero (indeed, they are deterministic). So certainly you can beat the market. Is there anything about this case that is fundamentally different from the “increase by 1 cent every day” case?

James

Sure, in this case future increments don’t have mean zero (indeed, they are deterministic). So certainly you can beat the market. Is there anything about this case that is fundamentally different from the “increase by 1 cent every day” case?I take “beat the market” to mean “do better than someone who buys and holds,” so in the case where the price increases by 1 cent every day, you’re making a profit, but not beating the market.

Presumably over long time-intervals? Over short times, it seems clearly false. If I know that a share has lost 50% of its value in the last hour, I will certainly expect some further wild swings, up or down or both, in the next hour too.Great question — I

thinkthat at least some version of EMH requires that volatility also be unpredictable, since it can be bought and sold; but I know nothing about the empirics.Anyway, you are not missing the point

Above you said that the weakest form of the EMF _was_ the random walk hypothesis?That’s the standard terminology, I think. But it seems to me that the terminology is misleading, in that the claims of the random walk hypothesis makes about the market are a lot weaker than anything you’d want to call “efficiency.”

Why do you say that the stock price is not the best estimate of future price in this case? If X has mean 0, and we don’t yet have any information about X, then surely the current price _is_ the best estimate of the future price?Another really good question; the way I wrote this was very confusing (reflecting my own confusion.) You’re right that it’s the best estimate in the expected value sense — but it’s arrived at without any rational actors actually estimating anything, which I think is against the spirit of the EMH. The kind of model I have in mind is that where, every day, some big sum of iid random variables perturbs everybody’s “mood” which induces them to increase or decrease the prices they’re willing to pay; this is a purely irrational process which produces the same kind of random walk behavior as the aggregation of rational choices envisioned in EMH.

The digits of pi puzzle is intriguing!

Let me suggest a simple model to tie things down a bit. Suppose that there is a large group of traders whose demand for trading is exogenously rigged to be such that the market equilibrium prices follow the process you give.

(They’re not trading for any informational reasons, but to fulfill some intrinsic desire — a hedging need arising from another market, say, or just for fun. This may seem like an immediate departure from the “rationality” assumption — that’s not what you wanted! But, as Quiggin mentions, it turns out that the “rational” finance models need such traders in order to overcame the famous “no trade theorem” of Milgrom and Stokey, which says that without them, trade would not happen at all. In a world with fully rational traders who have the same goal (to make money), no exogenous desire to hold the stock, and possibly different private information, the fact that you want to trade with me means — after a lot of hard thinking — that I don’t want to trade with you. So you need some exploitable guys around to make trading happen in equilibrium. This is a dirty little secret about the rational actor model — more about it at the end.)

There are also some rational traders with no intrinsic desire to trade who would like to exploit the other traders, but haven’t figured out the digits of pi rule, so currently they have nothing to do — the process seems wholly unpredictable to them.

A first question is why you haven’t sucked out the pattern all by yourself. After all, if you could borrow enough money, you could turn your small profits into big profits, and in the process would bid the price up or down by exactly the amount that would remove the pattern. In your story, you are implicitly assuming that you are too small to move the market, so you must not have enough credit. Then, as soon as the cat is out of the bag, (assuming that together you and all the other rational traders collectively have enough money, of course) the pattern would be sucked out and the price process would follow its “truly random” part, minus its now commonly known deterministic component.

Of course, this kind of end-runs the puzzle by assuming a simple structure about where the original pattern was coming from, and with the two-groups feature: “moody guys” and “informed guys”. But, as I said, it turns out that this is an essential feature of all rational finance models (moodiness is just sold as “intrinsic trading need”).

The seminal micro-founded model of trading is the Kyle model (3500+ citations!). I’m told it’s a tour de force of economic modeling: the moody guys and the informed guys fit together in a magical way to produce beautiful, random-walk price behavior that incorporates all private information — satisfying the super-strong EMH.

But, of course, it is an article of faith that the traders should fit together this way, and that the price should behave as it does for “good informational reasons” rather than “completely random reasons”.

So, to sum up a rather long-winded comment (sorry!) your “confusion” is quite deep and sensible. You need moody guys to make rational models tick, and the stronger forms of the EMH turn on certain dearly held modeling assumptions about them and how they fit together with other guys.

To me, the real puzzle is the incongruity between these hefty conceptual challenges (which I think are understood and respected in academic finance) and the rhetoric that one sees about the EMH!