Tuesday, June 09, 2009

Myth of the Rational Market

Most non-economists find the 'efficient markets hypothesis' the most absurd belief that most economists believe. The latest broadside is Justin Fox's book The Myth of the Rational Market. A lot of this gets into semantics. If you think efficient markets mean they are always correct, then clearly this is a stupid theory. But it only means that the market price is an unbiased predictor of future prices, conditional on all the current information (there is a risk premium that complicates this, but I write about this in my upcoming book and will talk about that when released in a few weeks).

I haven't read the book, but there appear to be a lot of straw men suggested by the book jacket. It states "The efficient market hypothesis has evolved into a powerful myth". So, markets are predictable? Do tell. "A new wave of economists and scholars who no longer teach that investors are rational or that the markets are always right." Who were these guys saying markets are always right? "Investors overreact, underreact, and make irrational decisions based on imperfect data." Collectively? In what cases? Half?

Here's my defense of Rational Markets:

Think of the problem this way. Say one can buy a contract that Global Warming implies temperatures will be 3 degrees higher in 2100. Assume it were a traded contract, such that there was a way to generate some validation, say, that average temperatures in the troposphere done by NASA, paid to legal beneficiaries of current bettors. Now, people strongly disagree on this, and most people think those who disagree aren't merely making an honest mistake, but have biased or stupid beliefs, though unintentionally (tools of bigger forces, a malevolent Borg), yet the key is you cannot prove which side is wrong via indisputable logic today. The set of information is large, and it is not clear what is relevant to this forecast (climate models are very complicated). In 90 years, with hindsight, the losers will look like stupid ideologues, and that will pertain to a significant number of otherwise smart people. Is this market then 'inefficient', because those taking the other side of a losing bet will be not merely unlucky, but 'wrong'?

I think not. Truth is not obvious. People do their best, and usually the phalanx of assumptions and theories that underly a belief are so comlex you can not fully articulate why you believe something. That does not mean your belief is irrational, just that in the real world, many things are very complicated, and you can't work backward to isolate essential differences. Even if you could, there would be many assumptions that are also really unverifiable opinions, not that they don't have data, just like saying the minimum wage causes unemployment, you don't have enough data to prove it one way or the other to a suffficiently skeptical person. So it's an infinite regress. Are these disagreements, manifested in markets where prices change all the time, sometimes violently, irrational? It would be nice if we all could agree on the facts and theories, and that they be correct, but that's rather naive.

The bottom line is that most investing experts underperform passive benchmarks, and counter-examples like Warren Buffet are merely reused again and again. Sure, some may have alpha, but they are relatively few, and almost all of them follow the Peter Principle in finance and accept money until their alpha is gone (the dominant strategy, it appears). That fact has held up pretty well since first discovered by the Cowles commission back in the 1930s. I would like to see the final paragraph of every book touting the stupidity of rational markets to give specific advice, like "buy gold, GM, Sell the VIX and the dollar", or that investors generally overreact. That is, take an actual position, not a can't-be-wrong position. To say the market is biased, but you don't know which way, is equivalent to saying it is unbiased, and that it will fluctuate. The fact that people don't know the sign of the market's incorrectness is the reason we call the market efficient.

A good example of what seeming market inefficiency looks like in practice is the appearance and disappearance of the ‘convexity bias’ between swap forwards and Eurodollar futures. The arbitrage worked like this. If future and forward rates were equivalent, one could go long swaps (forwards) with short Eurodollars (futures), and the daily mark-to-market of the Eurodollars vs. the future mark-to-market of the swap would allow one to lock in a sure thing. In equilibrium, precluding arbitrage, futures should be slightly higher than forward rates. The mechanism underlying this opportunity is subtle, but the effect added up to 15 basis points in present value if done with 5-year swaps, and it was truly risk free. Several banks made tens of millions of dollars on it in the early 1990s. It was written up in RISK magazine in 1990 (Rombach, 1990). It disappeared around 1994, after which academic economists wrote about it in the Journal of Finance (Grinblatt and Jegadeesh, 1996), and today you see the convexity adjustment right there in Bloomberg so traders don't think forwards are futures.

That 'inefficiency' is history. True arbitrage profit opportunities such as these do exist, and they don’t disappear immediately, but they do go away eventually, usually well before academics have proven they are arbitrage. Market efficiency skeptics would see this as a fantastic embarrassment, I see it as disequilibrium phenomenon, a temporary aberration that is of interest only to those lucky few who identified it while it still conferred a profitable opportunity. Disequilibrium behavior has always been difficult for economists to explain, as all such activities are idiosyncratic or ephemeral.

The problem for those who think the market is irrational is to generate a model that is better. To merely state, with hindsight, that people were overreact in one case, underreact in another, leads to an unbiased market in real time, and it is unbiasedness, not zero price variance, that is the essence of the efficient markets hypothesis.

The behavioralists like to portray themselves as rebels, Davids versus the theoretical Goliath, but in reality the efficient markets folks deserve the true rebel status. Almost everyone outside this literature is sympathetic to behavioral theories over the rational markets assumption, regardless of one’s political bent. I used to work for a bank where our swap salesmen could always sell swaps to companies by telling their Treasurers how these instruments could make money given their personal view of the market.

To these Treasurer's detriment very few of them had an efficient markets prejudice (“perhaps the forward prices and their implied volatilities are as good a forecast as mine, and so including commission this is a negative-sum speculation!”). It has been well documented by people like Odean and Barber that people trade too much, based on the mistaken idea that markets are not efficient. It would be harmless fun, but it's a lot of money. Sure some are right, but most are burning money via transaction costs in zero-sum bets.

One doesn’t have to love Dilbert to know that there are lots of irrational business people, and that many business decisions are made by former B- students who are both boundedly rational and internally inconsistent. The real question is whether or not these irrational actions generate useful hypotheses about economic behavior, and thus far most of these predictable actions relate to volume and volatility. The over and underreaction hypotheses seem about as promising as the adaptive expectations assumption that underlies it, and Keynesians worked with that for years without bearing fruit.

The efficient markets paradigm is a triumph of economics because it is so counterintuitive to the layman, so restrictive in what it allows, and so pervasive in its application. A healthy respect for the rationality of markets is a hugely advantageous mindset for the researcher and practitioner. This is the most useful base from which one identifies anomalies, and then explains them with specific frictions or cognitive biases. If you start out thinking all prices are wrong, odds are you are gambling, and the house wins.


J said...

Your new book seems interesting and I am going to buy one. Why are "sociopaths" those like "risky" nondiversified investments? Sociopath (Antisocial personality disorder - ASPD) is defined by the American Psychiatric Association's Diagnostic and Statistical Manual as "...a pervasive pattern of disregard for, and violation of, the rights of others that begins in childhood or early adolescence and continues into adulthood." I think it is profitable to have a healthy disregard for others, specially in the investing business.

J said...

If you start out thinking all prices are wrong, odds are you are gambling, and the house wins.

Most prices are, by definition, wrong. Prices include a strong component of future expectations, and being the future unknowable, current prices must be wrong. The price of ICL (a corporation or something) last week was wrong, as shown the price today (which is much higher, alhamdulillah).

paul said...

Eric, excellent critique as usual. I look forward to the book.


But What do I Know? said...

"Accept money until their alpha is gone." Good one!

N N said...

I'll be another buyer of the book based on the excellent work in this blog.

BTW, Amazon shows used books available even though it hasn't been published?

Eric Falkenstein said...

regarding 'sociopath'...I was thinking 'a person, as a psychopathic personality, whose behavior is antisocial'...my argument being that all 'normal' people should not like risk in general.

Chris Wicks CFP said...

Whilst I agree with Eric's basic viewpoints regarding the operation of the markets, I do think that to comment on a book, having just studied the fly cover, or at least to confess to having done so, is more than a bit intellectually embarassing. If he tried that in a college seminar he would have instantly and quite rightly been shot down in flames, irrespective of the merits of any other observations he may make.

Anonymous said...

by creating mental fictions, we can free ourselves of responsibilty and perhaps even conscience.

corporation are on such fiction. we ascribe human-like qualities to them, but they are not people. they have little or none of the personal responsibility individual people have.
we can hide behind them, use them to give us credibility or use them as targets to express our disatisfactions with individual people who work for them who we briefly engage.

markets are another fiction. they allow us to depersonalise commerce.

instead of talking about our own choices and actions and the people we know (=our world), we can talk about "the market".

it is a way of escaping our immedaite world, i.e. the people we come in contact with. how we interact with them is how we affect the "big picture" (perhaps "ecosystem" is the buzzword).
everyone has an effect.

but by talking only about fictions, like markets, we can avoid focussing on the person with we are directly interacting.
we can apply our opinions and frustrations (or elations) to these fictions instead of to the people with which we interact. we can ignore the most efficient ways to affect the "ecosystem".: e.g. to place our focus on each person we interact with, one at a time.

i see many people focussing on fictions and ignoring the people in front of them (effecting larger scale change to their lives through their incremental, individual interactions), perhaps because it makes things easier.

talking about "markets" eases tensions of having to deal with one's environment directly.

Anonymous said...

to me, rational markets means "i realise that to some degree we are all automatons controlled by parts of our brain we do not have conscious access to, and, i accept this. i will consider whatever happens that is out of my immadeiate control to be rational."

does anyone want to see themselves as an animal, outside of certain circumscribed situations where we are "allowed" to show that side of us?

i think not.

if believing the uncontrollable is "rational" makes one more at ease, then embrace it.

we can, however, make choices and accept responsibilty for them. as opposed to continually looking outside our immediate world to the idea of "the market". the constant looking to "the market" reminds me of the peoples of antiquity looking to "the gods" to explain the occurrences they observed in their world.

economics, like journalism, as a "prfessional field" could be in trouble, the more people learn about the world through our newfound technolgy enabled worldwide connectness.

psychology, however, i think could be a field set to expand exponentially.

Anonymous said...

To quote you:

"I haven't read the book, but there appear to be a lot of straw men suggested by the book jacket."

Seriously?!? Do you not see the hypocrisy of that statement? Talk about straw men. If you haven't read the book and yet spend so much effort trying to counter what you imagine it must be saying, then you, sir, are the one relying upon straw men.

You should read it. It's not a diatribe, but rather an historical perspective on some competing views in economics and finance.

Michael Meadon said...

It's a Bad Thing to review (or even have opinions about) books you haven't read. Critiquing a book based on the dust-jacket is unfair: often authors don't have much control over what goes there.

See Cosma Shalizi's superb review of the book, here: http://www.americanscientist.org/bookshelf/pub/twilight-of-the-efficient-markets