Monday, October 31, 2011

Real Investors Lag Indices by 6%


I recently discovered there is considerable amount of data documenting how much investors underperform indices, and it's being ignored. In the 2008 Journal of Pension Benefits, N. Scott Pritchard documented that that individual investors have done much worse than the indices that everyone assumes reflect investor returns. He looked at data from 401(k) plans, and found that from 1988 through 2007, while the S&P 500 returned 11.81 percent annually and Treasury bills returned 4.53 percent, the average investor achieved a return of only 4.48 percent.

Pritchard relied on the annual Dalbar study, which consolidates data from the Investment Company Institute, and is availble for investment advisers as a way to show them what the 'conventional wisdom' is on asset allocation and investor performance. More recent data found that over the twenty years ending in 12/31/2010, the average annual equity return for investors was only 3.27%, while the S&P500 was 9.14%.

So, this 6% investor underperformance you would think would be very interesting news, because the risk premium is one of the most important facts in all of economics, being the subject of thousands of research pieces, but instead it has had about zero impact. No one finds it interesting because it is not useful to academics or investment companies.

To put this into perspective, one of the most important research findings in twentieth century finance was when two professors at the University of Chicago, James H. Lorie and Lawrence Fisher, created what has become the preeminent database on stocks in the United States, what is now known as the Center for Research on Security Prices (CRSP) database.

The front page of the New York Times financial section heralded the pair’s findings, and their Journal of Business article reported the average of the rates of return on common stocks listed on the NYSE was 9 percent for the NYSE from January 30, 1926, to through 1960. Included in the NYT article was a flattering picture of them in a room with a big computer, emphasizing that this was very scientific. Interestingly, if you read their paper, you will see how academics bury their lede by noting that this final result is not conspicuous, highlighting that academics like to emphasize their technique, not the results.

Now, this was 3% higher than the average annual return on the Dow Jones index, but with dividends added, about the same, so why the big deal? It was important because while this old data probably still contains large amounts of survivorship bias, it meticulously corrected for all sorts of other issues and therefore added the patina of academic rigor to a fundamental financial constant: the equity risk premium.

This result stood for decades, and led to many to think the equity risk premium was on the order of 6-8% up to the internet bubble of 2001. It validated the new 'risk premium' paradigm that was being created in the early 1960s, where risk, properly measured, generates an observable return premium over time, because without higher expected returns no one would invest in risky assets. When combined with a stock's beta, it generated the expected return for each asset, making a seemingly qualitative problem one amenable to linear programming.

Much has happened since then, most importantly, the poor returns since 2000, which have reduced most estimates of the equity risk premium to around 3.5%. This excludes transaction costs, and adverse timing, which are captured in the Dalbar data. And of course this excludes taxes.

18 comments:

Anonymous said...

When you say the "average investor" do you mean the median investor?

The average investor earned the market return minus transaction costs no?

Jim Glass said...

The average investor earned the market return minus transaction costs no?

No. A great many investors enter the market after it starts rising, missing part of the rise, chasing returns -- then stay in it after it starts falling, catching the fall from the start until they finally bail. Then they are out when it turns back up ... repeat. So their final return is much lower than the market's. The amount invested in mutual funds lags the market, so there's more invested in them when the market turns down than when it turns up.

Many do the same thing within different market sectors, moving between them while buying high and selling low repeatedly, with much the same result.

And of course they pay commissions and fees on all these in-and-out decisions.

The joy of market timing.

J said...

What the losing investors lose, the winning investors must win. At the end of the year the difference must be equal to what the index did.

There must be some explanation. Let wait till Falken explains it to us.

Eric Falkenstein said...

indiv investor returns equal the index minus transaction costs, adverse timing. The index assumes transaction costs, and adverse timing effects, are zero. This simply isn't true. Now, one can make them very nearly zero, but in practice, people don't.

zby said...

But what is adverse timing for some investors is perfect timing for their counter-parties, what the first lose the others gain. So do you mean that the counter parties that gain on the poor timing are not investors but rather those who create the initial stocks?

Eric Falkenstein said...

adverse timing is measured corporate distributions (dividends, bankruptcies) and offerings (IPOs, SEOs).This does not wash out. See Ilia Dichev's work.

Matt R said...

The difference is between individual investors (401(k)'s) and professional investors, right? It's sad to think that all of Wall Streets' profits above and beyond the normal stock growth and dividend payments come from individual investors dumb investing decisions. The pension model wasn't perfect, but 401(k)'s are far from perfect, too.

fr said...

I didn't read the report, so I'm not sure what is meant by "stocks". But the SP500 index definitely takes into account creation and destruction of stocks. That is, whenever the index changes, the index must sell some stocks that are currently in the index to buy new stocks entering the index. Thus for every dollar that market-times SPY and loses, another dollar must market-time SPY and win. If "average" investors are losing big by market-timing, someone out there is winning big by taking the other side of these losing trades. I cannot believe that all of the loses come from the bid-ask spreads, which are negligible with SPY. Someone must be market-timing SPY so as to get very high returns.

Eric Falkenstein said...

S&P stocks changes in capitalization over time, independent of its change in market value: distributions and issuances. Further, the average spread is much much greater than that for the SPY, which is a very small fraction of the equity market investors actively trade in. You can't assume that the SPY is 'the' market, because it isn't: many investors own specific stocks.

Jim Glass said...

What the losing investors lose, the winning investors must win.

Investing is not a zero-sum game. This is as true on the downside as on the upside.

fr said...

SPY tracks the SP500 pretty well. It certainly isn't lagging by 6%. Nor is VTI, the Vanguard total stock market index, lagging the Wilshire 5000 by that much.

French did an analysis (http://qed.econ.queensu.ca/faculty/milne/322/ECON322%282008%29%20Kenneth%20R%20French.pdf) of the costs of active investing and concluded the cost was about .67% per year for the market as a whole.

Unless this dumb money in 401k is a very small part of the market, so that the 6% they are losing is fully accounted for by the .67% the active investors as a whole lose, then some smart people out there must be making money market-timing. A fact you don't seem to want to admit because it conflicts with your ideology that no one can be smart money.

fr said...

As an aside: note that Madoff is in the list of people who aided French is compiling data for his study!

SRSFinance said...

@Jim Glass "Investing is not a zero-sum game."

Because credit expands market size?

Anonymous said...

This kind of great info is why I read this blog.

David said...

Very interesting idea of the average equity investor being way behind the index return. Like Jim Glass, I can't quite wrap my mind around that.

Using a really simple example, suppose the stock market index is worth $1000 at time 0, $2000 at end of year 1, and $2000 at end of year 2. All the gains in the first year, second year trading flat.

Investor 1 buys the whole market in time 0 for $1000. He sells $500 of that at end of year 1 to investor 2 for $1000, and keeps that in cash.

Return of the index = 100% over 2 years or 42% a year.

Return of investor 1 = 100% over 2 years (he has $1000 in cash and $1000 in stock at the end) or 42% a year.

Return of investor 2 = 0% over 1 year.

So the average return is way under the index return? Because investor 2 missed out on the big first year and only jumped in the flat years? While investor 1 didn't gain much from his timely sale because he doesn't have much of a choice other than hold cash (buying back into stock market would have gained him the same 0% return in the 2nd year)?

Where did the "implied" money go then? It's kind of a mathematical illusion.

David said...

Also, another thing that has never been completely clear to me is how much these annualized Dow or S&P 500 returns properly account for survivorship bias. There are companies in the index who have completely gone of business during the study period. Also, new companies have been added in.

Does the return in the index take into account the loss of value in owning shares of those defunct companies? And what price do new companies buy into the index at? Or is the index a weighted average of the price of X companies at a particular time, where X gradually changes over time.

Jim Glass said...

Like Jim Glass, I can't quite wrap my mind around that.

Jim Glass has no problem wrapping his mind about it at all.

... Where did the "implied" money go then? It's kind of a mathematical illusion.

You are missing the point that people enter and leave the market. E.g., mutual funds issue and redeem shares. People chase returns, so there are fewer investors and shares issued at a market low after people have sold out during a fall, and more at a market high after people have bought in during a rise.

So at a market bottom there are say 20x investors who enjoy the ride up to the high, at which point there are say 30x investors who enjoy the ride back down back to the original point. The mutual fund's price returns to its point of origin at which its price change is $0 net. But as there were more investors going down in it than there were going up, the average investor in the fund during the time period has suffered a loss.

This occurs not only as people go into and out of the market, but also as they run from one specialty mutual fund to another, chasing after returns by buying hot market segments high and selling them low, repeat.

Etaoin Shrdlu said...

The results obtained by 401(k) investors are not at all necessarily equal to those of all investors. Fees and fund expenses are relatively high, and disclosure is disfunctional. The participants are typically quite ignorant of how to invest. Market timing and performance chasing, you name it, it's all over. There are people who spend most of their careers with their money in stable value funds. It is a wonder people do as well as they do.