Last week I noted there are several reasons why people over invest in high volatility stocks: overconfidence, investors chasing extreme performers, winner's curse, cheaper information, alpha discovery, lottery preferences, representativeness bias, and signaling (many of these are related, but slightly different, as where signaling is related to investors chasing extreme performers, but there are different assumptions about what is exogenous). The bottom line is there are many factors 'outside the box' that skew the traditional mean-variance preferences, and researchers such as Baker, Bradley, and Wurgler (2010) and Robert Haugen have pointed out this may underlie the low returns to highly volatile stocks. As highly volatile stocks are risky by any metric--default probability, beta--this creates an anomaly that exists not only within equities, but corporate bonds, currencies, options, everything. This creates a major problem because if an equity premium exists due to 'risk', this risk does not generalize.
A consensus is forming that risk premia exist between asset classes, but not within asset classes due to two confounding effects: risk aversion and the investor preference for highly volatile stocks. This cannot stand as the sole explanation. If there's an equity risk premium between classes like bonds and stocks, but not within bonds and stocks, any investor in broad indices necessarily also holds a lot of the crappy high risk assets without any of the benefits. The benefits listed above work only for picking specific stocks or subsets of stocks.
As many sophisticated and large investors do invest in broad asset index strategies, how can this make sense? I think it is because people are inveterate benchmarkers. If risk is a return relative to what everyone else is doing, the indices are low risk. You can show this using a utility or arbitrage argument, but the idea is pretty simple.
While my argument in this vien has never made it into an academic journal, the idea is bubbling around in piecemeal applications. Abel (1990) first showed that a ‘keeping-up-with-the-Joneses’ utility function--one that defines itself relative to others--the equity risk premium would be smaller than otherwise. Gali (1994), DeMarzo, Kaniel, and Kremer (2004), and Roussanov (2010), all show how relative utility preferences can affect cross-sectional asset pricing, basically lowering the risk premium for highly volatile stocks, if not making it negative. These papers tend to be very qualified and parochial, but that's only because the status quo is too entrenched, so they take baby steps. Yet, a relative status orientation can be efficient for an individual, as modeled by Rayo and Becker (2007) or Pesendorfer (1995). The relative status utility function generates a more accurate description of the world, as people generally prefer to live in societies where everyone earns less but they earn relatively more, than in societies where everyone earns more but they earn relatively less. Adam Smith's seminal Wealth of Nations is as consistent with a utility function based on envy, as he is on one based solely on greed.
The preference for low volatility assets is not directly related to the relative utility preferences. The factors that lead one to highly volatile stocks are pretty independent from utility functions. It's just that the effect of this love for high volatility does not get arbitraged out because it's too risky. Shorting high volatility stocks is risky, and because of benchmarking, merely under weighting highly volatility stocks is risky too.
The bright side is, if you can trade your envy for greed, there's a simple way to create a better portfolio. An investor who prides himself on maximizing return relative to risk, and ignores the 'keeping-up-with-the-Jones' effects, would do well to invest in low volatility equity portfolios that avoid those lottery ticket stocks.