Sunday, November 27, 2011

Overconfidence vs. Optimism

In 1971 Robert L. Trivers gave us the idea of reciprocal altruism, the idea that helping someone else although incurring some cost for this act, could have evolved since it increases the chance that such niceness might be returned some day. This really expanded the conception of self-interest to be semantically indistinguishable from altruism in many cases. Biologists have observed this behavior in bats, where they share blood harvested, but only with those who share with them.

Trivers has some great anecdotes in his new book The Folly of Fools: The Logic of Deceit and Self-Deception in Human Life. In one, people's pictures are distorted to make people better and worse looking than they really are, using data on what 'good looking faces' look like. Looking at fMRI data, we actually recognize ourselves in pictures faster when we are about 20% better looking than we really are (they had models that generated statistically validated better and worse refinements to pictures). In other words, we really think we look better than we actually do.

In another study, children were told not to look in a box that contained intriguing toys while a researcher left them alone. Most of the children did look, of course, and most lied about it. Interestingly, the higher the IQ of the child, the more often they lied.

In this TED video, Trivers argues that we continually paint a distorted picture of the world so that we might more easily get our way with others. This involves constantly inflating our acheivements and abilities, and playing down our failings and rationalizing our mistakes. You are a better liar to others if you first lie to yourself. Lying to others and ourselves is evolutionarily selected for the same way that pale skin is selected for in northern latitudes. [The video is better than the book. He has many riffs on his Chomskian view of the world. I don't mind so much, remembering that Isaac Newton was a grade-A looney]

This is important for asset pricing because highly risky assets within any asset class often have lower-than-average returns, and Edward Miller (1977), Diether, Malloy and Scherbina (2002), and Baker, Bradley and Wurgler (2011) argue that overconfidence--and short sale constraints--lead to poor return for highly volatile stocks. We know people are overconfident, where most of us think we are better drivers, leaders, etc., and one prominent mechanism is that the higher the volatility of the stocks, the greater the disagreement, the more these miscalibrated optimists overbuy, which pushes down stock returns. Thus, understanding why people are overconfident is important.

Overconfidence is puzzling in many dimensions of our lives, not just financial. Intellectually, it lacks the knowledge and honesty that comes with wisdom, where one recognizes we are neither as good as our mothers think, nor as bad as our enemies think. Thus, I've always been a little perplexed by the idea that people find confidence so sexually attractive. Confidence is something I was lacking as a young man, and generally I would need several beers to initiate various courting rituals. While I understood that confidence was sexy, something I wanted to be, I couldn't fake it because I couldn't understand why something was both ignorant and desirable.

Looking back, I had it all wrong, but it my defense, it flustered good people like Woody Allen and Allen Brooks too. Girls don't like overconfident boys merely because they miscalibrated. The key is the nature of one's overconfidence. People who are confident about the future continue trying, even when it's hard. They assume the good faith of others, which makes for good first impressions. People like optimists because they tend to be less defensive, less easy to offend, and so easier to be around. Optimism is related to positive reframing and a tendency to accept the situation's reality, because optimists see the silver lining to problems. Optimists are copers, and pessimists defeatists. It is not confidence in specifics, but in the future, their future, that makes people attractive, and this is a rational disposition. A confident person need not be miscalibrated, though they often are, but rather, their confidence is really simple optimistism about their life in general.

Thus, our instinct to be overconfident is a salutary disposition, suitably framed. We should see the future brightly because if we simply do the best we can things will work out as well as they can. To the extent things are out of our control and work against us, it doesn't help us to worry about such matters. Insecurity on the other hand leads to pettiness, which is why we find such characteristics so unappealing (eg, insecurity led Anakin Skywalker to The Dark Side, and presumably more genocides than 100 Hitlers--he blew up an entire planet to make an example!). Like any virtues optimism and confidence are useful only in thoughtful moderation, in the right context.


Andrew L said...

A small side point: The reason people recognise themselves more quickly when they are made more beautiful isn't necessarily because they think they look better than they are. It could be because when people are made to look more beatiful than they are, they are made to look a bit more like the "average" or "idealised" person. The brain can process this image quicker because it is closer to an idealised person. If the "beautified" image is made too much like an idealised person it will loose the distinguishing features all together, and be hard to recognise, but with just a small amount of the "uniqueness" removed there may be less to distract the brain and it does it's job quicker.

If you want to find out if people mis-judge their own beauty, you need them to rank groups of people that include themselves in order of beauty, and then see how the self rankings compare to non-self rankings. I suppose this has been done. I might try to look for something on this if I get time.

Aaron Brown said...

I think you're conflating two ideas: optimism about outcomes and the belief that your efforts make a difference. A guy who thinks his behavior has no effect on his success with women will fail a lot whether he is pessimistic or optimistic. Neither one puts effort into personal appearance, sociability or attention to others. The pessimist sulks in the corner at the party, feeling sorry for himself. The optimist finds an unfortunate victim and announces that it's her lucky day.

Needing “several beers to initiate various courting rituals” doesn’t sound like an issue of either confidence or belief in effective efforts as alcohol does not change either one. If the beers are for you, it suggests an issue of utilities, as beer can increase the perceived pleasure of success and numb the pain of rejection. If the beers are for her then it is a time-honored technique, legal in moderation, to level the strategic playing field.

Among investors, people who believe their efforts have no effect on investment performance are indexers. There are optimists ("if I invest in stocks I have to get rich in the long run") and pessimists ("you can't beat the market even though its future returns are likely to be anemic with sickening volatility”). Among active investors there are optimists (“I can find great portfolios”) and pessimists (“maybe I can avoid some of the worst bubbles and frauds”). Only the active group will influence relative asset prices, the optimists are the ones attracted to volatile stocks while the pessimists will avoid them. Since volatile stocks tend to have lower returns, it suggests there are more optimists than pessimists among active investors.

However, that does not necessarily mean the optimists lose money as a group. You can make a lot of money in a volatile stock even if it doesn’t go up. You just have to get in and out at the right times. If you do, you can make money from the index investors as they have a tendency to put more money in the market after recent gains and take money out after recent losses. A passive investment in a volatile stock seems to be a bad idea, but it’s not clear whether the price is paid by passive investors or active optimists.

Mercury said...

The Psych. Today article is a little too PC. The notion that males value "confidence" in females as much as females do in males is absurd.

In general I think that the modern conception of “self-esteem” is overrated and pessimism, underrated. Pessimism can lead to defensive strategic planning which sometimes results in a better payoff. Anyone who isn’t pessimistic about the solvency of the government (and who they’re going to try and stick it to when TSHTF) needs to have their head examined.

Anonymous said...

"Interestingly, the higher the IQ of the child, the more often they lied."

It is usually the other way around. a good liar is one who has a higher IQ for reasons obvious.

Anonymous said...

The reason people recognise themselves more quickly when they are made more beautiful may be because people recall and envision themselves at their personal best -- freshly groomed, well dressed -- rather than at typical times, or times when they look like Driver's License photos.