Overconfidence vs. Optimism

In 1971 Robert L. Trivers gave us the idea of reciprocal altruism, the idea that helping someone else although incurring some cost for this act, could have evolved since it increases the chance that such niceness might be returned some day. This really expanded the conception of self-interest to be semantically indistinguishable from altruism in many cases. Biologists have observed this behavior in bats, where they share blood harvested, but only with those who share with them.

Trivers has some great anecdotes in his new book The Folly of Fools: The Logic of Deceit and Self-Deception in Human Life. In one, people's pictures are distorted to make people better and worse looking than they really are, using data on what 'good looking faces' look like. Looking at fMRI data, we actually recognize ourselves in pictures faster when we are about 20% better looking than we really are (they had models that generated statistically validated better and worse refinements to pictures). In other words, we really think we look better than we actually do.

In another study, children were told not to look in a box that contained intriguing toys while a researcher left them alone. Most of the children did look, of course, and most lied about it. Interestingly, the higher the IQ of the child, the more often they lied.

In this TED video, Trivers argues that we continually paint a distorted picture of the world so that we might more easily get our way with others. This involves constantly inflating our acheivements and abilities, and playing down our failings and rationalizing our mistakes. You are a better liar to others if you first lie to yourself. Lying to others and ourselves is evolutionarily selected for the same way that pale skin is selected for in northern latitudes. [The video is better than the book. He has many riffs on his Chomskian view of the world. I don't mind so much, remembering that Isaac Newton was a grade-A looney]

This is important for asset pricing because highly risky assets within any asset class often have lower-than-average returns, and Edward Miller (1977), Diether, Malloy and Scherbina (2002), and Baker, Bradley and Wurgler (2011) argue that overconfidence--and short sale constraints--lead to poor return for highly volatile stocks. We know people are overconfident, where most of us think we are better drivers, leaders, etc., and one prominent mechanism is that the higher the volatility of the stocks, the greater the disagreement, the more these miscalibrated optimists overbuy, which pushes down stock returns. Thus, understanding why people are overconfident is important.

Overconfidence is puzzling in many dimensions of our lives, not just financial. Intellectually, it lacks the knowledge and honesty that comes with wisdom, where one recognizes we are neither as good as our mothers think, nor as bad as our enemies think. Thus, I've always been a little perplexed by the idea that people find confidence so sexually attractive. Confidence is something I was lacking as a young man, and generally I would need several beers to initiate various courting rituals. While I understood that confidence was sexy, something I wanted to be, I couldn't fake it because I couldn't understand why something was both ignorant and desirable.

Looking back, I had it all wrong, but it my defense, it flustered good people like Woody Allen and Allen Brooks too. Girls don't like overconfident boys merely because they miscalibrated. The key is the nature of one's overconfidence. People who are confident about the future continue trying, even when it's hard. They assume the good faith of others, which makes for good first impressions. People like optimists because they tend to be less defensive, less easy to offend, and so easier to be around. Optimism is related to positive reframing and a tendency to accept the situation's reality, because optimists see the silver lining to problems. Optimists are copers, and pessimists defeatists. It is not confidence in specifics, but in the future, their future, that makes people attractive, and this is a rational disposition. A confident person need not be miscalibrated, though they often are, but rather, their confidence is really simple optimistism about their life in general.

Thus, our instinct to be overconfident is a salutary disposition, suitably framed. We should see the future brightly because if we simply do the best we can things will work out as well as they can. To the extent things are out of our control and work against us, it doesn't help us to worry about such matters. Insecurity on the other hand leads to pettiness, which is why we find such characteristics so unappealing (eg, insecurity led Anakin Skywalker to The Dark Side, and presumably more genocides than 100 Hitlers--he blew up an entire planet to make an example!). Like any virtues optimism and confidence are useful only in thoughtful moderation, in the right context.