Don't Innovate Too Much

Dick Thaler gives out some advice to econ students:

My advice for young researchers at the start of their career is… Work on your own ideas, not your advisor’s ideas (or at least in addition to her ideas).

I think this is bad advice.  Kids love to think that they can do it all themselves in an Ayn Rand fantasy, independent of others.  Yet, to succeed is to succeed socially, because we are social animals, so we need not just good ideas, but good ideas people are receptive to. This means, understand what customers/readers/users want, and what colleagues want to help you with.  Sure, you can do it yourself with sheer brilliance, but it's infinitely more difficult than simply extending someone else's work.

 In the book I’ll Have What She’s Having, the anthropologist authors argue that we mostly copy everyone else: first our parents, then our peers, then anyone who seems to be doing well. Generally, it works, as if you want to figure out how to ride the subway, you could do worse than stand back and watch everyone else. Evolutionary biologist Mark Pagel takes this a step further by stating emulating others is the basis for almost every idea we have, and notes that very few of us create things that are later used by many others; think the number zero, the wheel, soap, iron, the alphabet. Kids aren't good at listening to their parents, but they are predictable imitators because that's a good strategy.

We are fundamentally an imitating species. Most innovators spend their formative years producing derivative work: Bob Dylan's first album contained 11 cover songs, comedian Richard Pryor began his career doing imitations of Bill Cosby, Hunter S. Thompson retyped The Great Gatsby just to get a feel for writing a great novel, and most of Freud's theories were riffs on Greek literature and Shakespeare.  We copy to acquire knowledge that becomes the foundation for variations and extensions that appear to outsiders as thinking outside the box. As Isaac Newton said, we stand on the shoulders of giants (itself, a rephrase of a saying by 12th century French philosopher Bernard de Chartres)

You can't make variations on a theme without understanding that theme, and having the theme your are riffing on well-accepted already.  Personally, I wrote my dissertation on what interested me, low volatility stocks, and thought it was fascinating.  It was a total flop--I got zero flyouts--mostly because it had no supporters in our finance department.

You need mentors and advocates.  Sure, there's the occasional Ken Arrow with his dissertation on the impossibility theorem, or John Nash's theory on, well, Nash equilibrium, but those are  exceptions.  Most successful academics published their first paper by merely extending their dissertation advisor's ideas (Joe Stiglitz edited Paul Samuelson's collected works as a grad student, giving him a good feel for how a paper should be presented). As Thaler is mainly famous for some papers finding that 3-year winners and losers mean-revert, I think his main take-away is that it helps to be lucky, because those papers were seriously flawed and wouldn't have made it out of many incubators (see here), and that result has been orphaned because it doesn't hold up.  For example, the low-priced bias was so large, there was a January bump in portfolio returns 12 and 24 months after the portfolio formation period as these low-priced 'loser' stocks bounced from their bid of 1/2 to their ask of 5/8 each January.

Back in the 1990's, successful students simply applied the latest technique (eg, vector autoregressions, generalized method of moments), using TeX, which then was instantly credible because it looked a lot like what those top economists were doing. Add a top prof as your advocate because you are adding a reference useful to his reputation, and you have job offers. Copying form is probably more important than substance, because it's not only easier to do, but easier to evaluate, and as much as decision-makers say they love innovation, they just mean, the kind of innovative work they and their circle of colleagues are doing (and, of course, Einstein, Kepler, etc.).

I really like this old piece by Robin Hanson on The Myth of Creativity:

To succeed in academia, my graduate students and I had to learn to be less creative than we were initially inclined to be. Critics complain that schools squelch creativity, but most people are inclined to be more creative on the job than would be truly productive.