Saturday, September 11, 2010

Is Levitt's Abortion Study as Good as it Gets?

When someone questioned the value of economics, Tim Harford tried to counter with an example of an abstruse economic result that was important, and came up with the following
What, for instance, of the famous contention by the economist Steven Levitt and his co-author, John Donohue, that legalised abortion in the US reduced the crime rate about 18 years later? This is a hypothesis about history, but one that no historian is well-qualified to judge. Instead, the hypothesis has been tested statistically with some ingenuity; the statistical models themselves have been contested, pulled apart, found wanting in some respects, double-checked using alternative data and tested against the experience in other countries. The debate continues. Is this process “science”? I am not sure. But it certainly isn’t idle banter.

Is the Freakonomic's signature study the best example of clever and important economics? That example was also mentioned when Russ Roberts asked economists to name an empirical study that uses sophisticated statistical techniques that were so well done, it ended a controversy and created a consensus.

Levitt's abortion paper is a result celebrated by many for reasons they would never admit--that abortion is really eugenic because given the option people won't have babies they anticipate they cannot adequately nurture (Planned Parenthood founder Margaret Sanger was a big eugenicist). The Levitt abortion paper seemed to document this, showing via state data that the relative difference in crime rate changes post 1973 when abortion was legalized in all states corresponded to the abortion dummy. Freakonomics 2 has been much less well received, I think in large part because it had no such useful rationales (prostitutes are lazy and like their work? Mean and gross!).

Any complex assertion, such as that abortion lower crime, involves many different forces affecting cause and effect independently. In this case, the secular crime rise from 1960 to 1975 that occurred internationally, the crack crime peak around 1990 in the USA, and demographic trends.

The bottom line is that any fact should be discernible through several independent lenses. In Levitt's case, he used the fashionable tactic of an 'instrument', that different states had different abortion laws (illegal in some states, not in others, prior to 1973). That gets you kudos among economist who think models are more important than their application, in this case a fad that started with Angrist and Krueger's estimation of education effects via instruments, highlighted in Mostly Harmless Econometrics. But does the effect stand up to more mundane corrections, such as looking at arrests on a per-capita basis as opposed to mere numbers? No.

Consider that the peak years for serious violent crime by 12-17 year olds, as reported in the FBI's authoritative annual survey of crime victims were 1993 and 1994, well after abortion was legalized across all states in 1973. How does that square with the hypothesis? Obvious it does not, but the modus operandi of the 'cute instruments' crowd is to look at the data via one very clever test, and ignore independent views such as the aforementioned issues. It's pretentious intellectually bullying, and profoundly short-sighted because in the long run, no one remembers clever papers that backed incorrect or irrelevant hypotheses like that abortion is eugenic.

Levitt notes that following the legalization of abortion in the USA across all states, conceptions rose by nearly 30 percent but births actually fell by 6 percent, implying most abortions merely soaked up increased conception, an alternative form of birth control. Birth rates did not change much in responding to changes in abortion rates (birth rates and abortions post-1973 peaked in 1990). Abortion appears more common among forward-thinking moms who would be good moms, less common among bad moms who view life as a series of random events that happen to them, making it dysgenic (i.e., the Levitt-Donahue's hypothesis is 180 degrees wrong).

Another inconvenient data point to the Levitt-Donahue thesis is that teen black homicide rates have been much higher (9x) than teen white homicide rates, and peaked in the early 1990s. After abortion was legalized the black fertility rate declined more, and the black abortion rate rose more, which should have implied a lower black-white homicide ratio, via increased eugenic behavior among blacks. Instead, the differential worsened.

The best empirical work looks at a variety of data, as opposed to being very clever with one set of data. Robustness comes not from standard errors as much as noting that if 'X' is true, it should show up in various statistics, not one test on one set of data. There are important issues in the data, like how you measure crimes, or how punishment for crime changed over the period. This latter work isn't as interesting to economists because it's so parochial, and economists want to find eternal, generalizable truths like Newton's laws because that's what scientists do. This makes them truly academic, uninterested in facts and issues related to messy, real data because such facts do not play into their comparative advantage, which is the slick econometrics.

So, if Harford thinks that Levitt and Donahue's abortion paper is the best example of what economics can do for us, he might as well admit economics is basically sociology with more math, giving it smarter practitioners, but they are no closer to the truth when they actually apply to the data.


Caveat Bettor said...

Great post. 'Hartford' should be 'Harford'.

Stilts said...

Steve Sailer addressed this in a back and forth exchange with Levitt back in 1999. Levitt didn't address all the concerns.

Lance said...


How about the weak and semi-strong contentions of the efficient market hypothesis?

Samuelson's & Fama's initial theoretical articles somewhat laid the groundwork for further empirical work that slowly pushed people to accept that, in general, stocks reflect all publicly available information and there is no systematic manner in which to "beat the market".

It seems the professional investment community has accepted these findings, given their focus on beating certain benchmarks, such as the S&P 500.

Whether people consciously think about the empirical studies confirming this is another matter, however.