Is Levitt's Abortion Study as Good as it Gets?

When someone questioned the value of economics, Tim Harford tried to counter with an example of an abstruse economic result that was important and came up with the following

What, for instance, of the famous contention by the economist Steven Levitt and his co-author, John Donohue, that legalised abortion in the US reduced the crime rate about 18 years later? This is a hypothesis about history, but one that no historian is well-qualified to judge. Instead, the hypothesis has been tested statistically with some ingenuity; the statistical models themselves have been contested, pulled apart, found wanting in some respects, double-checked using alternative data and tested against the experience in other countries. The debate continues. Is this process “science”? I am not sure. But it certainly isn’t idle banter.

Is the Freakonomics signature study the best example of clever and important economics? This study was also mentioned when Russ Roberts asked economists to name an empirical study that uses sophisticated statistical techniques that were so well done, it ended a controversy and created a consensus.

Levitt's abortion paper is a result celebrated by many for reasons they would never admit--the thought that abortion is eugenic because it allows parents to avoid having a child they cannot nurture (Planned Parenthood founder Margaret Sanger was a big eugenicist). But there is good reason to think that abortion costs would be dysgenic. Abortion for anticipated lack of parental resources would be common among thoughtful moms, less common among moms who view life as a series of random events that happen to them. It is not obvious how this plays out.

The Levitt abortion paper seemed to document that abortion was eugenic, showing the relative difference in crime rates declined in those states that legalized abortion in 1973. The second Freakonomics book was far less celebrated, I think in large part because it had no such useful rationales (prostitutes are lazy and like their work? Gross and mean!).

Any complex assertion, such as that abortion lowers crime, involves many different forces affecting cause and effect independently. In this case, the secular crime rise from 1960 to 1975 that occurred internationally, the crack wave created a crime peak around 1990.

The bottom line is that any fact should be discernible through several independent lenses. In Levitt's case, he used the fashionable tactic of an 'instrument', that different states had different abortion laws (illegal in some states, not in others, prior to 1973). That gets you kudos among economists who think models are more important than their application. Angrist and Krueger's estimation of education effects via instruments, highlighted in Mostly Harmless Econometrics, is considered 'best practice' econometrics. But does the abortion-crime effect stand up to more mundane corrections, such as looking at arrests on a per-capita basis as opposed to mere numbers? No.

Consider that the peak years for serious violent crime by 12 to 17-year-olds, as reported in the FBI's authoritative annual survey of crime victims were 1993 and 1994, well after abortion was legalized across all states in 1973. How does that square with the hypothesis? Obviously, it does not because the peak crime demographic should have been affected by eugenic abortions, lowering crime. The econometrics crowd is to look at the data via one very clever test and ignore independent views such as the aforementioned issues. It's pretentious intellectually bullying and profoundly short-sighted because, in the long run, no one remembers clever papers that backed incorrect or irrelevant hypotheses like that abortion is eugenic.

Another inconvenient data point to the Levitt-Donahue thesis is the black-white homicide rate.  After abortion was legalized the black fertility rate declined more, and the black abortion rate rose more, which should have implied a lower black-white homicide ratio via increased eugenic behavior among blacks. Instead, the differential worsened.

The best empirical work looks at a variety of data, as opposed to being very clever with one set of data. Robustness comes not from standard errors as much as noting that if 'x' is true, it should show up in various statistics, not one test on one set of data (though, with a really clever and abstruse metric!). There are important issues like how you measure crimes, what other secular trends were occurring (crack), or how punishment for crime changed over the period. This latter work isn't as interesting to economists because it's parochial and economists want to find eternal, generalizable truths like Newton's laws because that's what scientists do. This makes them truly academic, uninterested in facts and issues related to messy, real data because such facts do not play into their comparative advantage, which is the slick econometrics.

So, if Harford thinks that Levitt and Donahue's abortion paper is the best example of what econometrics can do for us, he might as well admit that econometrics is just another way to make an argument. It has no privileged insight into profound truths, and the eternal debates will never be 'settled' by econometric methods. Reality will reveal itself over time, and econometrics does not speed up our discernment of this reality by that much.