Will Releasing Convicts from Prison Increase Crime?

The Open Philanthropy Project has long been interested in the effects of America's mass incarceration. Now that the tide seems to be turning in their favor, they decided to commission a major study of whether reducing the prison population might lead to a major increase in crime. The study was carried out by David Roodman. He calls his project a "mass replication" of the best available studies on prison and crime, meaning that he went very thoroughly back through the data used in these studies, checked all their math, and asked the original investigators pressing questions about decisions they made. (Why did you look at this variable rather than that one? Why did you use this data set rather than some other?)

Alex Tabarrok is one of the study authors that Roodman put through this process, and Roodman ended up concluding the opposite of what Tabarrok's paper argued. But Tabarrok was nonetheless very impressed by Roodman's diligence:

My paper on Three Strikes with Eric Helland was one of the papers that Roodman replicated. (Fortunately, it did replicate with the exception of one error in a table.) I can vouch that Roodman gave us tougher scrutiny than did the peer reviewers.

Not surprisingly, I don’t agree with all of Roodman’s conclusions but rather than pushing back I think it more important to underline how impressive the replication project is. There are many review papers in economics but a replication project of this magnitude is nearly unprecedented. In our paper on the National Science Foundation, Tyler Cowen and I advised the NSF to put more efforts into replication. We wrote:

The NSF could support replication studies on a significant scale. A significant fraction of economic research does not easily replicate…Replication and reproducibility studies are true public goods that are not rewarded highly by most top journals or by the tenure process at research universities.

Roodman and OPP have demonstrated the value of replication on a large scale.

And what did Roodman find?

I estimate, that at typical policy margins in the United States today, decarceration has zero net impact on crime. That estimate is uncertain, but at least as much evidence suggests that decarceration reduces crime as increases it. The crux of the matter is that tougher sentences hardly deter crime, and that while imprisoning people temporarily stops them from committing crime outside prison walls, it also tends to increase their criminality after release. As a result, “tough-on-crime” initiatives can reduce crime in the short run but cause offsetting harm in the long run. Empirical social science research—or at least non-experimental social science research—should not be taken at face value. Among three dozen studies I reviewed, I obtained or reconstructed the data and code for eight. Replication and reanalysis revealed significant methodological concerns in seven and led to major reinterpretations of four.

Note one key detail about what Roodman says above: "at typical policy margins." He is not saying that freeing all prisoners would have no significant effect on crime, only that the kind of sentencing reform efforts now under way have no significant effect. So it might be that if we ratcheted the prison population down to its size in 1980, when it was less than half what it is now, something bad might happen. But the sort of 5 to 20 percent reductions states like Georgia are seeking do not seem to lead to crime waves.

I'm not going to tell you that Roodman is right; these are very hard problems, and studies have come to varying conclusions. But as I have said before in similar cases, when researchers can't even agree on the direction of an effect, some finding it negative and some positive, it is not likely to be very big. Cutting our prison population in the way reformers are actually seeking would likely not lead to major crime problems, and it might have good effects in the black and Hispanic communities from which so many prisoners come.