The City That Became … [How Much] Safer?

Posted on 12 November 2012 by


I just picked up a copy of the most racy book available to crime analysts this month, The Crime Numbers Game: Management by Manipulation (Advances in Police Theory and Practice) by John A. Eterno and Eli B. Silverman. Eterno was a NYPD officer who rose through the ranks and retired a captain; he and Silverman, who’s an Emeritus professor at John Jay College of Criminal Justice, have collaborated on other work about COMPSTAT and what they allege is the NYPD’s literary-culinary proclivity.

The Crime Numbers Game makes the case that the NYPD has engaged in systemic book-cooking when it comes to COMPSTAT, and concludes through a series of interviews whose methodology is to be applauded that a percentage of the heralded crime drop in the city over the past two decades is thanks to widespread, culturally encouraged and brutally enforced mis-statement and downgrading of crimes in the city – for example, classifying a felony attempted forcible rape as a misdemeanor “forcible touching”.

I have no idea if what they are saying is true. I have no skin in this game: I believe that there are clear abuses of the COMPSTAT culture at NYPD, but that’s totally anecdotal, and I’ve not done anything like the research that the authors have done.

I do like how transparent the authors are about their sources and their methodology (especially about the flaws in it that they themselves detected, and rich descriptions of how they have strived to address them), and if they leap too quickly to conclusions in their text (that is, they sometimes come straight out and state their conclusions before they’ve finished establishing the evidence), they have made every attempt to provide peer-reviewable breadcrumbs for us to follow. The references they list in each chapter are an outstanding resource (and I’m not just saying that because they quote me and Police Led Intelligence): they effectively become a top 100 reading list for anyone interested in crime analysis.

But I must raise the point that they refer to to the work of Frank Zimring, which we have discussed here and here. In their book, the authors not-so-gently paint Zimring as something of a COMPSTAT-friendly NYPD stooge. They raise two key points which I address here merely because I have written about Zimring’s work with some fondness for it and, if true, Eterno and Silberman’s comments could change my opinion dramatically. For example,  the second of those links above to PLI articles (this one), an article on Zimring’s correlation of prison budgets and law enforcement, was what the authors suggest is a good place to seek additional criticism of Zimring’s work. To be fair, they are right: I was pretty incredulous at the linkage.

First, Eterno and Silverman claim that, while Zimring has stated that he used independent assessment to determine the veracity and integrity of the COMPSTAT figures, the fact that he chose health records is not as transparent as Zimring claims, and that there is a major problem with Zimring’s comparison of auto-theft claims matched against NYPD auto theft stats.

“What [Zimring] fails to advise readers is that the police department is in continuous contact with the Medical Examiner (ME). They constantly check and verify homicides. Thus, the ME is not an independent source of confirmation because the police constantly change reports based on the ME’s findings. Nothing could be further from independent. We also point out that homicides are most likely to be accurate. It is harder to fudge these. However in our research we were told about how even these are gamed. One interviewee advised of a commanding officer who showed up at an obvious murder and ordered the complaint report to be written as “investigate aided.” While at some point this will likely be listed as a homicide, it is clearly an attempt to hold down the count as long as possible. (Eterno and Silverman, 2012 [1])

That’s an interesting argument.

On the vehicle theft, I’m not sure I agree with the authors when they say the effects of an “earth-shattering” policy shift by NYPD in 1999 changes the game when it comes to Zimring’s dissection of auto-theft numbers, as he compares NYPD’s to those of the insurance industry. The “earth-shattering” policy change is that NYPD made victims of car theft more responsible to document the theft, including signing a deposition statement and participating in investigation.

The authors imply that this scares away legitimate would-be complainant victims of auto theft, thereby driving down the reports, which have the effect of driving down the claims because claims are based on police reports.

On this one I have a mixed statement. First, they’re right that a reduction in reports would result in fewer claims, so as reported car thefts go down, so naturally would claims. And they are right that the effect of more intensive documentation would have a chilling effect on reports of car theft.

I disagree entirely, however, that this has a chilling effect on legitimate reporting. Consider that (a) anyone whose car is stolen is going to jump through hoops to get the report filed (they will grumble, to be sure, but they will do it), precisely because (b) the insurance companies require it as the basis of a claim, and cars cost enough that few are going to walk away from a legitimately stolen vehicle on the strength of some extra forms and time. I believe, based on anecdotal evidence and what my mother likes to call, “common sense” that the numbers of reported car thefts went down after the policy change because it kept fraudulent, not legitimate, reporters away.

Having read through a bit more than half the book I will say it’s absolutely worth reading. It raises serious concerns which, if true, amount to a terrible management system which has been allowed to run amok – raising some frightening civil liberties issues. It should be read by anyone involved in law enforcement and public safety statistical analysis because it highlights many possible ways to game the system and then describes the unintended consequences of such gaming.

If their comment about the tight involvement between the ME’s office and NYPD is accurate (and it certainly rings true in my experience) then I agree with them that Zimring has based many of his conclusions about the accuracy of COMPSTAT data – and thence his other conclusions – on flawed or certainly non-independent data. The authors stop short of saying Zimring’s work is without merit, and agree with Zimring that the crime drop in New York is real.

Now the analysts are arguing over how much of a crime drop occurred.


[1] Eterno, J and Silverman, E (2012) “The Crime Numbers Game: Management by Manipulation (Advances in Police Theory and Practice)” CRC Press; 1st edition (January 31, 2012).