Challenges in Implementation and Impact Spotlight

Challenges Spotlight Report Cover

Since 2009, the Bureau of Justice Assistance (BJA) has provided more than $14.4 million to 35 local, county, and state law enforcement agencies conducting 38 Smart Policing Initiative (SPI) projects. Though many SPI sites have documented significant crime decreases in the targeted areas with sound research methodologies, others have been less successful.

The reasons why some SPI sites have struggled are varied and include implementation problems, data analysis issues, and leadership turnover. Other sites have implemented evidence-based strategies and evaluated those strategies with rigorous research methodologies; but in the end, crime reductions were not realized. We consider such initiatives successful because they identify strategies, practices, and lessons that other jurisdictions can learn from, and they are evidence-based because of the strong research designs they employed.

This SPI spotlight reviews the experiences of three sites—Cincinnati (OH), Joliet (IL), and Lansing (MI)—that fall into this last category. All three sites were led by police officials and criminal justice scholars who were well-versed in evidence-based practices and researcher/practitioner partnerships. Each site engaged in intensive data analysis to examine the underlying conditions and causes of the targeted crime problem (robbery in Cincinnati, drug dealing in Lansing, and gun violence in Joliet). Each site implemented a comprehensive, collaborative data-driven strategy to address their respective crime problems, from interventions based on the problem analysis triangle in Cincinnati and hot spots in Joliet, to focused deterrence and offender call-ins in Lansing. Each project was evaluated using rigorous quasi-experimental research designs. Despite these ingredients for success, none of the three sites experienced statistically significant crime declines that could be tied to their SPI.

This spotlight identifies a number of common challenges to implementation and impact that were experienced by the three sites, including: lapses in continuous, real-time problem analysis; insufficient program dosage; stakeholder limitations; and tension between operational decision-making and research design integrity. In Cincinnati, for example, geographic analysis of the robbery problem led the SPI team to increase the size of the original target area, which necessarily weakened the intensity of the intervention. In Joliet, probation and parole officers were active participants in the SPI, but restrictions on their authority limited the team’s ability to conduct compliance checks and to initiate revocation proceedings. In Lansing, the nature of drug dealing shifted from a traditional turf-based model to mobile transactions coordinated through cell phones, which forced the SPI team to alter their interventions “on the fly.”

These experiences (and others) in Cincinnati, Joliet, and Lansing highlight the importance of devising a strong process evaluation that allows for detailed documentation of implementation processes and challenges, and for a thorough understanding of why a program did or did not produce the intended crime reduction benefits. The Cincinnati, Joliet, and Lansing SPIs also underscore the importance of thinking broadly about program impact. Impact can be measured in terms of knowledge gained, organizational change, and new partnerships—developments that are not easily quantified in terms of statistical significance but represent positive change in a law enforcement agency.