DOI: 10.3886/icpsr38188
title: Instilling a Culture of Continuous Learning From Criminal Justice System Errors: A Multi-Stakeholder Sentinel Event Review of Process in Philadelphia, Pennsylvania, 2006-2016
title: Archival Version

description: In January 2016, the Quattrone Center for the Fair Administration of Justice at
the University of Pennsylvania Law School was awarded grant 2015-R2-CX-K040 for
a proposal entitled "Instilling a Culture of Continuous Learning from
Criminal Justice System Errors: A Multi-Stakeholder Sentinel Events Review of Process in Philadelphia." The work performed under the grant built upon
a successful pilot program supported by NIJ, in which a group of criminal
justice professionals in Philadelphia, PA conducted a
multi-stakeholder sentinel event review (SER) of a criminal case in the United States. The Quattrone Center proposed to build upon
the pilot project by implementing the Philadelphia Event Review Team (PERT) as
a more permanent working group to identify cases of error in the Philadelphia
criminal justice system and conduct sentinel event reviews of those
cases over the three-year period of performance for the grant.
The PERT formed a group of dedicated criminal justice professionals across
multiple agencies - the Philadelphia District Attorney's Office, the
Philadelphia Police Department, the Defender Association of Philadelphia, the 1st
Circuit Court of Common Pleas of Pennsylvania, and others - who agreed to meet
regularly and review cases where all of the participants felt an undesired
outcome had occurred. 
The process for conducting the second SER was lengthy and circuitous, and was slowed by a
number of factors: the churn of elected and appointed personnel in the participating agencies, the overhang of
potential impending litigation on cases being reviewed for possible selection,
and in one instance, the inability to waive attorney/client privilege for a
defendant whose treatment by the system was viewed as suboptimal by the PERT. Both the successes and the challenges of SERs
in criminal justice as experienced by the PERT are set forth below.
The SERs conducted as part of this project followed the definition of SER set forth in the NIJ publication Mending Justice: Sentinel
Event Reviews. Sentinel events, which have a history of
successful application in fields as diverse as aviation, healthcare, and
military operations, are multi-stakeholder reviews of instances of error in
complex human systems that seek to reveal and understand the root cause(s) of
such "never events" as plane crashes, surgical errors, and other
accidents. They involve a formal procedure to review errors in a non-blaming atmosphere and determine ways to avoid such errors in the future.
Applying the methodology of Sentinel Event Reviews (SER) to the criminal justice system involves reviewing
a criminal investigation and prosecution that demonstrated some "error," as defined by the participants of the SER.
The PERT took the position that any wrongful conviction, cold case,
officer-involved shooting, or death of an individual in the custody of the
state was by definition a case of error; this definition is completely separate
from an assessment of blame and simply held that any instance of these outcomes
was per se unwanted, even if such an outcome may have resulted by good faith
professionals acting within established training and protocols. The PERT attempted to conduct SERs of
selected cases in the criminal justice system using principles of "just culture
review," defined as "a culture that recognizes that competent professionals
make mistakes and acknowledges that even competent professionals will develop
unhealthy norms (shortcuts, "routine rule violations"), but has zero
tolerance for reckless behavior." (Agency for Healthcare Research
and Quality Glossary) 
A substantial question regarding the application of SERs to criminal justice was whether the adversarial nature
of the criminal justice system, which is different than the structure of the
systems in which SERs were originally developed, would be a barrier to their
successful application in criminal justice. The adversarial culture unique to the criminal justice process creates
additional challenges, including the complexity of gaining consensus from
cross-agency participants on the definition or occurrence of errors, the
ability to define shared goals within a system, and the ability to implement
reforms that require changes in agencies other than one's own. Within that context, the work
performed under this grant provides additional information to help answer the
questions posed by, and the methodology outlined in, Mending Justice--namely, (1) how might SERs be applied to the
criminal justice system; (2) can they effectively reduce future errors; and (3)
is this approach sustainable?
 The datasets are the result of two data requests. The first data request asked for appeals court cases for the Philadelphia County Court of Common Pleas involving criminal cases with an appeal filed from 1/1/2011 through 12/31/2015 as recorded in the Pennsylvania Appellate Court Management System (PACMS). The second data request asked for data on case level data related to the criminal cases disposed in the Philadelphia County Court of Common Pleas from 11/17/2006 through 11/17/2016 with one of the following final case dispositions: dismissed, judgement of acquittal, mistrial, nolle prossed, nolle prossed (case dismissed), quashed, demurrer sustained, or withdrawn. For each kind of data (cases, case dispositions, offenses, offense dispositions, docket entries) there are two files, one for each of the data requests. The two files for each type of data share the same structure and fields.
description: This project evaluated the ability of a multi-stakeholder, cross-agency review team, the
Philadelphia Event Review Team (PERT), to regularly prioritize and evaluate both Sentinel
Events and more routinized errors in the administration of criminal justice.
description: To facilitate the identification of cases of error, the Quattrone Center also worked with
the Administrative Office of the Pennsylvania Courts (AOPC) to create the Case Review
Database, or CARD, a unique data set of cases believed to have unintended outcomes that
occurred in the regular administration of criminal justice, and which was designed to allow novel
data analytics of errors and potential preventative steps. The PERT and AOPC designed a
database of approximately 5000 criminal cases adjudicated by Philadelphia courts and having
procedural outcomes that reflected potential undesired outcomes (e.g., cases that were dismissed
by the District Attorney's Office after a jury was empaneled; convictions that were overturned on
all charges and for which no subsequent criminal charges were pursued, etc.). 
The members of the PERT were assembled by the Quattrone Center at Penn Law with
assistance from the Philadelphia County Criminal Justice Advisory Board, a sitting organization
that consists of a broad cross-section of criminal justice stakeholders in Philadelphia that was
supportive of the goals of the PERT. Regular stakeholders included: Philadelphia Police
Department (PPD); Philadelphia District Attorney's Office (DAO); Defender Association of
Philadelphia; and First District Court of Common Pleas of Philadelphia (CCP). Representatives
from the Quattrone Center briefed the CJAB on multiple occasions about the progress of the
PERT, and sought out the CJAB for assistance on adding new members for specific cases that
were being considered for review (e.g., the CJAB helped the Quattrone Center connect with the
Philadelphia Department of Behavioral Health when a case dealing with defendants with
behavioral health challenges was identified as potentially ripe for SER). Members also included the Criminal Justice Coordinator for the City of Philadelphia and prominent advisors from the
criminal defense bar who were not affiliated with the Defender Association. 
The PERT proceeded with the discovery of cases to review in three ways. First, member
agencies were asked to nominate cases that had affected their personnel and were believed to be
cases of error and/or cases that had resulted in undesirable outcomes. Second, the Quattrone
Center conducted a review of Philadelphia cases in the National Registry of Exonerations and a historical review of
media reports on overturned Philadelphia criminal cases. Third, the Quattrone Center and the
AOPC generated the CARD and the Quattrone Center began to conduct searches of the CARD to
reflect areas of interest to the PERT participants.
description: Variables in this study include information about appeals, case and offense dispositions, and docket entries from the Philadelphia County Court of Common Pleas.
description: Datasets: 
DS0: Study-Level Files
DS1: Appeals Court Case Data
DS2: Case Disposition Data
DS3: Case Disposition II Data
DS4: Case Data
DS5: Case II Data
DS6: Offense Disposition Data
DS7: Offense Disposition II Data
DS8: Official Docket Entry Data
DS9: Official Docket Entry II Data
DS10: Offense Data
DS11: Offense II Data
description: Criminal court cases from the Philadelphia County Court of Common Pleas. Smallest Geographic Unit: Zip Code
description: record abstracts