U.S. Department of Justice

Office of Justice Programs

National Institute of Justice SPECIAL REPORT

NATIONAL INSTITUTE OF JUSTICE

MENDING JUSTICE:

Sentinel Event Reviews

SEPTEMBER 2014
U.S. Department of Justice

Office of Justice Programs

810 Seventh St. N.W. 

Washington, DC 20531

Eric H. Holder, Jr.

Attorney General

Karol V. Mason

Assistant Attorney General

William J. Sabol, Ph.D.

Acting Director, National Institute of Justice

This and other publications and products of the National Institute of Justice can be found at:

National Institute of Justice

http://www.nij.gov

Office of Justice Programs

Innovation • Partnerships • Safer Neighborhoods

http://www.ojp.usdoj.gov

The National Institute of Justice is the research, development and evaluation agency of the U.S. Department of Justice. 

NIJ’s mission is to advance scientific research, development and evaluation to enhance the administration of justice 

and public safety.

The National Institute of Justice is a component of the Office of Justice Programs, which also includes the Bureau 

of Justice Assistance; the Bureau of Justice Statistics; the Office for Victims of Crime; the Office of Juvenile Justice 

and Delinquency Prevention; and the Office of Sex Offender Sentencing, Monitoring, Apprehending, Registering, and 

Tracking.

The findings and conclusions in this publication are those of the authors and do not necessarily reflect the official 

position or policies of the U.S. Department of Justice.

Photo Source: ThinkStock NCJ 247141


Mending Justice

National Institute of Justice | NIJ.gov

Contents 

1 Introduction

3 Learning From Error in the Criminal Justice System:

Sentinel Event Reviews

By James M. Doyle

20 Moving Beyond a Culture of Defensiveness and Isolation

By John Chisholm

22 To Learn Something, Do Something

By Michael Jacobson

24 No Sticks: Safe Spaces and a Desire to Get Ever Better

By Maddy deLone

26 The Dilemma of the Moral Imperative

By Bernard Melekian

28 Front-end and Back-end Solutions

By Dan Simon

30 Stepping Back to Move Forward: Recognizing 

Fallibility and Interdependency

By Mark Houldin

32 Egg Heads Matter: Academic/Agency Partnerships 

and Organizational Learning

By Jack R. Greene

34 An Opportunity We Cannot Afford to Lose

By Greg Matheson
Mending Justice

National Institute of Justice | NIJ.gov

36 The Blame Game

By Jennifer Thompson

38 Innocence Commissions: The Case for Criminal Justice Partnerships

By Russell F. Canan

40 High Expectations, Good Intentions and Normalized Policy Deviation: 

A Sentinel Event

By Jim Bueermann

42 Using Sentinel Events to Promote System Accountability

By George Gascón

44 Cold Case Homicides: Ideal Candidates for a 

Sentinel Event Review

By Frank P. Tona 

46 Building a Learning-From-Error Culture in Policing

By John R. Firman

48 Punishment-Based vs. Education-Based Discipline: 

A Surmountable Challenge? 

By Sean Smoot

50 Reducing Failure: A View of Policing Through an Organizational 

Accident Lens

By Jon Shane

53 Appendix
Mending Justice 1

National Institute of Justice | NIJ.gov

 Introduction

C

onfidence in our nation’s criminal justice 

system rests on several core beliefs: First, 

that most justice work is routine, following 

a fairly prescribed path that renders error a 

rarity. Second, that in the rare instance when a mistake 

does occur, it is typically a clear case of negligence or 

misconduct, and “the system” readily detects and fixes 

it through its many separate (and characteristically 

adversarial) components, which “backstop” each other. 

Finally, when an error occurs, we believe that there are 

processes in place to make sure that type of error will 

never happen again. 

The problem is that these beliefs may be largely 

unfounded.

Errors — such as a wrongful arrest, the wrongful release 

from prison of a dangerous offender who harms another 

victim, the conviction of an innocent person, a wrongful 

police shooting — not only occur in our criminal justice 

system, but they can occur in seemingly routine cases. 

Errors often go undetected and, when they are detected, 

the detection frequently seems to be the result of 

extraordinary luck or perseverance after many years. 

Certainly, most criminal justice agencies have errordetecting processes in place: consider, for example, police 

internal affairs reviews and prosecutors’ professional 

ethics boards. Too often, however, these become a 

“gotcha” process that assigns blame, which can drive 

errors underground, making them harder to detect and 

correct. In other words, the criminal justice system lacks 

what medicine, aviation, and other high-risk enterprises 

have found essential to detecting and addressing 

organizational errors: a nonblaming, all-stakeholder, 

forward-leaning mechanism through which we can learn 

from error and make systemwide improvements that go 

beyond disciplining rulebreakers and render similar errors 

less likely in the future. 

In this publication, the National Institute of Justice (NIJ) 

explores the feasibility of mobilizing an “organizational 

accident,” learning-from-error approach in the criminal 

justice system. We introduce the notion of the “sentinel 

event”: a bad outcome that no one wants repeated and 

that signals the existence of underlying weaknesses in 

the system. 

“Sentinels stand watch,” says James Doyle. “They detect 

the first signs of a looming threat and sound a warning 

that should not be ignored.” Beginning on page 3, Mr. 

Doyle — who served as a Visiting Fellow at NIJ for two 

years — discusses how the medical field first heard 

sentinel event warnings with the rise of unexpected 

infections acquired in hospitals and when “wrong patient” 

surgeries occurred. In aviation, sentinel event warnings 

are sounded each time an airplane crashes or a near miss 

occurs. 

In criminal justice, a sentinel event could be similarly 

easy to recognize: the exoneration of an innocent person; 

the release from prison of a dangerous person; or even 

a near miss in which an innocent suspect was arrested, 

processed, and held until the error was finally discovered 

and greater harm was avoided. Could these sentinel 

events signal underlying weaknesses in the justice 

system? Could an all-stakeholder, nonblaming, forwardlooking review of such events lead to greater system 

strength and effectiveness?

Mr. Doyle explains that, when bad outcomes occur in a 

complex social system — like our justice system — they 

are rarely the result of one individual’s mistake. Rather, 

multiple small errors combine and are exacerbated by 

underlying system weaknesses. After the exoneration 

of an innocent person, for example, the answer to 

the question, “Who is responsible for this wrongful 

conviction?” is, almost invariably, “Everyone involved, 

to one degree or another ... if not by making a mistake, 

then by failing to catch one.” And “everyone” can include 

not only those who operate at the sharp end of the 

system, like the police, but also, Doyle writes, “the distant 

actors who set their budgets, assign their caseloads and 

define their legal authority.”

We also present a collection of commentaries from 

highly respected “early adopters” who offer their unique 

perspectives regarding the innovative notion of a sentinel 

event review process in the justice system. 
2 Mending Justice

National Institute of Justice | NIJ.gov

Milwaukee District Attorney John Chisholm reflects on 

his experience as a young Army officer when “afteraction reviews” were instituted in an effort to improve 

system performance. “The goal was to encourage leaders 

to honestly acknowledge and learn from mistakes,” 

he writes. “It also encouraged nonlinear thinking and 

initiative by junior leaders (like me), by elevating the 

status of all participants and treating them as equals.” 

Madeline deLone, executive director of the Innocence 

Project, endorses the idea of a sentinel events approach 

and builds on lessons she learned as a prison health 

care administrator on Rikers Island. Dr. Barney Melekian, 

former director of the Office of Community Oriented 

Policing Services (COPS), writes that all-stakeholder, 

nonblaming reviews would push beyond whether a police 

officer, for example, “utilized a ‘workaround’— whether he 

or she zigged instead of zagged — but would address why 

and how the system put him or her in a position where 

that seemed like the best or least bad choice available.” 

And Greg Matheson, former director of the Los Angeles 

Police Department laboratory, writes that developing a 

sentinel event process, aimed at continuous systemwide 

quality improvement, “is an opportunity we cannot afford 

to lose.” 

Several commentaries look at the important element of 

nonblaming in a sentinel event review process. Jennifer 

Thompson — whose erroneous identification of the 

man who raped her played a role in sending an innocent 

man to prison for more than 10 years — speaks truth to 

power about “the blame game” that can prevent us from 

learning from error. 

At first glance, these individual commentaries may 

seem to speak only to the separate components, or 

“silos,” of the criminal justice system — the police or 

victims, prosecutors or crime labs, defense counsel or 

academics or judges. But, as you read them, we invite 

you to remember that each silo operates within a much 

larger system. And it is the interdependence of these 

separate silos that lies at the heart of NIJ’s Sentinel 

Events Initiative. Indeed, it is the great “gravitational pull” 

generated by cultures within each silo that argues for the 

creation of an opposing force: a true systemwide analysis 

of errors across the entire criminal justice system.

Building the Science

Every error that occurs in our criminal justice system — 

every episode of failed justice — inflicts specific harms: 

An individual is wrongfully convicted, a criminal goes free, 

a victim is deprived of justice, a community is ill-served, 

and the agencies of justice emerge more tarnished and 

less trusted than before. Although it is imperative to 

address these specific harms, that alone is not enough. 

Errors must be recognized as potential sentinel events 

that could signal more complex flaws that threaten the 

integrity of the system as a whole. 

As the science agency of the Department of Justice, NIJ 

is focused on answering key research questions about the 

sentinel events approach of learning from error:

■■ Can the many parts of the justice system participate 

fully in a nonblaming review of an error that moves 

beyond ascribing blame toward future, “preventive” 

accountability?

■■ Does such an approach provide a means to achieve 

desired outcomes, such as increased effectiveness 

and fewer errors, and other public safety dividends, 

such as greater public perception of integrity of the 

nation’s justice system?

■■ Can a sentinel events approach be sustained over 

time and incorporated into the routine activities of 

state and local justice processes? 

The successes of sentinel event reviews in other 

professions inspire us to imagine a justice system that is 

constantly working to understand itself and its errors and 

is strengthening its processes by embracing a forwardleaning approach where shared responsibility prevails 

over finger-pointing and blaming. Yet we do not take this 

inspiration as an article of blind faith: NIJ’s commitment 

to testing, analyzing and objective evaluation remains 

uncompromised. The evidence may show that efforts 

to adopt a sentinel events approach in criminal justice 

are not feasible or effective, or it may reveal that these 

are indeed the first formative steps in a revolution that 

ensures a system that is fair, unbiased and worthy of our 

highest ideals.
Mending Justice 3

National Institute of Justice | NIJ.gov

Learning From Error in the Criminal Justice System: Sentinel Event Reviews

By James M. Doyle

D

NA exonerations of wrongfully convicted 

defendants have thrown new light on the 

problem of error in criminal justice, revealing a 

gap in our system’s design. The U.S. criminal 

justice system lacks a feature that medicine, aviation 

and other high-risk enterprises see as critical: a way to 

account for unintended tragic outcomes, to learn lessons 

from our errors, and to use these lessons to reduce future 

risks.

Many fields facing high-risk incidents have responded to 

the dangers exposed by known errors by developing — 

■■ The consistent practice of an all-stakeholder, 

nonblaming, forward-looking examination of known 

errors and other sentinel events, and

■■ The means for mobilizing and sharing the lessons of 

sentinel events in an ongoing conversation among 

practitioners, researchers and policymakers.

Sentinels stand watch. They are the first to see threats, 

and they sound a warning before those threats can do 

harm. A sentinel event in the criminal justice system 

warns us of threats to justice, and it calls us to act. 

It is a significant, unexpected negative outcome that 

signals a possible weakness in the system or process. 

Sentinel events are likely the result of compounded 

errors and — if properly analyzed and addressed — may 

provide important keys to strengthening the system and 

preventing future adverse events or outcomes.

Medical professionals use sentinel event reviews to 

examine unexpected patient deaths, medication errors, 

wrong-patient surgeries, “near misses” and similar 

incidents to account for their root causes. These reviews 

focus on reducing future risk, not on fixing blame for 

past mistakes. They look over the horizon to intercept 

preventable harms. 

Can our criminal justice system develop this capacity 

for forward-looking accountability?1

 Can we enhance 

professionalism by accepting error as an inevitable 

element of the human condition and studying known 

errors in a disciplined and consistent way? Can we focus 

on addressing future risk instead of fixing blame for past 

events? Can we share lessons learned to prevent future 

errors? 

During my two years as a Visiting Fellow at the National 

Institute of Justice, I began a reconnaissance: a 

preliminary exploration of the potential for mobilizing in 

our criminal justice system the lessons that industry, 

aviation and medical safety reformers learned as they 

used sentinel events to develop “cultures of safety.” 

Lessons Learned From the Medical Field

One way to see the learning opportunities presented 

by criminal justice sentinel events is to consider 

contemporary medicine’s encounter with its own version 

of the problem: “iatrogenic” injuries to patients or harms 

caused by medical treatment.

Just as the criminal justice system is haunted by the fact 

that it sometimes convicts the wrong person, medicine 

is haunted by the fact that it sometimes operates on 

the wrong patient.2 When modern medical researchers 

began to look carefully into wrong-patient events, they 

uncovered surprising insights. For example, one intensive 

examination of a wrong-patient surgery discovered 

at least 17 errors — among them, that the patient’s 

face was draped so that the physicians could not see 

it; a resident left, assuming the attending physician 

had ordered the invasive surgery without telling him; 

conflicting charts were overlooked; and contradictory 

patient stickers were ignored. The researchers’ 

analysis showed not only mistakes made by individual 

doctors and nurses but also latent systemic problems. 

Communications between staff members were terrible, 

and computer systems did not share information. 

When teams failed to function, no one was concerned 

because of a culture of low expectations that “led [staff] 

to conclude that these red flags signified not unusual, 

worrisome harbingers but rather mundane repetitions of 

the poor communication to which they become inured.”3
4 Mending Justice

National Institute of Justice | NIJ.gov

Deviations from good practice had become normal — 

and a tragedy resulted.

The crucial point for the researchers, however, was that 

no single one of the 17 errors could have caused the 

wrong-patient surgery by itself.3

The bottom line: Researchers in the medical field 

determined that many avoidable harms, including wrongpatient surgeries, were the result of an “organizational 

accident.”

Criminal Justice Errors as Organizational 

Accidents

No single error can cause an organizational accident 

independently; the errors of many individuals (“active 

errors”) converge and interact with system weaknesses 

(“latent conditions”), increasing the likelihood that 

individual errors will do harm. The practitioners and 

organizations involved in these tragedies do not choose to 

make errors. These events involved normal people, doing 

normal work, in normal organizations,4 and they suffer, in 

Charles Perrow’s memorable phrase, “normal accidents.”5

Like the Challenger launch decision, the outcomes reflect 

“mistake[s] embedded in the banality of organizational 

life.”6

Consider our traditional wrongful conviction narrative: 

The witness picks the wrong man, the cops and district 

attorney believe the witness, and so does the jury. The 

inadequacies of this narrative emerge as soon as we 

apply the organizational accident concept. 

A wrongful conviction is not the result of a single error, 

nor is it the fault of one operator or one investigative 

technique. As in a wrong-patient surgery, many things 

have to go wrong before the wrong person is convicted. 

Yes, the eyewitness does have to choose the wrong man 

from the photo array, but before that, law enforcement 

officers have to decide to put him into the array, design 

the format of the array and choreograph its display. 

Forensic evidence at the crime scene could have been 

overlooked or — even if properly collected and then 

tested in the lab — distorted during the courtroom 

presentation. Cell phone, mass transit card data or other 

alibi information could have been ignored. Tunnel vision 

— augmented by clearance rate and caseload pressures 

from above — may have overwhelmed the investigators 

and the prosecutors.7

 Poorly funded or untrained defense 

counsel may have failed to investigate alternative 

explanations or to execute effective cross-examination. 

The witness erred; the police erred; the technicians erred; 

the prosecutors erred; the defense erred; the judge and 

the jury erred; and the appellate court erred, too. 

No single error would have been enough. The errors 

combined and cascaded — then there was a tragedy.

In an organizational accident, the correct answer to the 

question, “Who is responsible?” is almost invariably, 

“Everyone involved, to one degree or another,” if not 

for making a mistake, then by failing to catch someone 

else’s. In the instance of a wrongful conviction, 

“everyone” may include not only witnesses, police, 

forensic scientists and lawyers at the sharp end of the 

What is a Criminal Justice Sentinel Event?

What would constitute a sentinel event in criminal justice? Wrongful convictions, certainly, but also “near 

miss” acquittals and dismissals of cases that at earlier points seemed solid; cold cases that stayed cold too 

long; “wrongful releases” of dangerous or factually guilty criminals or of vulnerable mentally handicapped 

arrestees; and failures to prevent domestic violence within at-risk families. 

Sentinel events can include episodes that are “within policy” but disastrous in terms of community relations 

(such as the arrest of Harvard professor Henry Louis Gates), whether or not everyone agrees that the event 

should be classified as an “error.” Even the lengthy and expensive incarceration of a harmless geriatric 

prisoner, where the excessive cost constitutes the harm, could be examined as a sentinel event.

In fact, anything that stakeholders can agree should not happen again could be considered a sentinel event.
Mending Justice 5

National Institute of Justice | NIJ.gov

system, but also legislators, policymakers, funders and 

appellate judges who were far from the scene of the 

event but who helped design the system and dictated the 

conditions under which the sharp-end operators work. 

The Problem With Single-Cause Approaches 

to Understanding Error

When we apply the organizational accident concept 

to a criminal justice sentinel event, it illuminates the 

limitations of two conceptions of error that criminal justice 

reformers, horrified by miscarriages of justice, have 

adopted almost by reflex:

■■ Bad apples: This conception of error focuses on 

punishing individual actors to guarantee overall 

system reliability.

■■ Swiss cheese: This conception of error focuses on 

performing a sequence of independent protective 

“screens” that culminates in an end-of-process 

adversary inspection to ensure quality control. It 

inspires efforts to repair the component screens 

individually — the police reform investigation 

practices, the district attorneys reform prosecutorial 

practices, and so on. 

Safety experts see these approaches as inadequate. In 

fact, they see them as dangerous traps.8

Bad apples: Why this approach is not enough

In criminal justice, we traditionally take an approach to 

error that assumes a “bad apple” operator is responsible. 

Someone must be to blame for the error, so the impulse 

is to find and discipline that person. This is what people 

typically mean when they call for “accountability” in the 

aftermath of the exoneration of an innocent person.9 The 

bad-apple orientation, however, is inadequate to describe 

how things go wrong, and it has a crippling impact on 

efforts to prevent future errors. 

Traditionally, medicine was governed by a similar 

assumption. As Dr. Lucian Leape, a professor at the 

Harvard University School of Public Health and a pioneer 

in the patient safety movement, wrote in his seminal 

essay, “Error in Medicine”:10

Physicians are expected to function without 

error, an expectation that physicians translate 

into the need to be infallible. One result is that 

physicians, not unlike test pilots, come to view 

error as a failure of character — you weren’t 

careful enough, you didn’t try hard enough. This 

kind of thinking lies behind a common reaction 

by physicians: How can there be an error 

without negligence?

Medicine often convened its “morbidity and mortality” 

reviews following adverse events in this spirit, and in the 

eyes of the front-line practitioners, they became exercises 

in “blaming and shaming.”11

Medical culture’s “good man, good result” attitude 

translates seamlessly to criminal justice. In the “bad 

apple” approach to error analysis, the error occurred 

because some doctor (or police officer), nurse (or forensic 

scientist), or x-ray technician (or lawyer) was lazy, illtrained, venal or careless. In the bad apple approach, 

the task of conscientious professionals is to act as the 

custodians of a presumptively safe system and to protect 

it from incompetent and destructive individuals.4

It may be human nature to think that a big tragedy must 

have a big cause and that a tragic event requires that 

tragic punitive consequences fall on somebody. Besides, 

no field can function without employing disciplinary tools. 

A sentinel events approach to reviewing mistakes does 

not eliminate disciplinary consequences for consciously 

unethical behavior or knowing violations of settled 

rules.12 But it does see punishment of the lone bad apple 

as the wrong place to stop. We cannot discipline our 

way to safety, and equating “accountability” exclusively 

with blame and punishment has potentially crippling 

consequences. 

By focusing exclusively on ascribing blame, we drive 

many valuable reports of errors underground and leave 

latent system weaknesses unaddressed. Practitioners do 

not want to be blamed, and they do not want to become 

entangled in the unpredictable machinery of blaming 

colleagues. Inevitably, in a blame-oriented system, less 

and less gets reported and less and less is learned. 
6 Mending Justice

National Institute of Justice | NIJ.gov

This dynamic applies at the agency as well as the 

individual level. When a notorious sentinel event cannot 

be buried completely, the impulse to keep it “in house” 

or to try to shift the blame to someone else’s “house” 

intensifies. But because no individual house can ever 

fully explain an organizational accident, this approach 

allows overlapping weaknesses that might be studied 

and understood to remain latent in the system. Searching 

for a single cause prevents us from understanding how 

complex systems fail through the confluent, cascading 

errors — active and passive — of multiple contributors 

from many houses.13

Even where we can identify a bad apple — a corrupt 

or incompetent forensic scientist, for example, or a 

prosecutor who buries plainly exculpatory evidence — 

the lone villain approach is incomplete. Surrounding the 

bad apple are the people who hired him, created his work 

environment and failed to catch his mistakes. They, and 

the vulnerabilities they contribute, will still be with us 

after the bad apple is removed. We never ask the critical 

question, “Why did this decision look like the best (or, 

perhaps, the least bad) choice to the bad apple at the 

time?”

The question is whether discipline and forward-looking 

risk reduction can be held in balance. Can we hold people 

accountable and still stay mindful of the future — that 

is, can we give the “good guys” in the system something 

to do besides trying to hunt down and punish the “bad 

guys.” 

Swiss cheese: Why this approach is not enough

The “Swiss cheese” approach is an alternative 

conception of error that sometimes supplements the 

“bad apple” theory. In this view, error moves in a straight 

line from its origin (often in the act of a bad apple) to its 

tragic result unless it is blocked somewhere by one of a 

succession of barriers: a sequence of increasingly fine 

screens, each “inspecting” the output of the preceding 

screen. The system is envisioned as a model of defense 

in depth. So, in the criminal justice system, an erroneous 

“wrong man” prosecution must pass through a police 

supervisory screen, a crime lab screen, a prosecutorial 

screen, a grand jury screen, an adversary trial screen and 

an appellate review screen, among others, before it can 

take effect. This will happen only when — in a kind of 

folk version of James Reason’s famous “Swiss cheese” 

model of accident causation — there is a hole in each of 

the screens and those holes happen to line up, allowing 

the error a clear path to its horrific final impact.14,15

Viewed in this way, the systemic problem that, for 

example, a wrongful conviction reveals is a failure in 

component structures. As a result, solutions are most 

often seen in structural terms. This interpretation of 

systemic failure offers two strategies for preventing 

wrongful convictions by making structural reforms: (1) We 

might independently patch holes in each screen internally 

by adopting new best practices, such as double-blind 

sequential lineup techniques in police investigations 

or reforming indigent defense services by providing 

checklists, or (2) we might add a new screen through a 

prosecutor’s conviction integrity unit or a post-conviction 

actual innocence commission. 

The first approach appeals directly to officials’ natural 

inclination to keep problems “in house” — to “clean up 

our own mess.” (And, after all, in this view, repairing the 

hole in any single screen will be enough to block the path 

of error.) Blue ribbon commissions and working groups 

have pursued both approaches to generate specific “best 

practice” recommendations for reforms in components 

such as eyewitness identification procedures.16,17 Neither, 

however, gets at the issue of system reliability.

The fact is, no component of the criminal justice system 

functions in isolation. The work the prosecutors do is 

affected by choices the police make “upstream,” and 

the choices that the police make are often made in 

anticipation of what the “downstream” prosecutors and 

defenders will do. Any screen can open a hole in any 

adjoining screen. In addition, the options of all of the 

front-line operators are constantly shaped and reshaped 

by the distant actors who set their budgets, assign their 

caseloads and define their legal authority.

Addressing System Reliability

Either the “bad apple” or “Swiss cheese” orientation 

can improve the odds against another error, but because 

neither engages the systemic nature of the problems, the 

“solutions” they generate stop short of optimizing the 

system’s reliability. In fact, solutions that address only a 

single component may simply relocate the problem and 
Mending Justice 7

National Institute of Justice | NIJ.gov

create new dangers. Medical safety experts, for example, 

have learned that “Nothing threatens safety so much as 

the complacency induced when an organization thinks 

that a problem is solved.”18

No new set of best practices or checklists can cover every 

circumstance, so an irreducible zone of discretion always 

survives, and operators must manage life within that 

zone.13 The new sets of best practices and checklists that 

innocence commissions, technical working groups and 

other blue ribbon efforts generate have to be operationalized 

and executed, and they have to be maintained, monitored, 

evaluated and perhaps replaced when environments change 

or science or technology advances.18

Every new checklist comes under immediate and 

constant assault from caseload, clearance rate, budget, 

political, media and other environmental factors from 

the moment it is written. Workers at the sharp end of 

the system may feel forced to decide which of the new 

checklist’s 10 steps they can live without this time. Triage 

is required, and workarounds multiply. No one had more 

(or more carefully devised) checklists than the National 

Aeronautics and Space Administration (NASA), but the 

agency launched Challenger and Columbia anyway.

Drift toward failure is a threat to the new best practices 

just as it was to their now discredited predecessors.15 As 

Sidney Dekker observes:19

The organizational decisions that are seen as 

“bad decisions” after the accident (even though 

they seemed like perfectly acceptable ideas at 

the time) are seldom big, risky steps. Rather, 

there is a long and steady progression of small, 

incremental steps that unwittingly take an 

operation toward its boundaries. Each step away 

from the original norm that meets with empirical 

success (and no obvious sacrifice of safety) is 

used as the next basis from which to depart 

just that little bit more. It is this incrementalism 

that makes distinguishing the abnormal 

from the normal so difficult. If the difference 

between what “should be done” (or what was 

done successfully yesterday) and what is done 

successfully today is minute, then this slight 

departure from an earlier established norm is 

not worth remarking or reporting on.

Going “down and in” to find a single broken component 

will not be enough to explain these events and prevent 

their recurrence; we also have to go “up and out” to 

assess the complex environment that shaped the choice 

of the component, allowed the component to fail and 

made the failure catastrophic.15 “Reliability” (and its 

opposite) in criminal justice can no more be seen in a 

single component than “wetness” can be seen in a single 

molecule of H2O. Both fine-grained local knowledge 

and alertness to the pressures from the system’s larger 

environment are indispensable.

Many tragic mishaps could never have been predicted 

(and cannot now be completely explained) by reference to 

individual components. These tragedies are “emergent” 

events with origins in the “greater than the sum of its 

parts” zone found in all systems.15 No structural fix 

provides permanent protection. 

Culture, Not Structures

Although the phrase “criminal justice system” is 

everywhere, the system does not present itself as an 

arrangement of gears and switches that can be fixed with 

a wrench or new spare part. One objection to applying the 

organizational accident model to criminal justice might 

be, “Where is this ‘organization’ you are talking about?” 

This is a fair question, but it is clear that the criminal 

justice process at least functions as an ecosystem, like 

a pond or a swamp in which something (funding, for 

example) dumped on the near coast has mysterious and 

unanticipated effects on the far shore. Improving property 

crime investigations by swabbing every crime scene could 

create a backlog of rape kits in the lab. A backlog in the 

lab means a backlog in the courts; a backlog in the courts 

means more pressure for plea bargaining. 

The medical reformers, facing an analogous situation, 

became convinced that patient safety could not be dealt 

with as a matter of structure but must be addressed as a 

question of culture. They advocated that hospitals facing 

a rising tide of patient injuries should mobilize the findings 

of “human factors” researchers like James Reason, who 

argued that errors are inevitable in human performance 

and that the best path toward reliable performance in 

complex organizations is the creation of a “culture of 

safety.”14 
8 Mending Justice

National Institute of Justice | NIJ.gov

A culture of safety exists when an organization: 

1. Is informed about current knowledge in its field.

2. Promotes the reporting of errors and near misses.

3. Creates an atmosphere of trust in which people are 

encouraged to report safety-related information.

4. Remains flexible in adapting to changing demands 

(e.g., by shifting from steeply hierarchical modes into 

flatter team-oriented professional structures).

5. Is willing and able to learn about and adjust the 

functioning of its safety systems. 

Ironically, we can see a critical vulnerability in the 

culture of criminal justice most clearly by noting the 

absence of a structural feature: a vehicle for “forwardlooking accountability” that treats mistakes as sentinel 

events from which all stakeholders could learn the 

lessons that are important to preventing future harms.1

Preventing future harms requires more than a catalog 

of current defects in existing screens; it also requires 

an understanding of the processes by which those 

defects were created — that is, the processes from 

which tomorrow’s defects will emerge. If we do not fully 

understand how each screen is related to the others 

(or how all of the screens are related to the entire 

environment), we will always stay one tragedy behind.

The police operate a “production stage,” during which 

they make the cases, often with the participation of the 

prosecutors. Then the prosecutors, together with defense 

lawyers and judges, conduct an “inspection stage” 

that culminates in an adversary trial, at which the law 

enforcement team is required to account for the work it 

has produced. 

It is axiomatic in all industries that end-of-process 

inspection schemes, although they are necessary 

components of quality-control systems, are poor routes 

to achieving overall system quality.8

 Inspection processes 

tend to be captured by the people being inspected: people 

whose principal concern is their own security and who 

learn to “game” the inspection when they cannot evade 

it. Criminal justice practitioners are not exceptions to this 

rule. 

Besides, the criminal trial is designed to protect an 

individual citizen; it is designed to inspect outcomes, not 

to improve processes. A jury that believes it has caught 

a faulty investigation says “not guilty,” but nothing more. 

An appellate court that believes an error is “harmless” 

does not probe further for the sources of the error. The 

inspection is entirely retrospective, and no one claims that 

its function is to analyze the investigative and charging 

processes and make those processes more reliable in all 

future cases.

An Ethic of Shared Responsibility

The aftermath of an exoneration case like Connick v. 

Thompson,20 in which the prosecutors were shown to 

have hidden proof of innocence, embodies a failure in 

forward-looking accountability. 

In Connick, the trial prosecutor withheld crime lab results 

from the defense, removed a blood sample from the 

evidence room, and failed to disclose that Thompson 

had been implicated by someone who had received a 

reward from the victim’s family. The conviction and death 

sentence were ultimately overturned on appeal, but no 

one learned anything from the Connick appellate opinions 

about the deeper, abiding issues in the case’s narrative, 

and those issues were left to surface again in future 

cases. 

From an organizational accident perspective, the question 

that Connick raises is not whether the choices of the 

front-line prosecutors as individuals were wrong; of 

course those choices were wrong. The real question is 

why the mistaken choices seemed to be good choices 

at the time.15 Why did the prosecutors zig instead of 

zag? The answer cannot be that there was a missing 

structural element, because a formal structural element 

was firmly in place: Brady v. Maryland,21 which requires 

the disclosure of exculpatory evidence, unquestionably 

applied to the buried evidence in the Connick case. 

A sentinel event review process would take the 

opportunity to explore that question. It would ask what in 

the prosecutors’ environment motivated their mistaken 

choices and what accounted for the performance of other 

actors. Were the prosecutors so starved of resources by 

the city or state that they felt they could not successfully 

prosecute guilty violent offenders by following the rules? 
Mending Justice 9

National Institute of Justice | NIJ.gov

Had their caseloads crept up to a level where competent, 

thorough practice seemed impossible? Did they feel that 

they were so swamped that they needed to bluff the 

defendant Thompson into a guilty plea by withholding the 

evidence that might have demonstrated his innocence? 

Did supervisory oversight slacken for the same reasons? 

Did they feel acutely vulnerable to irresponsible media 

or political pressure? Or did the prosecutors believe that 

the police department was so under-resourced and illmanaged that no prosecutors could ever convict anyone, 

no matter how guilty, if they played the woeful hand the 

police dealt them? Were they right? Had the prosecutors 

moved by small increments down the inculpatoryto-exculpatory spectrum over the years, withholding 

progressively more exculpatory material but seeing no 

negative impact from doing so? 

Why did the defenders not find the evidence 

independently? Was it poor training? Inadequate funding? 

Caseload pressure? 

Why did the detectives not know about and address the 

lawyers’ failure to make use of the exculpatory facts that 

the police investigation had generated? Why did they 

decide to stand by silently and watch the trial unfold or 

cooperate in the suppression of the facts? After all, the 

police were likely to take most of the public blame for any 

error in the end. Did the prosecutor’s office, over time, 

convince the police that a police practice of “Don’t write it 

down” was a helpful supplement to their own practice of 

“Don’t turn it over”? Were the detectives or the front-line 

prosecutors caught in the classic administrative doublebind: held accountable for an outcome they did not feel 

they had the authority to control or influence?13 Were they 

like the Korean Airline copilots of the 1990s, described 

by Malcolm Gladwell in his 2011 book Outliers, who were 

culturally compelled to sit in deferential silence while the 

senior pilots flew the planes into mountainsides?22

If by studying a sentinel event — with all system 

stakeholders working together in a nonblaming review 

— we learn that the answer to any of these questions 

is “yes,” or even “yes, up to a point,” then we have 

uncovered something that we can address. This is where 

an organizational accident approach to the etiology of 

an error in the criminal justice system is helpful. An 

organizational accident, sentinel event review process can 

allow us to see the consequences of small, incremental 

local decisions (e.g., raising the caseload by 10 cases or 

failing to document a single witness interview) that never 

show immediate and locally visible destructive impacts 

but contribute to emergent tragedies when they combine 

with other small errors and system weaknesses and 

eventually cascade.15

Through a sentinel event model, we can begin to 

recognize where and how correlations that are visible 

from 30,000 feet reach for, and ultimately affect, 

work on the ground. It is a model that makes visible 

hidden correlations between actions that led up to and 

contributed to the event. It does not allow actors to 

escape responsibility, but it does allow them to modulate 

and share responsibility by identifying all who contributed 

to the error and how they contributed. 

Read about precursors from aviation and medicine 

at http://nij.gov/topics/justice-system/documents/

precursors.pdf.

Is Criminal Justice Ready for a Sentinel 

Events Approach?

The question, “Can you build a sentinel event review 

vehicle?” is useful only if it is asked in concert with, “If 

you build it, will they come?” 

A number of precursors indicate that the criminal justice 

system in the U.S. may be ready for a sentinel event 

review process. To name just a few that have occurred in 

recent years: 

■■ The Westchester County, New York, District Attorney 

arranged to have two judges, a former prosecutor 

and a defense attorney examine the wrongful 

conviction of Jeffrey Deskovic.23

■■ The Will County, Illinois, sheriff commissioned a review 

by law enforcement experts in a private consulting 

firm of the near-miss prosecution of a father wrongly 

accused of the murder of his daughter.24

■■ The city of Cambridge, Massachusetts, convened a 

diverse group to conduct an examination of its police 

practices after the highly publicized arrest of Harvard 

professor Henry Louis Gates.25
10 Mending Justice

National Institute of Justice | NIJ.gov

■ The Milwaukee Homicide Review Commission is an 

effective, ongoing interagency group that takes a 

prevention-oriented, public health approach to the 

lessons from individual homicides.26

■■ The Allegheny County (Pennsylvania) Court of 

Common Pleas has explored a case review process 

that uses close examination of cases to illuminate 

chronic issues.27

These and other efforts show that criminal justice error 

reviews can analyze error without resorting to “gotcha” 

humiliation of the sharp-end operators.

For an overview of more criminal justice system efforts, 

see the appendix.

Numerous jurisdictions have demonstrated a broad 

willingness to work — often at a “blue ribbon” level — 

in diverse groups. However, these groups have generally 

focused on creating a product (e.g., a new set of best 

practices) and typically disband once the product is 

produced. They rarely deliver close analyses of specific 

events. Actual innocence commissions that focus on the 

quasi-adjudication of claims of wrongful conviction, such 

as the North Carolina Actual Innocence Commission, 

perform a different, retrospective role. Their goal is not to 

mobilize the culture-changing routine practice of learning 

from error that a sentinel event review contemplates.

Still, all of these efforts are encouraging harbingers. 

Fifteen years ago, a group that included prosecutors, 

police and defenders would have been an anomaly; today, 

it is an accepted approach to examining a perceived 

problem. Significantly, the diverse stakeholders who 

have participated in these efforts often describe them 

as among the most satisfying experiences of their 

professional lives. There seems to be room for the 

system’s adversarial traditions to coexist with an ethic of 

shared responsibility for just outcomes.

As promising as these precursors may be, the fact that 

nothing quite like a sustained, fully developed nonblaming 

approach that engages all stakeholders has yet appeared 

indicates that the course for any further exploration 

should be charted with care. This recognition led me to 

apply for a fellowship at NIJ — and led NIJ to support this 

investigation over the last two years. 

Listening to the Field: Reactions From 

Criminal Justice Stakeholders

In assessing whether a novel sentinel events approach 

could succeed in introducing forward-looking 

accountability into criminal practice, raising the right 

questions with the right people became very important. 

Two sources of relevant questions were readily available. 

The first was the rich body of theoretical and empirical 

literature that examines the diffusion of innovation, 

asking, “Which innovations take hold and flourish? Why?” 

The second was a more recently developed body of 

business literature analyzing the decision of whether to 

bring a new service or product to market. 

Read about the framework for evaluating field receptivity 

at http://nij.gov/topics/justice-system/documents/

field-evaluation.pdf.

Finding the right people to answer the questions derived 

from these disciplines was not a complex task: The idea 

was to talk to as many stakeholders as possible. During 

my fellowship, I engaged many stakeholders in many 

forums. My work included encounters — sometimes 

brief, sometimes extended — with crime victims; victims’ 

advocates; police executives; police investigators; police 

labor representatives; prosecutors; defenders; judges; 

corrections experts; academics from law, criminal justice 

and allied social sciences; journalists; municipal risk 

managers; medical reform and patient safety leaders; 

violence prevention experts; plaintiffs’ lawyers for exonerees; 

civil lawyers defending misconduct cases; and print and 

online publication and dissemination professionals.

I used several vehicles to introduce basic sentinel 

event concepts to these stakeholders and to solicit 

stakeholders’ responses:

■■ Publishing articles in journals aimed at 

stakeholder communities in law, criminology and 

criminal justice, policing and the judiciary. The 

articles presented core concepts from the medical 

and aviation reform movements and the potential for 

a criminal justice sentinel event review for critique.28 

■■ Making presentations to — and receiving 

responses from — stakeholder audiences at 

numerous venues. The venues included the Police
Mending Justice 11

National Institute of Justice | NIJ.gov

Foundation’s “Ideas in American Policing” series, 

the International Association of Chiefs of Police’s 

Wrongful Convictions Summit, the Innocence 

Network’s National Conference, the Executive Session 

on Policing and Public Safety cohosted by NIJ and the 

Harvard Kennedy School, and the National Defense 

Investigators Association’s annual meeting.

■■ Conducting stakeholder interviews. My interview 

outline was based on the diffusion of innovation and 

new service marketing research, but the interviews 

were conversational and allowed practitioners to 

discuss what they felt was most important. 

■■ Organizing a more formal set of focus groups. 

These included a police executive group, a police 

investigators group and a prosecutorial/judicial group 

at the University of New Haven.

The conversations that occurred during my two-year 

fellowship at NIJ indicated that state and local 

stakeholders would welcome an effort to exploit the 

lessons of a sentinel event review process. That put NIJ 

at the threshold of a move forward. Still, there was a 

general sense that added doses of criticism and analysis 

were needed before NIJ could shape a concrete, testable 

effort. My NIJ colleagues were well aware of both the 

general theory of the diffusion of innovation and the 

specific lessons learned during the medical campaign 

against patient injuries. This led them to organize an allstakeholders, expert roundtable modeled on medicine’s 

“communities of insight”: a group with members who 

could critique the application of “culture of safety” 

concepts to criminal justice and mobilize their personal 

networks in diverse practice communities to seek out 

early adopters and — just as important — to hear out 

skeptics.2

This sentinel events roundtable was held in May 2013 

and included police leaders, an elected district attorney, 

defenders, criminal justice researchers, a medical safety 

expert, policymakers, a crime victim and others. The 

discussion exposed this wide range of stakeholders to 

sentinel event concepts, to each other’s concerns, and 

to the findings of various professionals and researchers 

already working in this area.29 It provided NIJ with the 

opportunity to test in greater depth the idea of developing 

a criminal justice version of the culture of safety approach 

that hospital medicine has found transformative. 

And, importantly, the roundtable provided a venue for 

developing the testable questions regarding a sentinel 

event review process, which is crucial to NIJ’s mission as 

a science agency. 

The roundtable discussion, like the interviews I 

conducted during my fellowship, examined the potential 

for further exploration of a sentinel events approach in 

criminal justice and, particularly, how such an effort 

could capitalize on research regarding diffusion of 

innovation and new services development. Ultimately, as 

I found in my interviews of the wide breadth of criminal 

justice stakeholders, the consensus of the roundtable 

participants coalesced around the assertion of Mike 

Jacobson, then the director of the Vera Institute and 

a roundtable member, who said: “If you want to learn 

something, do something.” There was consensus that 

an experimental program — testing the potential of a 

systematic, nonblaming, all-stakeholder effort to learn 

from error in the field — would be a valuable next step. 

From Listening to Doing

After considering responses from the field and the expert 

advice of the roundtable participants, NIJ stepped across 

the threshold from listening to doing and launched the 

Sentinel Event Initiative (SEI). 

In the spring of 2014, NIJ issued a solicitation for 

research to explore issues of organizational change 

and other features that could be unique to using an 

all-stakeholder, nonblaming error-review process in 

the criminal justice system. NIJ also selected three 

jurisdictions to participate as “beta” pilot sites. This 

ongoing project is receiving support from the Diagnostic 

Center of the Office of Justice Programs to execute 

preliminary prototypes of sentinel event reviews. Each 

beta site has formed an all-stakeholders team, selected 

a “sentinel event” in their jurisdiction, and is currently 

engaged in a nonblaming review process. As stated in 

the Introduction to this publication — and consistent 

with NIJ’s belief that the best way to learn is by testing 

carefully framed inquiries in the field — the beta site 

explorations are designed to further refine the “testable 

questions” that a future, more comprehensive experiment 

of a sentinel events effort could examine.
12 Mending Justice

National Institute of Justice | NIJ.gov

As NIJ continues to explore the viability of a sentinel 

events approach to learning from error in the criminal 

justice system, many of the issues identified by 

stakeholders during my two-year fellowship will be 

addressed, including, as briefly discussed below, system 

legitimacy, resources, liability and confidentiality, risk 

management, and leadership and collaboration. 

Professionalism, legitimacy and self-respect

Research shows that people do not obey the law because 

they are certain they will be punished for their violations; 

people obey the law when they trust it and the people 

who administer it — when they are convinced that if they 

do obey the law they will get what they deserve and they 

will not get what they do not deserve.30

Criminal justice stakeholders recognize that the National 

Transportation Safety Board’s post-crash analyses are 

an important source of public faith in the aviation system. 

Many would like to see the day when a district attorney 

announcing an exoneration could say, “We will wait for 

the report to see what went wrong,” and have the public 

believe that the prosecutor has an objective analysis, not 

a whitewash, in mind. But the majority of criminal justice 

stakeholders I encountered do not see that day as having 

arrived, and they see barriers to its advent.

The practitioners who operate at the criminal justice 

system’s sharp end whom I interviewed — the people 

who do the work on the streets and in the courts — were 

not primarily interested in the debate over the precise 

rate of wrongful convictions that fascinates scholars 

and commentators. For conscientious practitioners, 

any wrongful conviction is one too many. Practitioners 

— especially police practitioners — know that every 

innocent defendant imprisoned means a guilty criminal 

left free to find further victims, and this undermines 

public confidence in criminal justice. For sharp-end 

practitioners, wrongful convictions and other errors are 

usually seen as matters of workmanship, professionalism 

and ultimately, self-respect — not of public policy. 

Stakeholders are convinced that the broad participation 

required by sentinel event reporting and review will 

produce its own benefits, distinct from — and potentially 

more important than — the value of the content of any 

reports. The practice of generating organizational error 

analysis can place local criminal justice systems on the 

threshold of a fundamental cultural change. It can provide 

practitioners with a venue in which to express their 

commitment to accuracy. 

One central lesson from the medical experience is that all 

of the contending and isolated communities of practice 

within the hospitals shared a hatred of patient injury, 

providing a common ground on which they can work 

together to evaluate past errors in order to eliminate 

future errors.31

Devising and operating experimental sentinel event 

reviews within local jurisdictions will help determine 

whether the stakeholders’ ethic of shared responsibility 

for just outcomes in criminal justice can sustain a frank, 

nonblaming analysis of events that will allow the public 

to witness the professionalism and commitment of the 

system’s practitioners in action and nourish public trust in 

the system and its operators.

Time and money

A sentinel events effort will not require new buildings 

or new technology or new staff, but it will not be free. 

An experimental effort to examine the feasibility of a 

sentinel event review process could develop an informed 

estimate of the level of local governmental and other 

support that would be needed to sustain the effort as 

an ongoing practice. At this point, it appears that the 

financial support necessary to attract and compensate 

pioneering participants in a criminal justice sentinel 

events effort may be quite modest, but a catalog of 

potential alternative sources of early-stage support, such 

as private foundations, is worth developing. 

One challenge will be to develop a format that does not 

require local officials to sign a blank check in terms of 

staff time. There is, after all, a sense in which there 

is always more to be learned from an event. A key 

product of the preliminary explorations could be a better 

understanding of how much analysis of a sentinel event 

will be enough. In medicine, the Joint Commission 

on Accreditation of Hospital Organizations (JCAHO) 

successfully took the approach of publishing a model 

form for reporting on the “root-cause analysis” of 

sentinel events. A similar model in criminal justice may 

allow stakeholders to weigh their willingness to become 

early adopters of the idea and the efficacy of targeting a 

specific candidate event for a sentinel event review.
Mending Justice 13

National Institute of Justice | NIJ.gov

Liability and confidentiality

Throughout my two-year NIJ fellowship, stakeholders’ 

general reaction to the idea of exploring a sentinel event 

review process was extremely positive. That said, some 

reactions took the form, “Sounds great, but my chief 

(or union or district attorney or defense lawyer, etc.) will 

never go along.” Sometimes these warnings were simply 

about a particular personality, but they often reflected 

deeper concerns. Many of these concerns — including 

inertia and the difficulty of translating the lessons of one 

field to another — are faced when trying to implement 

any innovation, but two key, related challenges stood out: 

concerns over liability and confidentiality. 

It is clear that stakeholders will have to grapple with their 

fear of lawsuits or internal discipline and assess how 

these should be weighed against the potential benefits 

of future risk reduction. No one in local criminal justice 

leadership will willingly expose his or her agency and its 

staff to aggravated financial liability or gratuitous public 

humiliation. Financial cost is not the only — and possibly 

not even the most painful — potential harm practitioners 

fear. Public embarrassment, internal discipline, partisan 

political vulnerability, and harm to individual professional 

reputations are all seen as dangerous. Even stakeholders 

who voice willingness to offer broad disclosure on 

their own part express concerns about exposing their 

colleagues’ actions to review. 

Such concerns should not be overstated at this point. 

The general stakeholder response during my exploration 

was not that these issues make sentinel event analysis 

impossible. Stakeholders seemed to feel that the liability 

challenge was one challenge among others — and 

represents the sort of challenge that people are used 

to working through, not a deal-breaking obstacle. The 

stakeholders recognized that other fields, such as 

medicine, where the liability fears are acute, have found 

ways to cope with issues of liability and confidentiality. 

Still, both real and imaginary liability issues will need 

to be investigated. Will a particular form of reporting 

and analysis prove necessary as we learn more through 

experimentation and exploration across a range of 

sentinel events? In cases of wrongful conviction, it could 

be argued that the worst exposure has already happened. 

In the review of a “near miss” event, liability concerns 

are likely to be significantly less acute, since the most 

catastrophic harm was prevented. Criminal justice 

stakeholders who participated in NIJ’s 2013 roundtable 

discussion and in other focus groups and forums seemed 

to agree that if you are going to be sued, then you are 

going to be sued, with or without a sentinel event review 

process. Indeed, once you have been sued, the usual 

discovery processes of civil litigation require very broad 

disclosure, far more extensive than a sentinel event 

discussion would likely provoke. 

With respect to liability issues, then, the additional, 

marginal costs of engaging in a sentinel event review may 

turn out to be quite limited. And, of course, if the process 

results in systemwide changes that prevent similar errors 

in the future, the cost-benefit analysis might reveal 

that reductions in potential future liability more than 

compensate for the “risks” of transparency. 

Other fields have deployed a broad array of protections 

and procedures designed to meet these concerns. 

JCAHO, for example, offers hospitals reporting sentinel 

events several ways to marshal facts and handle and 

retrieve documents, all designed to protect confidentiality. 

And, within the Justice Department, the Office for Victims 

of Crime has supported significant work on developing 

elder abuse review teams: Elder Abuse Fatality Review 

Teams: A Replication Manual provides an illuminating 

picture of the paths that demonstration projects in 

diverse jurisdictions took in addressing and resolving 

confidentiality questions. 

Each jurisdiction is likely to present its own complex 

legal landscape of peer-review privileges, open-meeting 

laws, attorney work-product privileges, and publicrecord laws. Because the exposure of local stakeholders 

and the vehicles that might shelter them will vary from 

place to place, the most satisfactory resolutions of 

confidentiality issues are likely to be locally designed. 

The considerations may shift depending on, for example, 

whether the organizer of the review team is a local 

judicial entity, a city attorney, a police department or 

a school of criminal justice. In some jurisdictions, for 

instance, a sentinel event review conducted by a school 

of criminal justice or the judiciary might provide a shield 

against Freedom of Information Act and public records 

requests. 
14 Mending Justice

National Institute of Justice | NIJ.gov

Local stakeholder teams may decide to negotiate 

customized agreements for each sentinel event that they 

review. Ultimately, this learning-from-error approach 

envisions the self-conscious study of issues such as 

confidentiality that are (and will always be) implicated in 

the risk/liability calculations. The identification — and 

defense — of a common ground on confidentiality is 

indispensable to sentinel event reviews, and the design 

of future confidentiality provisions should be a common 

subject of investigation as we continue to experiment with 

a sentinel event review process. Simply put, local system 

participants must carefully discuss and agree on — 

through, for example, a Memorandum of Understanding 

— the confidentiality rules under which they will operate. 

Risk management

The hospital environment is no less complex than 

the criminal justice system. Both encompass many 

stakeholders with conflicting and overlapping interests, all 

acutely concerned with potentially devastating exposure 

to professional liability. The modern medical approach 

to accepting error as an inevitable feature of human 

performance and working to provide resilient protections 

against its dangers has paid dividends not only in cutting 

the risks of future error but also in increasing public 

understanding (and reducing the number of lawsuits) 

when errors are voluntarily disclosed.32

There was a general consensus among the stakeholders 

encountered during my reconnaissance that error 

prevention as a risk management issue has moved onto 

(or at least into the range of) the criminal justice agenda 

in their jurisdictions. Fear of liability drives many decisions 

and may provide a goad toward preventive action; yet 

it may also inhibit steps toward all-stakeholder reform. 

Many criminal justice stakeholders feel that the time is 

ripe for considering incident liability and risk-reduction 

concerns in tandem: for an examination that identifies 

and makes explicit the currently submerged trade-offs 

between the strategies and tactics involved in pursuing 

one goal or the other.

Local leadership and collaboration

Initiating a local sentinel event review will require 

innovative leadership. Although someone in a leadership 

role must convene other stakeholder groups and 

encourage them to marshal resources to perform a 

nonblaming, forward-looking sentinel event review, 

the “culture of safety” model requires that no single 

stakeholder can “own” the effort. Leadership does not 

mean control, nor can this type of initiative be rammed 

down the throats of subordinates within or across silos. 

The effort must be collaborative.

Success will depend not only on avoiding single-silo 

dominance but also on reaching beyond the usual actors 

in the criminal justice system’s sharp end. In many of my 

discussions with criminal justice practitioners, including 

at the NIJ roundtable, it became clear that simply 

convening the district attorney, the public defender and 

the chief judge would not fully exploit the SEI model. 

Participation by those familiar players will be necessary, 

but it will also be important to involve other stakeholder 

communities, such as state and local government risk 

managers, victims, employee unions, researchers, 

academics and exonerees. 

Choosing sentinel events

The most productive cases for pioneering sentinel event 

reviews will not always be the biggest or the most 

notorious or the most shocking cases. Many experts 

at the NIJ roundtable noted that there is no particular 

correlation between how much can be learned from an 

episode and its “bigness.” In fact, notoriety might inhibit 

the innovative efforts of early adopters, and smaller 

events could yield the most informative accounts. 

Much can be learned from a sentinel event review 

of “near miss” events that are rarely studied — or 

barely even noticed — now. The narrative of an 

individual mistakenly arrested because of a show-up 

misidentification on the night of a crime and freed six 

months later by DNA results or by the late discovery of 

important cell-phone records33 can be as instructive 

as an exoneration after a trial and sentencing. In fact, 

because documents and memories in “near miss” cases 

are easier to access, a “near miss” episode might be 

more informative. A “near miss” review could be regarded 

as accounting for a sort of success by stakeholders 

(medicine sometimes refers to these as “good catch” 

events), perhaps diminishing fears of liability and 

embarrassment. By cultivating an awareness of the 

value of a “near miss,” or a “good catch,” a sentinel 

event review begins to build a feedback loop into criminal 

justice operations that is currently missing. 
Mending Justice 15

National Institute of Justice | NIJ.gov

The shift of focus from blame to risk in a sentinel 

event review can engage events beyond the traditional 

“spectacular” incident, where public outcry or 

overwhelming media pressure compels a review. One 

of the greatest values of undertaking a sentinel event 

review is that a jurisdiction’s leaders can show that they 

choose to engage — without pressure from the media or 

the public — in a forward-looking process to learn and 

prevent future errors. 

Recognizing limitations and managing expectations

As the criminal justice community explores the value 

of a sentinel event review process, it is important to 

recognize resource limitations and manage expectations 

regarding the final product. Most jurisdictions lack the 

resources required to produce an exhaustive report such 

as the three-volume study produced in Canada after the 

exoneration of Guy Paul Morin.34 The goal should be to be

accurate and useful, not to be perfect, and participants 

should acknowledge the process’s limitations. In the 

end, the shift from assigning blame to understanding 

risk should make the inevitable imperfections and 

gaps in a review record less daunting. Unlike a strictly 

disciplinary review, a sentinel event review should allow 

for the intelligent drawing of inferences and even for the 

consideration of hypothetical alternative explanations.

Transformative Goal and Modest Means

NIJ’s Sentinel Event Initiative marks a cautious first step 

toward an ideal of forward-looking accountability. By the 

“testable questions” it has framed, NIJ indicates that it 

fully recognizes the possibility of failure. When tested, 

routine sentinel event reviews may prove to be a bad idea 

or one of those good ideas that cannot be executed. 

Still, encounters with hundreds of criminal justice 

stakeholders during my two years as a Visiting Fellow 

at NIJ have convinced me that sentinel event reviews 

can be transformative if they can be successfully 

performed. They may present an opportunity for building 

a criminal justice community in which, as in aviation 

and medicine, the lessons of sentinel event reviews 

mobilize a continuous conversation among practitioners, 

researchers, policymakers and citizens. To appreciate the 

power for improving criminal justice outcomes that such a 

community might generate, one need only review the 16 

commentaries by diverse criminal justice stakeholders in 

this publication.

Beyond the benefits that sentinel event reviews would 

provide for the local jurisdiction, they could also offer the 

raw material for a voluntary program of learning from 

error on a much larger scale. A national template for error 

review — enacted locally and informed and challenged by 

diverse local experiences — could substantially mitigate 

the fragmentation of the criminal justice system and the 

isolation of its practitioners. Reading analyses of a distant 

system’s experience could alert practitioners to dangers 

latent in their own local systems. Reading analyses of 

remote near misses could reveal both dangerous latent 

features and potential fail-safe devices or procedures 

that are not present locally. Such a template would 

require a vehicle for sharing the results of local reviews 

— perhaps via a “Wiki” or other online tool. But it is 

precisely this kind of sharing that would promote ongoing, 

interjurisdictional conversations that could counteract 

the endemic tendency of today’s best practice standards, 

which are designed only to provide a minimum floor for 

performance, calcifying into a disciplinary ceiling that 

blocks further improvements. 

At the beginning of the medical reform movement, 

Lucien Leape observed that “[e]rror is an inevitable 

accompaniment of the human condition, even among 

conscientious professionals with high standards. Errors 

must be accepted as evidence of systems flaws, not 

character flaws.”10 The same is true in criminal justice. 

There is no reason to avert our eyes from episodes of 

dishonesty and incompetence when they occur — and 

they do occur. But cutting the risks of future harm 

requires working continuously to understand and repair 

our system, not just slaying the occasional dragon. 

Building a culture of safety in criminal justice can 

begin with a simple commitment to the routine, candid, 

nonblaming examination of as many errors — completed 

tragedies and “near misses”— as we can reach. 

An effort to adopt modern medicine’s experience to 

contemporary criminal justice can hold the researchers’ 

statistical findings in productive tension with the gritty 

narratives of victims, exonerees and front-line operators. 

It can be both modest and ambitious at the same time: 

modest in the financial investment and the degree of 

federal interference required; ambitious in that it seeks 

to change a culture to one that routinely, every day, 

concentrates on improving the reliability of the criminal 

process for the victims, the accused and the public.
16 Mending Justice

National Institute of Justice | NIJ.gov

In medicine, sentinel event reviews helped hospitals 

introduce a transformative culture of patient safety by 

putting forward-looking accountability at the center 

of operational performance. Achieving a comparable 

transformative effect in criminal justice will require 

leadership, thoughtfulness and — perhaps above all 

— collaboration.

There are, of course, no guarantees that the successes 

of aviation and medicine can be transplanted into the 

complex and idiosyncratic environment of criminal justice. 

Still, when the DNA catalog of wrongful convictions 

delivered a shock in the criminal justice world, the 

system’s operators responded to that shock with 

extensive investments of time and energy to try to make 

things right. 

As commendable as these efforts have been, they 

have, for the most part, been isolated within a single 

stakeholder’s stovepipe — and the Sentinel Events 

Initiative seeks to explore whether the return on these 

investments can be compounded if we analyze criminal 

justice errors as “organizational accidents” in which 

complex events comprising small mistakes combined 

with each other and with latent conditions hidden in the 

system to produce unexpected tragedies. Introducing 

an organizational accident approach to criminal justice 

does not call for a domineering, one-size-fits-all federal 

mandate. A nationwide commitment to fostering the local 

practice of routinely developing National Transportation 

Safety Board-style factual reports via a sentinel event 

review process will provide a more accurate and useful 

understanding of the causes and means of preventing 

recurrent errors.

Working steadily on organizational error analysis 

creates an increased system consciousness among the 

practitioners who staff the criminal justice system. The 

forward-looking accountability that this practice creates 

can be an important — and arguably indispensable — 

element of a new criminal justice professionalism. Today’s 

police lieutenants, for example, will make better police 

captains next year thanks to their participation in the 

rigorous organizational accident analysis of a known error 

or near miss. All of the system’s stakeholders will gain a 

better understanding of their individual responsibility for 

the system’s collective outcomes from working on allstakeholder reviews.

About The Writer

James Doyle was a Visiting Fellow at the National Institute of Justice from 2012 to 2014. He is a veteran 

litigator and writer. The former head of the statewide Public Defender Division of the Committee for Public 

Counsel Services in Massachusetts, Mr. Doyle practices law in Boston as counsel to the firm of Bassil, 

Klovee & Budreau, concentrating on the defense of indigent defendants in homicide cases. He is the author 

of True Witness (2005), the history of the collision between the science of memory and the legal system, 

and the co-author (with Elizabeth Loftus) of Eyewitness Testimony: Civil and Criminal, a treatise for lawyers 

in cases involving eyewitness testimony. He has published numerous articles on evidence, race in criminal 

justice, and capital punishment. Mr. Doyle was the founding Director of The Center for Modern Forensic 

Practice at the John Jay College of Criminal Justice. He received a B.A. from Trinity College in 1972, a J.D. 

from Northwestern University School of Law in 1975, and an LL.M from Georgetown University Law Center, 

where he was an E. Barrett Prettyman/LEAA Fellow, in 1979. 
Mending Justice 17

National Institute of Justice | NIJ.gov

Notes

1. Sharpe, V.A. (2003). “Promoting Patient Safety: An Ethical Basis for Policy Deliberation.” Hastings Center Report 

Special Supplement 33 (5) (July/August): S1-S20.

2. Kenney, C. (2008). The Best Practice: How the New Quality Movement Is Transforming Medicine. New York, New 

York: Public Affairs.

3. Chassin, M.R., and Becher, E.C. (2002). “The Wrong Patient.” Annals of Internal Medicine 136: 826-833.

4. Dekker, S. (2007). Just Culture: Balancing Safety and Accountability. Farnham, England: Ashgate Publishing Ltd.

5. Perrow, C. (1984). Normal Accidents: Living With High-Risk Technologies. New York, New York: Basic Books.

6. Vaughan, D. (1996). The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASA. Chicago, 

Illinois: University of Chicago Press.

7. Findley, K.A., and Scott, M.S. (2006). “The Multiple Dimensions of Tunnel Vision in Criminal Cases.” Wisconsin Law 

Review 2006: 291-321.

8. Berwick, D. (1989). “Continuous Quality Improvement as an Ideal in Medicine.” New England Journal of Medicine

320: 53-59.

9. Rudin, J. (2011). “The Supreme Court Assumes Errant Prosecutors Will Be Disciplined by Their Offices or the Bar: 

Three Case Studies That Prove That Assumption Wrong.” Fordham Law Review 80: 537-561.

10. Leape, L.L. (1994). “Error in Medicine.” Journal of the American Medical Association 272, 1851-1857.

11. Gawande, A. (1999, 2002). “When Doctors Make Mistakes,” reprinted in Complications: A Surgeon’s Notes on an 

Imperfect Science. New York, New York: Picador Press.

12. Wachter, R., and Pronovost, P. (2009). “Balancing ‘No Blame’ With Accountability in Patient Safety.” New England 

Journal of Medicine 361: 1401-1445.

13. Woods, D.D. (2005). “Conflicts Between Learning and Accountability in Patient Safety.” DePaul Law Review 54: 

485-502.

14. Reason, J. (1997). Human Error. Cambridge, England: Cambridge University Press.

15. Dekker, S. (2011). Drift Into Failure: From Hunting Broken Components to Understanding Complex Systems.

Farnham, England: Ashgate Publishing Ltd.

16. Saloom, S. (2010). “Adversaries as Allies: Joining Together to Prevent Criminal Injustice.” Albany Law Review 73: 

1235-1243. http://www.albanylawreview.org/articles/06%20Saloom.pdf%20.

17. Mumma, C. (2004). “The North Carolina Innocence Commission: Uncommon Perspectives Joined by a Common 

Cause.” Drake Law Review 52: 647-654.

18. Bosk, C.L., Dixon-Woods, M., Goeschel, C.A., and Pronovost, P.J. (2009). “The Art of Medicine: Reality Check for 

Checklists.” The Lancet 374: 443-445.
18 Mending Justice

National Institute of Justice | NIJ.gov

19. Dekker, S. (2005). Why We Need New Accident Models. Ljungbyhed, Sweden: Lund University School of Aviation.

20. Connick v. Thompson 563 U.S.____ , 131 S. Ct. 1350 (2011).

21. Brady v. Maryland 373 U.S. 83 (1963).

22. Gladwell, M. (2011). Outliers: The Story of Success. Boston, Massachusetts: Back Bay Books.

23. Snyder, L., McQuillan, P., Murphy, J., and Joleson, R. (2007). “Report on the Conviction of Jeffrey Deskovic.” 

Westchester, New York: Westchester County District Attorney’s Office. http://www.westchesterda.net/Jeffrey%20

Deskovic%20Comm%20Rpt.pdf.

24. Andrews International Group. (2010). “Comprehensive Operational Assessment, Criminal Investigative Unit, 

Sheriff’s Office, Will County, Illinois.” December 16. http://www.scribd.com/doc/47496706/Andrews-Report.

25. Cambridge Review Commission. (2010). “Missed Opportunities, Shared Responsibilities: The Final Report of 

the Cambridge Review Commission.” Cambridge, Massachusetts: Cambridge Review Commission. http://www.

cambridgema.gov/CityOfCambridge_Content/documents/Cambridge%20Review_FINAL.pdf.

26. O’Brien, M., Woods, W., and Cisler, R.A. (2007). “The Milwaukee Homicide Review Commission: An Interagency 

Collaborative Process to Reduce Homicide.” Wisconsin Medical Journal 106: 385-388.

27. Barron, B. (2011). “How Case Reviews Transformed Allegheny County’s Criminal Justice System.” http://www.

alleghenycounty.us/WorkArea/DownloadAsset.aspx?id=35204.

28. Doyle, J.M. (2010a). “Learning from Error in American Criminal Justice.” Journal of Criminal Law & Criminology

100: 109-148; Doyle, J.M. (2010b). From Error Toward Quality: A Federal Role in Support of the Criminal Process.

Washington, D.C.: American Constitution Society; Doyle, J.M. (2011). Learning About Learning From Error. Washington, 

D.C.: The Police Foundation; Doyle, J.M. (2012). “Ready for the Psychologists: Learning from Eyewitness Error.” The 

Court Review 48: 4-12; Doyle, J.M. (2013). “An Etiology of Wrongful Conviction: Error, Safety, and Forward-Looking 

Accountability in Criminal Justice,” in M. Zalman and J. Carrano, eds., Wrongful Conviction and Criminal Justice 

Reform: Making Justice. New York, New York: Routledge.

29. National Institute of Justice. (2013). Sentinel Events Roundtable Summary. Washington, D.C.: U.S. Department of 

Justice, National Institute of Justice. http://nij.gov/topics/justice-system/sentinel-events/roundtable.htm.

30. Tyler, T., and Huo, Y. (2002). Trust in the Law: Encouraging Public Cooperation with the Police and Courts. New York, 

New York: Russell Sage Foundation.

31. Clinton, H.R., and Obama, B.H. (2006). “Making Patient Safety the Centerpiece of Medical Liability Reform.” New 

England Journal of Medicine 354: 2205-2208.

32. Mariner, W.K. (2001). “Medical Error Reporting: Professional Tensions Between Confidentiality and Liability.” Health 

Policy Forum, Nov. 6, 1-35.

33. Shane, J. (2013). Learning from Error in Policing: A Case Study in Organizational Accident Theory. New York, New 

York: Springer.

34. Kaufman, F. (1998). “Report of the Kaufman Commission on Proceedings Regarding Guy Paul Morin.” http://www.

attorneygeneral.jus.gov.on.ca/english/about/pubs/morin.
COMMENTARIES
National Institute of Justice | NIJ.gov

20 Mending Justice

Moving Beyond a Culture of Defensiveness and Isolation

By John Chisholm

As an Army 2nd Lieutenant in the 1980s, I participated in a major reform instituted by the United States military to improve system 

performance. It centered on the adoption of an objective process to review all operations conducted across the spectrum, from small 

unit missions to strategic decisions. The reason? Failure. The military services understood they had failed to critically challenge 

themselves during the Vietnam War and the years that followed, resulting in poor performance and, in some cases, tragedy. The 

military recognized that to achieve success in a lethal, complex and challenging environment, it had to develop and explicitly demand a 

culture of accountability in its leaders by teaching them to critically analyze their performance. 

The process was known as the “after-action review,” and it taught leaders to assess every mission, regardless of the outcome, to 

extract valuable lessons learned. The emphasis was primarily on analyzing the things that went wrong as opposed to highlighting 

and emphasizing success. The goal was to encourage leaders to honestly acknowledge and learn from mistakes in training so you 

minimized those mistakes when lives and mission success were on the line. It also encouraged nonlinear thinking and initiative by 

junior leaders (like me), by elevating the status of all participants and treating them as equals.

The military also recognized that most missions involve multiple organizations, often with diverse responsibilities and priorities. Every 

after-action review convened all the system actors to discuss their role and performance. The process did not seek blame; it sought 

clarity and elevated even small support players to coequal status in the discussion. It was not uncommon to learn that the most 

undervalued part of the operation was the primary cause of failure. 

The National Institute of Justice (NIJ) Sentinel Events Initiative looks to develop a similar process in the criminal justice system by 

looking at significant events in the justice process that resulted in failure. I believe this is important because the criminal justice 

system has not developed the kind of systemic accountability culture pioneered by the military and by experts in such diverse fields as 

aviation safety and medicine. 

Let me be clear — there is plenty of appropriate accountability in the existing adversarial system followed by appellate review. But 

that tends, in my view, to reinforce a culture of defensiveness and isolation, where the review is focused on the actions of prosecutors, 

judges and defense attorneys in their respective roles, not on the entire system. My experience as a prosecutor and an elected official 

teaches me that, in the context of public safety, we cannot afford to limit ourselves to viewing the system in such exclusive ways. 

Creating a better justice system requires us to expand our definition of the critical actors involved in any event, from citizens, police, 

corrections, pretrial services, public defenders and defense bar, as well as prosecutors and judges. And we have to create a process 

where everyone feels empowered to speak the truth about his or her role in any given event. 

All-stakeholder, nonblaming, forward-looking sentinel event reviews are by definition retrospective, but if structured properly they 

can have tremendous prospective value in developing the tools to minimize or prevent failure in the future. A wrongful conviction is by 

definition a system failure. High recidivism rates, high victimization rates, crowded and inefficient jails and dockets, and historically 

entrenched pockets of crime are all signs that the system is strained and, as a consequence, more likely to fail in the basic charge of 

protecting public safety. 

NIJ selected three jurisdictions to isolate an event, bring together everyone who had a role in the event, and, in a disciplined, 

structured way, analyze what occurred, what the actors knew at the time they made decisions, and what could be done to prevent 

the occurrence in the future. My jurisdiction, Milwaukee, is one of these. We will analyze a tragic event involving a juvenile on 

supervision for an armed robbery who committed a horrific murder while under supervision in the community. The goal of our review 

of this “sentinel event” will be to better understand how this event happened and examine the implications across the spectrum of 

responsibility. What was the juvenile’s history with the court system, and what risk factors did the system determine applied to his 

situation? Should he have been waived to adult court for the first offense? Should information related to his status have been shared 

with more system actors (juvenile records are closely guarded in Wisconsin)? What were the intervention opportunities, and ultimately, 

how could a similar event be prevented in the future? 

Some of the lessons learned may implicate discrete actions by a select few; others will have systemwide implications, requiring 

policy changes and training, structural reorganization and perhaps even legislation. This, indeed, is the “forward-looking” aspect of 

performing this type of review. With technical assistance provided by NIJ, we hope to create a model tool that can be applied to other 

decision points in varied areas where we experience notable failures. 
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 21

Milwaukee is not unique in the challenges of policing a major urban population afflicted with high rates of poverty, 

unemployment, crime, educational dysfunction and a host of other social challenges. Nor are we unique in our desire to 

address the needs of our community in a fair and effective way. However, the reality is that we rarely take the time to 

reflect because we are consumed by the exigent needs of the present. This deprives not only us but future generations 

of public servants with the lessons learned from hard experience. 

It is human nature to close ranks when bad things happen, and the criminal justice system is a deeply human endeavor, 

reflecting the best and worst of our society. Unlike other systems that have engaged in thoughtful systems analysis, 

like medicine, our nation’s criminal justice system is not infused with the scientific method, nor are we a linear 

authority model like the military, where once the order is given, everybody must comply. But we can learn from those 

systems and incorporate the methods they used to improve their respective fields of expertise.

The criminal justice system is increasingly recognizing the need to open up and collaborate with experts from the 

academic and public health sectors. We can learn from them — and they just might learn a thing or two from us.

About The Writer

John T. Chisholm is the District Attorney of Milwaukee County. He expanded his nationally recognized 

Community Prosecution program, designed a Child Protection Advocacy Unit, and formed a Public Integrity 

Unit to focus on public corruption and a Witness Protection Unit to thwart crime victim and witness 

intimidation. Mr. Chisholm chairs the Milwaukee County Community Justice Council, is a member of 

the NIJ-funded Executive Sessions on community corrections, and chairs the Board of Directors of the 

Association of Prosecuting Attorneys. He also serves on the Milwaukee Homicide Review Commission and 

the Governor’s Council on Offender Reentry boards. Mr. Chisholm is a graduate of Marquette University and 

the University of Wisconsin Law School.
National Institute of Justice | NIJ.gov

22 Mending Justice

To Learn Something, Do Something

By Michael Jacobson

On the wall of the main conference room in the Vera Institute of Justice is a quote from its founding director, Herb Sturz. It says, “A 

wonderful way to develop knowledge is by doing something.” That quote, for me, acutely summarizes my feelings about the next step in 

the National Institute of Justice’s (NIJ’s) Sentinel Event review process. Given the dynamics of criminal justice policymaking in the United 

States, there is a yawning need to build a capacity at local and state levels to conduct post-facto and evidence-based reviews of highprofile or significant events (whether they be wrongful convictions, prison riots, heinous crimes committed by parolees or unjustified use 

of force by law enforcement officers, among hundreds of others) in the criminal justice system. It is an essential government function that, 

I would argue, is in desperate need of this process and the knowledge that would result from any serious effort to plan and demonstrate 

sentinel event reviews.

Why do I say that criminal justice especially is in “desperate need” of a well-structured, substantive, research-based and nonpolitical 

review for meaningful or sentinel events? Because no area of public policy has, over the last 40 years, been more heavily politicized 

and had policy driven by high-profile or sentinel events more than criminal justice. This is especially true of our policies around 

punishment, sentencing and incarceration. Every field, of course, has its own high-profile cases or failures. Doctors make mistakes, 

and patients can die as a result. Yet a case of a doctor who amputates the wrong leg doesn’t lead to the elimination of all amputations. 

High school students can leave school being functionally illiterate, yet that doesn’t lead to the elimination of high school. Faulty 

construction on a bridge can lead to a fatal collapse, yet we still build bridges. In all those cases, these fields have their own version of 

a sentinel event review process that can lead to specific and practical policy proposals to better change practice going forward.

In crime policy, however, a parolee committing a barbaric and sociopathic crime can and does lead to almost immediate policy 

changes — but not based on a fact- and policy- and procedure-based review — no, it is based on the (understandable) collective 

public anger and political outrage that something like that could occur, and the jump to the “obvious” solution that parole should 

be eliminated and sentences severely increased is easy. And that cycle of high-profile crimes is followed by public anger, cries for 

“justice,” and politicians eager to garner the cheap political capital that follows from ratcheting up punishment, eliminating most 

judicial discretion. This results in stuffing our prisons and jails with 2.3 million people — an end result that almost every piece of 

research now says is an unjust and ineffective, in pure public safety terms, use of scarce public resources. 

Crime is not just an emotional issue that is capable of provoking intense personal and public reaction but is also a “democratic” one 

in that almost everyone has an opinion about what to do about crime and criminals — “lock ‘em up, execute him, treat him, etc.” 

Whether you are a cab driver or a brain surgeon, educated or not, an expert or not, opinions will flow like water about what needs 

to be done, whether it be about Richard Davis (a parolee who brutally murdered Polly Klass, an 11-year-old girl in California), Bernie 

Madoff or just a couple of kids who steal from the corner store. Not so with issues like how to build a bridge, how to educate special 

needs kids, how to mitigate environmental disasters, or how to slow the growth of communicable diseases. In those cases, the public 

will (usually) defer to the experts. In criminal justice policymaking, however, the public and their political representatives have become 

the “experts” that drive criminal justice policy — a field that over the last several decades has been almost immune to evidence and 

knowledge in the face of its overwhelming politicization, despite the fact that the entire issue of crime and why some people commit 

it and others don’t and what to do with those who do involves the most complex issues of human behavior, psychology, poverty, drug 

use and mental illness.

Buffering the political winds

So then, the introduction of a process that is at once a buffer between the political winds that have dictated much of our criminal 

justice policy and all the complex issues that are ultimately involved in sentinel events seems like an obvious gap in the field — a 

process, if done correctly, that the public and their political representatives can have some faith in.

Back to Herb’s quote, it is time to do and learn something in this area. The process by which NIJ begins to plan for these sentinel event 

reviews should be rigorous and draw on the best practice in other fields (and there is a lot of best practice in other fields — medicine, 

aeronautics, transportation), but in short order there should be some number (not a lot — start small) of demonstration projects that 

test this notion of expert and evidence-based reviews in criminal justice. Will they all work perfectly? Almost certainly not; even the 

most rigorous planning and design processes do not always result in projects that are successful. But some will be, and even ones 

that fail — because the politics are too difficult, because agencies refuse to work together, because labor/management issues are 

too intractable — will be valuable because we will learn from their missteps, develop knowledge and do it better the next time. And in 

making criminal justice policy, we have a lot of room to do better.
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 23

About The Writer

Michael P. Jacobson, Ph.D., is the Executive Director of the Institute for State and Local Governance at the 

City University of New York (CUNY), where he also teaches sociology in the Graduate Center. From 2005 

to 2013, Dr. Jacobson was the director of the Vera Institute of Justice. Before that, he was a professor at 

the CUNY Graduate Center and the John Jay College of Criminal Justice. He served as the New York City 

Correction Commissioner from 1995 to 1998, and was the city’s Probation Commissioner from 1992 to 

1996 and Deputy Budget Director of the New York City Office of Management and Budget from 1984 to 

1992. Dr. Jacobson is the author of Downsizing Prisons: How to Reduce Crime and End Mass Incarceration

(New York University Press, 2005). 
National Institute of Justice | NIJ.gov

24 Mending Justice

No Sticks: Safe Spaces and a Desire to Get Ever Better

By Maddy deLone

Over the past few years there has been an increased focus on sentinel events, all-stakeholder reviews, and learning from error in 

the criminal justice system. While the fact of wrongful convictions — and particularly what we have learned from studying the DNA 

exonerations — has propelled it, the discussion does not start or stop there. When I was approached by the National Institute of 

Justice to participate in a multistakeholder discussion, my thoughts quickly pulled me back in time to the mid-1980s when I was 

working as a health care administrator on Rikers Island in New York City, running a clinic providing health and mental health care in a 

2,600-person jail. 

At that time, whenever there was an unexpected death (a suicide or a death in a jail, rather than in a hospital), the Mayor’s office (and 

later the City’s Criminal Justice Coordinator) convened a meeting of all stakeholders to review what had happened and to figure out 

what could be done to prevent such adverse outcomes in the future. The purpose of the meeting was always to improve practices. 

The group was referred to as the Prison Death Review Board (PDRB).1

 The group was made up of representatives from all relevant 

agencies and facilities, including representatives from the Mayor’s office, the then-Department of Health and the Department of 

Mental Health, the Department of Correction, the Office of the Chief Medical Examiner, the Health and Hospitals Corporation, the 

Board of Correction (the independent oversight agency of the City Department of Correction), and other health care providers (often 

contract providers) involved in the care of the deceased. 

In preparation for the meeting, the Board of Correction staff gathered all relevant documents and reports from every agency involved 

and interviewed jail staff, prisoners in the area who had known or observed the person who had died, and others who might shed light 

on what had happened. The Board of Correction circulated a draft report prior to the meeting, but the most useful part of the process 

was the meeting that took place to review and supplement the Board’s draft report. 

In those meetings, which were always treated as confidential, I recall robust discussions of what had gone wrong. Whether it involved, 

in a prison suicide, for example, a failure to detect a mental health history on intake, insufficient supervision on a housing block, a 

problem with medication availability for a period preceding the event, problems with noncollapsible hooks (which were often used for 

successful suicides by hanging) or other observed and experienced problems, there was a full airing of the factors that contributed to 

the ultimate tragic outcome. In those meetings, missed opportunities for better collaboration between correctional staff and mental 

health staff were identified, needs for additional correctional staffing in some housing areas could be raised, and the inadequacy of the 

quality or quantity of specific clinical services was broached. Because all stakeholders were present — including the Mayor’s office 

with its ability to change policy at the highest levels of the government — when significant process errors were identified, they could 

be corrected. 

Several elements of the process were important for the successful outcomes:

1. Everyone agreed that the bad outcome was bad, and everyone wanted to prevent it from happening again.

2. Everyone at the table had a role in the outcome. There were opportunities to look inside and outside one’s own agency. In a 

discussion where every player could have done better, but no one could have solved the problem alone, there was real opportunity 

and incentive for problem solving. 

3. The events and the reviews were pretty contemporaneous, so that the suggestions for changed practices and policies had 

potential to save lives in real time. 

4. Someone was tasked with staffing the effort. Board of Correction staff and its consultants were given the time and resources to 

pull together information — it was not an additional task added to already too-full plates of agency staff. The Board staff created 

a synthesized report that formed the basis of the important discussion that followed. 

5. The conversations occurred in a “safe space.” The conversations were protected, much like privileged quality assurance 

discussions in health care settings. This allowed people to admit mistakes more freely on the road to improving the system.

Thinking about transferring this sort of process into the broader criminal justice arena has led me to think about what might have 

made the PDRB process even more powerful. What if, in the case of a jail suicide, the police, prosecutor, defense attorney and judge 

had been in the room? What other ideas about system change might have been suggested? Perhaps the expansion to the “whole 
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 25

Note

1. This process apparently started in the early 1970s in an effort to improve health services. In 1991, when the Board 

of Correction passed its standards governing the provision of health services in the jails, the Prison Death Review 

Board was codified as a required part of the Minimum Standards of the New York City jail system.

system” would have diluted the ability to actually implement changes. Perhaps it would have gotten closer to a rootcause analysis or fundamental reform. 

As local criminal justice systems get together to try out an all-stakeholder review process, the PDRB’s experience 

should provide some useful guidance. The process is possible. Better solutions came about through the PDRB because 

everyone recognized that systemic problems were the cause of preventable deaths. There were no sticks — or threats 

of penalties — in the process, just the desire to do much better. There is a lesson in there. 

About The Writer

Madeline deLone is the Executive Director of the Innocence Project, a position she has held since 2004. 

She began her career as a health care administrator. In the late 1980s, as the Deputy Director of the New 

York City Board of Correction, Ms. deLone was a member of the New York City Prison Death Review Board, 

a multidisciplinary and multiagency group that reviewed deaths in New York City jails. She is a graduate of 

Harvard and Radcliffe Colleges, holds an S.M. in Health Policy and Management from the Harvard School of 

Public Health, and a J.D. from New York University School of Law. 
National Institute of Justice | NIJ.gov

26 Mending Justice

The Dilemma of the Moral Imperative

By Bernard Melekian

By its very nature, the practice of policing produces a conflict between two moral imperatives — to adhere to the legal process and to 

maintain social order. This conflict is exacerbated because these imperatives travel on two distinct, noncomplementary timelines: the 

timeline of the judicial process and the timeline of action on the street. 

In no arena of the criminal justice system is this conflict more evident than when reviewing the actions of the uniformed patrol officer. 

The patrol officer serves as the gatekeeper for entry into the criminal justice system. In essence, everything that occurs after he or she 

acts merely serves to affirm or negate the officer’s original decision. 

Two distinct sets of principles and stakeholders define the conflict between these moral imperatives. On one hand, the officer is 

expected to adhere to the principles of the law and the Constitution. Her stakeholders in this quest are the courts, where she is 

evaluated based on whether or not she adhered to rules, policies and laws. The search for “truth” is often seen as secondary to 

adherence to process. On the other hand, the officer serves both the public and his peers, who demand that he serve the societal good 

— that is, he must maintain social order and affirm the public expectation that wrongdoing is always punished. 

Although the officer may understand intellectually that his role is, in theory at least, merely to serve as an agent of the judicial process 

and not the final arbiter, his experience often conveys an entirely different message. Moreover, it is not automatic — or even easy — 

for a front-line officer to remember that he is part of a system.

In the street, officers are often presented with situations that demand, or appear to demand, instant resolution. From the moment the 

officer arrives at the scene of an incident, pressure to resolve the situation quickly comes both from the people who called and from 

the policing system (i.e., dispatch), which needs the officer to resolve the situation to return to service. This second factor can be 

mitigated by a leadership philosophy that emphasizes problem-solving or by working in an environment in which a significant number 

of calls are not holding at any one time, but the pressure cannot be removed completely.

These conflicting expectations — quick resolution and adherence to process — do not always lend themselves to easy resolution. On 

the contrary, the effort to resolve the moral dilemma often creates a sense of ambiguity and ambivalence that lends itself to lapses in 

ethical decision-making. 

In the movie Tombstone, Val Kilmer’s character, a gunslinger and a gambler, performs an action significantly at odds with his normal 

values: he is found reading a religious book while on his deathbed in a church-run hospital. When asked why, he says, “It appears my 

hypocrisy knows no bounds.”

What he was attempting to say was that the situation in which he found himself did not lend itself to easy adherence to a set of 

principles, which, although logical in the two-dimensional setting of a conference room, did not assist him in the three-dimensional 

world (sometimes called reality) in which he was functioning. Similarly, the patrol officer often finds himself in situations that resist, or 

at least appear to resist, the ability to navigate between conflicting sets of principles and expectations. Life on the street often requires 

“workarounds.” 

Real-world values

The two-dimensional construct asks: What is the officer permitted to do or prohibited from doing? The real-world values construct 

asks: What should the officer do under the circumstances that present themselves? Replacing a rigid set of laws and policies with 

a more straightforward statement of values would allow the officer to make decisions within a broader and hopefully more flexible 

sphere of values orientation.

Supporting this paradigm should be a nonblaming, all-stakeholder review of a critical incident— a sentinel event — that could provide 

a safe environment where practitioners can discuss life on the borderlands where the two moral imperatives meet and sometimes 

clash. Such a review would provide the opportunity for people to articulate the rationale for their decisions. Analysis would not stop 

at whether the officer utilized a “workaround”— whether he or she zigged instead of zagged — but would address why and how the 

system put him or her in a position where that seemed like the best or “least bad” choice available. 
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 27

About The Writer

Bernard Melekian is the president of The Paratus Group, a public safety consulting company. In 2013, he 

retired from a long career as a public servant, most recently as the Director of the Office of Community 

Oriented Policing Services (COPS). Before that, Dr. Melekian was the Police Chief in Pasadena, California, 

where he also served as acting Fire Chief and Interim City Manager. He served in the U.S. Army from 1967 

to 1970 and retired from the Coast Guard Reserves in 2009 after 28 years, including two tours of active 

duty. Dr. Melekian holds a B.A. and an M.P.A. from California State University at Northridge and a Doctorate 

in Policy, Planning and Development from the University of Southern California. 

Even where the decision was ultimately proved mistaken, recognizing that it was made with good intentions might 

provide a means of communicating and refining stakeholder expectations. A discussion based on values compliance 

rather than strictly on adherence to rules might allow us to bridge the conflicting moral imperatives that patrol officers 

face.
National Institute of Justice | NIJ.gov

28 Mending Justice

Front-end and Back-end Solutions

By Dan Simon

It is patently obvious that mistakes abound in our personal lives and professional worlds. Who has never forgotten where they placed 

the car keys, failed to recall the name of a person just introduced at a dinner party, or shown up to a meeting at the wrong time? 

Likewise, every so often, physicians miss critical symptoms, NASA launches space vehicles that are not flightworthy, and politicians 

misjudge the public reaction to their miscues and transgressions. Yet many law enforcement personnel routinely insist that the criminal 

justice system errs rarely, if ever, and many deny reaching an incorrect result in any given case. How, one wonders, could the criminal 

justice process — unique among all other complex social systems — operate flawlessly? 

In truth, the criminal justice process is not, and cannot be expected to be, flawless. The inherently complex process relies on the 

contributions of hosts of legal actors, including witnesses, investigators, lawyers, jurors and judges. The process is driven by these 

actors’ memories, inferences, judgments and decisions, and the ensuing verdicts are unlikely to be any better than their constitutive 

ingredients. These inputs are, of course, the matter of psychological study. A vast body of experimental psychological research 

indicates that although people perform these tasks fairly well, a certain degree of error and bias inevitably creeps in. 

The key challenge facing reformers is how to prevent these errors from affecting the accuracy of the process. Analyses of known false 

conviction cases reveal that, on occasion, the mistaken verdict stemmed from the normal failings that affect every human. We can call 

these spontaneous errors. Far more often, however, faulty verdicts are driven by mistakes that were actually caused or exacerbated by 

the investigative procedures themselves. It is indeed disheartening that investigative procedures, which are intended to merely collect 

evidence, can actually introduce error into the process. We can call these induced errors. The fact that the investigative process can 

induce error should not be surprising, given that the majority of current investigative procedures are based not on scientific research 

but on age-old intuitions and habits that vary widely from one jurisdiction to the next. Psychological research shows the many ways in 

which these procedures can go wrong and provides a detailed framework for correcting them. 

It follows that the focal point of criminal justice reform should be to instill best practice procedures that are based on scientific 

research (coupled with greater transparency of the investigative procedures). By enhancing the accuracy of the evidence on the front 

end of the process, we are bound to reduce the prospect of mistaken arrests, prosecutions, convictions, post-conviction proceedings 

and the punishment of innocent defendants, just as we are poised to reduce the incidence of erroneous releases and acquittals of the 

guilty. 

The logic of best practice procedures is well entrenched in the fields of medicine and aviation. One could not imagine undergoing 

surgery by a physician who resorts to a substandard surgical technique or by a team of doctors and nurses who have not been briefed 

on the case. Indeed, surgical errors have been appreciably reduced by the recent introduction of a three-part presurgical procedure 

that includes verification of details pertaining to the patient and the surgical procedure, a marking of the surgical site, and a timeout 

for ensuring that all members of the team are on the same page. Likewise, one could not imagine boarding an airplane knowing that 

the company uses an outdated flight manual or that the pilots neglected to prepare themselves on the procedures for entering the 

airspace at their destination. 

A vital complement to best practices

But mistakes and near misses occur even when best practices are followed meticulously. This suggests that we need some form 

of intervention at the back end of the process to provide a retrospective analysis of what went wrong. Such sentinel event reviews 

are bound to provide a vital complement to best practice procedures. First, where best practices are not yet in place, sentinel event 

reviews will likely demonstrate the need for introducing them. Second, sentinel event reviews offer a good opportunity to examine the 

limitations, unintended consequences and possible failures of best practices so as to improve them. Third, sentinel event reviews can 

shed light on how best practices actually work in real-life situations, highlighting how they interact with other practices, professional 

skill, constraints and conflicting considerations. Most importantly, sentinel event reviews will provide the occasion and forum for 

communal self-reflection and reinforcement of the values of accuracy, professionalism and integrity. 

In sum, the combined strength of the mutually reinforcing front- and back-end solutions are bound to offer a platform for cultivating 

the kind of scrupulous and inquisitive investigation that befits the solemn task of convicting the guilty and sparing the innocent.
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 29

About The Writer

Dan Simon is a professor of law and psychology — with a focus on criminal justice processes — at the 

Gould School of Law, University of Southern California. He has been a visiting professor at Yale Law School 

and Harvard Law School. Before joining the Gould School of Law in 1999, Mr. Simon was on the faculty 

of the University of Haifa Law School. He also served as a human rights lawyer on the West Bank through 

the Association for Civil Rights in Israel. His book In Doubt: The Psychology of the Criminal Justice Process

(Harvard University Press, 2012) examines the existing empirical evidence regarding weaknesses in the 

investigative and adjudicative processes. He earned an S.J.D. from Harvard Law School, an M.B.A. from 

INSEAD in France and an LL.B. from Tel Aviv University. 
National Institute of Justice | NIJ.gov

30 Mending Justice

Stepping Back to Move Forward: Recognizing Fallibility and Interdependency

By Mark Houldin

A commitment to equality under the law must encompass a willingness to honestly identify and remedy errors — whether the cause 

be individualized or systemic — and continue to strive to prevent violations of fundamental rights. Logical though these principles may 

seem, a root-cause analysis of justice system errors has largely evaded criminal justice policy. 

Having served as a public defender in Pennsylvania at the time of the Luzerne County scandal, I witnessed the reaction to what has 

been called one of the greatest justice violations in American history. In this well-known event, it was uncovered that children charged 

with crimes in Luzerne County juvenile court were denied the most basic and fundamental rights, including the denial of access to a 

lawyer; coercion to plead guilty to minor offenses; removal from their families and communities; and commitment to juvenile prisons 

without regard to the law. What catapulted matters to such a high-profile event, however, was the allegation that the judges were 

receiving financial kickbacks for sending children to these for-profit prisons. During this time, I was representing youth in juvenile court 

in Pennsylvania, as well as assisting in training and supervising new attorneys.

In the wake of the public disaster, all three branches of Pennsylvania’s government participated in the creation of a multistakeholder 

Interbranch Commission on Juvenile Justice. The Commission’s charge was to thoroughly study the circumstances leading to the 

event in order to prevent similar occurrences in the state and to restore public confidence in the judiciary. While not officially a sentinel 

event analysis, the diversity of interests represented and the Commission’s sweeping charge bear striking resemblance to a sentinel 

event framework.

The Commission did not have an easy task. It was attempting to restore public faith in the judicial process while candidly exposing 

systemic flaws that could prevent future injustice. The detailed recommendations issued by the Commission after its review included 

the need for statewide funding for public defenders, implementation of newly created juvenile prosecution standards, creation of 

standards for juvenile probation officers, changes to court hiring procedures, and expedited appellate review for juvenile cases 

resulting in incarceration.1

 Despite the strength and breadth of many of the recommendations, their implementation to rectify the 

latent defects proved challenging.2 Much of the conflict seemed to stem from a difficult balancing act: using the judicial corruption as a 

catalyst for analysis while not casting the entire Pennsylvania juvenile justice system as in disarray. 

When presented with proposed changes in practice affecting local courtrooms and specific cases, previously displayed openness was 

replaced with palpable resistance. The barriers to implementing specific change are best illustrated by a common retort uttered often 

in courtrooms around the state: “We are not Luzerne.” It was widely believed that while what occurred in central Pennsylvania was an 

atrocity, things were far different in “our” jurisdiction. The event was no longer a symptom of broader problems. The Luzerne scandal 

was seen as an outlier produced by a few unethical actors. Practice largely continued as normal, as silent solace was enjoyed in 

knowing that the accused judges were taken off the bench.

This is not meant as a critique of the work done in Pennsylvania post-Luzerne, as the efforts were quite remarkable. Multiple 

stakeholders adopted a common vision of the problem and produced a candid exposition of the many failures that allowed for such 

injustice to occur. Blame was not assigned to any one group; rather, the role of all system actors was examined and real improvements 

resulted. 

This, I think, is an important lesson that can hopefully inform future attempts at learning from error. A learning-from-error culture 

shift at the policy level is a necessary prerequisite to local changes in practice. But at any level, culture change — especially in the 

law — is not simple. Dr. Lucien Leape, one of the key figures in sentinel event review reforms in medicine, wrote that doctors “come 

to view error as a failure of character.”3

 The same could be said for lawyers, who are prone to strive for perfection and internalize deep 

conviction for their positions. In testimony gathered by the Luzerne Commission, for example, the President of the Pennsylvania Bar 

Association was asked why, in his opinion, members of the Luzerne County bar did not file misconduct complaints about the offending 

judges. In his answer, he pointed to an element of acculturation that I believe could fairly be made of professionals in many fields. 

“Behavior starts to be the norm to everybody,” he said, “and nobody thinks things are that far off the mark; or they do, but they are 

uncertain and unsure about what they can do.” 4

Despite these dark times in Luzerne County, I nevertheless found cause for optimism. There was, among many, a willingness to 

embrace the need for improvement. After the Commission’s report was issued, I mentioned my frustration to some peers at hearing 

the comment, “We are not Luzerne.” One countered with a surprising and telling response: We embrace that saying, she said, adding 

that many attorneys who had been assigned to represent juveniles in Luzerne County were happy to be a part of the solution, in 
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 31

crafting a system that lives up to its promise of justice and fairness. Later, when speaking publicly — whether to 

judges, attorneys or community members — many would lead with the phrase, “We are Luzerne.”

Although criminal justice is referred to as a “system,” the processes are less cohesive and complementary than this 

term indicates. Yet, the system’s individual components are inextricably intertwined: actions at one point along the axis 

of the justice process impact behaviors at other points in ways that are often overlooked. The sentinel event process 

that the National Institute of Justice is exploring offers an approach by which this interdependency can be illuminated. 

The more we understand how the whole system operates — be it across stakeholder groups or jurisdictional 

boundaries — the more likely we are to understand how our actions interrelate. The lessons of Luzerne convince me 

that we must become more comfortable talking about the fallibility of law and the criminal justice process. Hopefully, 

the sentinel event learning-from-error approach can move us in that direction. 

Notes

1. Interbranch Commission on Juvenile Justice, Report (Philadelphia, Pennsylvania: Interbranch Commission on Juvenile 

Justice, May 2010), http://www.pacourts.us/assets/files/setting-2032/file-730.pdf?cb=4beb87.

2. My observations are limited to events occurring through 2011 and are not intended as a thorough review of postLuzerne reforms. 

3. Leape, Lucian L., “Error in Medicine,” Jounal of the American Medical Association 272 (23) (December 21, 1994): 

1851-1857. 

4. Interbranch Commission, note 1, at 36. 

About The Writer

Mark F. Houldin serves as the Legal Representation Specialist at the Campaign for the Fair Sentencing of 

Youth. Prior to this, he was Defender Counsel at the National Legal Aid & Defender Association. He worked 

as a public defender for six years, handling a wide variety of criminal matters with a specialization in 

juvenile representation. Mr. Houldin has authored and contributed to a variety of publications and has been a 

frequent presenter on the right to counsel at national legal symposia and guest lecturer on Juvenile Law. He 

received his J.D. from Temple University Beasley School of Law.
National Institute of Justice | NIJ.gov

32 Mending Justice

Egg Heads Matter: Academic/Agency Partnerships and Organizational Learning 

By Jack R. Greene

In many ways, understanding criminal justice sentinel events can be seen to mirror the poem of John Godfrey Saxe recounting the 

Indian proverb of the blind men describing the elephant: each offered different descriptions of the beast depending on what part of the 

elephant he was touching.

It was six men of Indostan

To learning much inclined,

Who went to see the Elephant

(Though all of them were blind),

That each by observation

Might satisfy his mind....

And so these men of Indostan

Disputed loud and long,

Each in his own opinion

Exceeding stiff and strong,

Though each was partly in the right,

And all were in the wrong!

So it has been with understanding sentinel events in criminal justice: to date, our organizational learning and policymaking have been 

based on a fragmented perspective. The sources of such “blindness” are many, but four are briefly considered here. In my view, all 

contribute to semantic and conceptual cloudiness, and each, when addressed systematically, provides an opportunity to improve 

learning and, hence, responses that steer away from such mishaps. Of course, errors in judgment, policy and practice will continue 

to occur, but a focus on an all-stakeholder, nonblaming review of sentinel events can allow us to see the beast for what it is: a highly 

complex system affected by individual failures. Moreover, addressing the four “blind spots” builds on National Institute of Justice (NIJ) 

efforts to bring research to bear on criminal justice policy and decision-making. 

First, my experience has been that although people are the direct actors behind problems (bad decisions of police, prosecutors, judges 

or correctional officials) and are often identified with them (then publicly chastised), those problems are more deeply embedded in 

organizational policies and practices that often go unexamined — but which nonetheless greatly shape the attitudes and actions of 

justice system practitioners. There is a rich literature on organizational learning, organizational accidents (system failures) and sensemaking that suggests that a broader understanding of organizational life must include how the organization takes on and processes 

information, especially negative information. Today we have moved well beyond simplistic notions of environmental pressures leading 

to organizational responses; we now understand that the relationships between organizations and their environments — and indeed 

among and between organizations — are complex. Needless to say, this understanding applies to our justice system as well. 

Our justice system contains overlapping and divergent goals (stopping crime and violence while protecting individual liberties and 

improving institutional legitimacy) and reveals gaps in communications and recordkeeping systems (police arrests, court cases and 

correctional files) among other things. Overlaying all this “systems complexity” is the simultaneous and at times unrelenting influence 

of other actors in the broader policymaking arena (other government agencies, interest groups and private agencies). Although 

identifying systemic problems remains difficult, there are policy-relevant and organizational assessment tools that can shed light 

on these issues; systemic assessment requires systemic involvement. The rise of discussions about transparency in criminal justice 

creates a policy and political motivation and opportunity for such action; we are all in the tent, not just some. 

Second, I have learned over many years studying the police that organizational learning most often occurs when problems are 

surfaced, analyzed and addressed systematically, seeking explanation and withholding blame. In many ways, government agencies 

and programs are too quick to identify failures as person- rather than system-driven, while at the same time seeking broader positive 

reflections on agency programs and their impacts. Simply put, we live in a good-news world. But in science and public policy, learning 

from failures is important, not only to sharpen policy options going forward but also to stop doing things that produce unintended or 

negative results. Like individuals, we sometimes have to clean the criminal justice attic, discarding old regimes and practices and 

making room for new ideas and ways of doing business.
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 33

Third, sense-making in criminal justice organizations often takes its cues from past problems rather than future 

remedies. Research on organizational sense-making suggests that rather than environmental cues reshaping 

organizational thinking and learning, organizations selectively use these cues and interpret them from the view of 

the organization, which is invariably backward leaning. Like many individuals, organizations select information and 

environmental inputs that are most congruent with existing premises (personal or organizational) and in doing so 

are partially bound to repeat historical mistakes. Changing the lens and focus of analysis, as well as the internal 

discourse toward matters of systemic failure, creates opportunities for repositioning organizational policy and practice. 

Current research associated with evidence-based policymaking and organizational legitimacy — as well as specific 

assessments of failures associated with wrongful conviction, failures of eyewitness identification, gaps in forensic 

analysis and the like — all point to the need for greater assessment of the quality and validity of criminal justice 

decision-making. Simply put, criminal justice decision-making and its consequences are rarely individually based; they 

involve complex agency and individual arrangements as well as competing social and political pressures for justice 

agencies to behave in certain ways. 

Fourth, the academic community can play an important, analytically independent role in assessing sentinel events and 

brokering solutions. But it often fails to do so. My academic experience suggests that all too often academics have 

been positioned as social critics rather than honest brokers of information regarding agency success or failure. At the 

same time, agencies are often selective in their presentation strategies, showing only what they want the outside world 

to see. Nonetheless, universities and colleges, as producers and disseminators of knowledge, can play an important 

role in these matters — first by assessing sentinel events, systematically and independently, and then convening 

discussions within and across agencies about the findings of such assessments. NIJ partnerships that have been built 

between the academic community and the criminal justice agency world in the past can serve as an important platform 

for such efforts. 

As a concluding observation, let me reiterate that a failure to learn from history serves only to repeat the mistakes 

of our past. Enhancing organizational learning and sense-making in criminal justice through systematic reviews of 

sentinel events can go a long way to addressing systemic failure and improving justice system legitimacy. To do less 

returns us all to Indostan. 

About The Writer

Jack R. Greene is a professor and former dean in the School of Criminology and Criminal Justice at 

Northeastern University. Recognized as one of the country’s leading police scholars, Dr. Greene is widely 

published, including the Encyclopedia of Police Science and numerous articles, reports and policy papers 

on policing, both domestically and internationally. From 1984 to 1999, Dr. Greene was the director of the 

Center for Public Policy at Temple University, and he has also taught at the University of Wisconsin and 

Michigan State University. He holds a Ph.D. in Multi-Disciplinary Social Science (Sociology, Public Policy and 

Criminology), and is a Fellow of the Academy of Criminal Justice Sciences.
National Institute of Justice | NIJ.gov

34 Mending Justice

An Opportunity We Cannot Afford to Lose

By Greg Matheson

There have been a few moments in my career when I was given the opportunity to participate in or bear witness to significant 

improvements in the criminal justice system. However, before I participated in the National Institute of Justice (NIJ) Sentinel Events 

Initiative roundtable last year, all of them focused on improving the delivery of forensic science services. The 2013 roundtable 

discussion gave me an opportunity not only to consider improvements to the system as a whole but also to provide input at the very 

beginning of developing this type of review process. The goal of this process is to create a structure in which significant errors, or 

“sentinel events,” regardless of where they occur in the criminal justice system, will result in learning and improvement of the entire 

system.

NIJ’s inclusion of a wide variety of criminal justice stakeholders guaranteed that multiple points of view and opinions would be heard 

and discussed. Though there were several differing suggestions as to what the review process might look like and where it should 

focus, it was generally accepted that the criminal justice system would benefit from a broader review of its sentinel events and thus 

acquire the ability to learn from the events and limit their recurrence in the future.

My experience as a criminalist, supervisor and director of a large metropolitan crime laboratory taught me that laboratory errors 

are rarely, if ever, the result of a single action or failure of an individual. By focusing only on the actions of a single analyst, I, as a 

supervisor or manager, would miss the opportunity to improve the laboratory and limit the possibility of the same error occurring 

over and over again. But I witnessed how improvements to forensic laboratory accreditation standards over the years provided crime 

laboratory management with the template to review an error or issue. The accreditation requirement to investigate and determine the 

root cause of an issue provides the opportunity for improvement in the laboratory as opposed to just placing blame. 

A sentinel event in the criminal justice system, just like a laboratory error, does not occur due to a single action, individual or entity. 

Many failures must occur for an error to get through the entire criminal justice system and result in a failure of justice. However, law 

enforcement, including forensic science, is frequently the source of the first event in a sequence of system failures that eventually 

results in a sentinel event. As such, it is easy to see how a law enforcement individual or agency might be blamed because the error 

occurs, as Jim Doyle characterizes it, at the “sharp end of the stick.” The individual or agency is punished and the system moves on, 

failing to learn from the failure and take steps to prevent its recurrence. This process of focusing on blame, as opposed to learning and 

improvement, has resulted in the creation of negative chasms between the different “sides” in the process. A blame-oriented process 

often compels individuals to focus on avoiding blame (and, hence, punishment), which leads to less transparency and discourages 

sharing information. As a result, the entire system loses. By accepting and participating in a sentinel event review process, which is 

nonblaming and includes all stakeholders, each participant in the system can recognize and take responsibility for contributing to the 

event and improve the chances of it not recurring.

Learning from medicine and aviation

In the roundtable discussion, we were given the opportunity to hear how the aviation and medical professions deal with sentinel 

events, how their processes were developed, and how improvements to their processes continue. Being presented with these 

examples was helpful because it demonstrated how these fundamentally different fields approached the nuts and bolts of a learningfrom-error process. In bringing sentinel event review to the criminal justice system, we will, of course, need to forge our own 

specific process. The adversarial nature of our criminal justice system will make the process of developing a viable all-stakeholder, 

nonblaming, forward-looking sentinel event review process difficult. However, I believe that if we look at the system’s adversarial 

nature as a valuable checks-and-balances process — as opposed to the regularly held opinion of winning at all costs and placing 

blame — a nonblaming sentinel event review process can be developed.

At the roundtable, we held general discussions as to what constitutes a sentinel event and how we might initiate the development of 

a review process. The discussions were spirited and reflected the significant diversity of opinions and viewpoints consistent with the 

diversity of the participants. However, one thing became abundantly clear: For this idea to take root, we as a community must not 

get bogged down in the immensity of the issues but rather start with an obvious failure of the system, such as a wrongful conviction, 

create a system to deal with it, then expand it as our experience grows. By starting with a very focused issue, we can learn both the 

strengths and limitations of the system and the process by which a sentinel event review will work.
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 35

As the roundtable drew to a close, we reached another important agreement — a criminal justice system that values 

justice over anything else is paramount, and developing a sentinel event process that will guarantee continued 

improvement is an opportunity we cannot afford to lose. Moving forward with this concept is essential.

About The Writer

Gregory B. Matheson is currently the President of FSLResources, which provides forensic science 

supervisors and managers with the tools they need to improve the delivery of forensic science services 

to the criminal justice system. In 2012, Mr. Matheson retired from the Los Angeles Police Department 

as the Laboratory Director. As a criminalist, he was court-qualified in toxicology, serology, crime scene 

investigation, and the examination of explosives, flammable liquids and vehicle lamp filaments. During his 

career, Mr. Matheson has held numerous leadership positions, including on the board of directors of the 

California Association of Criminalists, the California Association of Crime Laboratory Directors, the American 

Society of Crime Laboratory Directors and the American Board of Criminalistics.
National Institute of Justice | NIJ.gov

36 Mending Justice

The Blame Game

By Jennifer Thompson

For some reason, it makes us feel better about ourselves, our neighbors and our world when we have someone to blame. It makes 

sense for us if we can find fault. Perhaps because it takes the pressure off of ourselves, making us feel absolved of responsibility or 

accountability. But blame and fault have never answered the big questions, such as, “How did this happen in the first place?” Why 

didn’t someone stop this before people were hurt? And if there is a solution, how do we implement it? Blame and fault-finding are 

simply Band-Aids on a large and hemorrhaging wound. 

I understand this gut reaction to find someone or something to blame. As a victim of a brutal rape, clearly the rage and hatred I felt 

towards my attacker was and is understandable. There was a clear victim and obvious perpetrator. But when a DNA test showed that 

the man I had picked from a photo lineup, a physical lineup and in court was innocent, where and on whose shoulders does the fault 

lie? After 11 years of false imprisonment, Ronald Cotton walked out a free man. His family cried, the crowd cheered and the media 

was enchanted by his gentle manner. Suddenly there was a new victim and someone had to be blamed, and that someone became 

me. I, the victim of a violent rape, took the place of the offender. Revictimized, but with a twist. “Have you heard from the girl?” “Are 

you angry at the girl who picked you?” “How could you ever forgive HER?” No one could have punished me to the degree I punished 

myself. Fear, shame and guilt were my daily diet of choice. And I was alone to digest it all.

Ronald’s forgiveness was an enormous gift; it not only freed my heart but also my body. Our friendship has been a blessing, and I am 

grateful for it every day of my life. But what few people know and understand is that the public has been slow to grant me the same 

kindness, if it does at all. As I have traveled throughout the country and in speaking with the public, I am constantly seen as the villain, 

the person who “did this” to Ronald. Without fail, a person (sometimes several) raises their hand to ask a question: “Mr. Cotton, how 

can you forgive her? I could never forgive someone who did that to me.” I sit there, knowing that it was not me that did this to Ronald; 

it was Bobby Poole who did this to us — and it was a series of events that led me to pick Ronald out. There were systemic problems 

that helped to contaminate my memory and create the perfect storm. And yet, once again, I am alone. Another trauma, one more 

nightmare, misunderstood. 

For 16 years, I have put my face out there, the face of mistaken eyewitness identification. The poster girl for getting it wrong. I will 

never regret doing this as I know I did it for all the right reasons, trying to make sense of what happened to me and to Ronald; but at 

times I have felt like a piñata, and rarely do I share these stories of what it has been like for me, because I sometimes get disgusting 

comments from men and judgments from readers of the book that Ronald and I wrote ... which have created wounds of a different 

sort.

How do we fix it?

At the National Institute of Justice’s Sentinel Events Initiative roundtable in 2013, as I sat in a room in Alexandria, Virginia, listening to 

professionals and scholars discuss how to bring nonblaming, all-stakeholder reviews of wrongful convictions into the justice system, I 

felt like a fish out of water. While everyone had amazing ideas to contribute and brilliant thoughts surrounding his or her field, I thought 

to myself (as I often do during conferences), “But how do we fix this right now?” There is real suffering going on right now. Jimmy, 

Willie, Ryan and Kalvin don’t have time to discuss which model — the review process that has worked in aviation or medicine — 

should criminal justice try to adopt. Regina, Jennifer, Yolanda and Debbie need to be protected now — we can’t afford to talk about 

what we are going to call the ongoing effort to bring a sentinel event review process into the justice system. And yet the problems and 

solutions are so huge, which finger do we put in what leak — and in which dike?

So let’s stop the blaming and searching for someone to burn at the stake. It will never solve the problems; it merely distracts us from 

what needs to be done. For me, it is a trickle-down effect. Policies need to be reformed, and better training and education are needed 

for those who are entrusted to protect and serve. I do believe a Sentinel Events Initiative would be of value. And I also believe that the 

victims in cases where mistakes have been made should have a seat at the table. That means victims like me, the crime victim — and 

victims like Ronald Cotton, the “system” victim. If all voices from across the criminal justice spectrum are present, a sentinel event 

review process could go a long way towards an open and honest discussion about the realities and inherent problems that impact not 

only the wrongfully convicted, the victims and their families but also the communities in which we live and our sense of trust in those 

who are required to protect and serve. 
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 37

About The Writer

Jennifer Thompson is an advocate for judicial reform. Her strong convictions were born of a brutal rape she 

suffered; her mistaken identification of the rapist was a factor in an innocent man, Ronald Cotton, being 

sent to prison — not once but twice — for the crime. Mr. Cotton was eventually freed nearly a decade after 

his first conviction when DNA testing proved his innocence. Since then, Ms. Thompson has successfully 

lobbied legislators in North Carolina to change laws so that Mr. Cotton and other wrongfully convicted 

people could be more generously compensated for mistakes made by the criminal justice system. Ms. 

Thompson has served as a member of the North Carolina Actual Innocence Commission, which instituted 

procedural reforms throughout the state, and she is currently a member of the advisory committee for 

Active Voices and the Constitution Project. 
National Institute of Justice | NIJ.gov

38 Mending Justice

Innocence Commissions: The Case for Criminal Justice Partnerships

By Russell F. Canan

The Innocence Project, a nonprofit legal clinic affiliated with the Benjamin N. Cardozo School of Law at Yeshiva University, has 

identified 316 post-conviction DNA exonerations in the United States since 1989. With the objective of ensuring the innocent not be 

arrested, tried, convicted, and sentenced, as well as convicting and sentencing those who commit crimes, many jurisdictions have 

examined the issue of wrongful convictions. Since 2000, several states have established innocence commissions to investigate 

wrongful convictions and propose reforms to the criminal justice system. Various branches of government and a bar association have 

used different means to establish these commissions. In five states, for example, judicial orders or efforts of judicial officers led to the 

creation of innocence commissions. Six state legislatures have passed laws forming commissions. One state bar association and one 

state governor created commissions. The active participation of the key actors in the justice system has proved to be a central factor 

in assessing the causes of wrongful convictions and creating proposals for meaningful reform.

In 2011, the Superior Court of the District of Columbia embarked on a close look into the causes of wrongful convictions. Following 

several local exonerations and at the suggestion of the Public Defender Service for the District of Columbia, Chief Judge Lee F. 

Satterfield established the Ad Hoc Committee on Wrongful Convictions — composed of judges, prosecutors, defense lawyers, police 

officers, members of the executive branch, a legislator and a scholar — to determine whether the District of Columbia needed an 

innocence commission. 

Whether judges should be involved in such a committee was, however, robustly debated within the Superior Court. Some argued it 

would violate the separation of powers principle if judges appeared to legislate or appeared to tell the executive branch how to do 

its job. Along these lines, Chief Justice John G. Roberts, Jr., famously described the role of a judge as limited to that of an umpire, 

calling balls and strikes. It is beyond dispute that the neutrality of an umpire is essential when a judge adjudicates cases. In the 

context of innocence commissions, however, judges have a different role, and their participation in and leadership of reform efforts are 

appropriate.

Chief Judge Satterfield ultimately reached the conclusion that judicial participation was consistent with the court’s role in achieving 

justice for all. In his letter creating the Ad Hoc Committee on Wrongful Convictions, he stated:

The Superior Court continues to be committed to adhering to the highest standards of justice for the residents of the District 

of Columbia. The case of Donald Gates has prompted this Court, and the broader criminal justice community, to reflect upon 

how we can improve upon these standards, and work together to ensure that no innocent person is convicted or imprisoned. 

We have already begun that process and we intend to continue, subject to the constraints of the separation of powers and 

the overarching imperative to preserve the independence of the judicial branch.1

With Chief Judge Satterfield’s guidance in mind, the Ad Hoc Committee on Wrongful Convictions commenced its project in 2011. After 

nearly two years of work, it found that the District of Columbia met or exceeded best practices regarding false confessions, pre- and 

post-conviction access to DNA testing, access to post-conviction representation, evidence preservation, resources to the defense bar, 

and remedies for those defendants who are exonerated. Concrete reforms were proposed in the areas of eyewitness identification 

procedures and policies concerning informants. Additionally, the Committee recommended monitoring the performance of the recently 

established District of Columbia Department of Forensic Sciences to evaluate the reliability of forensic evidence in court.2

The experience in the District of Columbia demonstrated that all branches of the government, especially the judiciary, ought to be 

included in innocence commissions that propose reforms to the criminal justice system. Involvement from all criminal justice actors 

ensures that people who are on the ground and aware of the system’s strengths and weaknesses are able to supply the most accurate 

information for a commission’s analysis. Furthermore, these partners are the ones best poised to implement the recommendations 

made by such a commission.

Judges are in a uniquely advantageous position to facilitate the functioning of innocence commissions on a very practical level 

because of their role in the criminal justice system. First, judges — naturally regarded as authority figures in the criminal justice 

context — are able to set a tone of formality, cooperation, efficiency and order. Second, as trained, neutral mediators, judges can 

effectively manage the often contradictory viewpoints among participants who are traditionally adversaries. Third, judges, free from 

the duty of advocacy, can contribute a unique perspective that is essential to formulating appropriate reforms. Finally, as disinterested 

members with a deep knowledge of the criminal justice system, judges can guide an innocence commission toward reforms that are 
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 39

Notes

1. January 11, 2011, at 5, available at: http://www.dccourts.gov/internet/documents/

OIGReportLetterFromChiefJudgeSatterfield.pdf.

2. Letter from Judge Canan to Chief Judge Satterfield regarding the Findings and Recommendations of the Ad Hoc 

Committee, February 12, 2013; see also Findings and Recommendations of the Ad Hoc Committee, February 12, 2013, 

available at: http://www.dccourts.gov/internet/documents/Ad-Hoc-Committee-Findings-and-Recommendations_

2-12-13CORRECTED.pdf.

About The Writer

Russell F. Canan was appointed to the Superior Court of the District of Columbia in 1993. He has worked in 

the Criminal and Civil Divisions and in Family Court and has served as the Presiding Judge of the Criminal 

Division, Chair of the Criminal Advisory Rules Committee, and Chair of Jury Management. Throughout his 

career, Judge Canan has been an adjunct professor at the Antioch School of Law, Georgetown University 

Law Center, the George Washington University Law School, Washington College of Law, and the University 

of the District of Columbia School of Law. He received his J.D. from Antioch School of Law. 

both practical and necessary. In these ways, judicial participants are acting more as mediators and less like activist 

policymakers and, therefore, honor the separation of powers principle. 

Judges promote the highest standards of justice by participating on committees exploring reforms of the criminal 

justice system. All criminal justice actors should be invested and have an active role in preventing wrongful convictions 

while pursuing society’s interest in convicting the guilty. The history of innocence commissions has demonstrated 

that a collaboration of criminal justice partners — such as that envisioned by the National Institute of Justice in its 

exploration of a sentinel event review process — can produce results that will benefit all.
National Institute of Justice | NIJ.gov

40 Mending Justice

High Expectations, Good Intentions and Normalized Policy Deviation: A Sentinel Event

By Jim Bueermann

When I was the police chief in a midsized Southern California city, I watched any number of “sentinel events” play out. I witnessed 

first-hand how effective a sentinel event review approach can be — and, although policing represents only one component in our 

complex criminal justice system, I believe that this approach promises similar dividends if it can be applied within the system’s other 

components and, most importantly, to the system as a whole. 

In my department, one of our clear officer safety policies required that two officers respond to every burglary alarm. Not unlike other 

departments, however, 98 percent of our burglary alarms were false. Over time, environmental factors began to erode this two-officer 

policy; as patrol forces were steadily downsized, supervisors who “ran out of officers” during a busy shift would occasionally handle 

an alarm call by themselves, playing the odds that the call was going to be a false alarm. What began as a periodic practice increased 

at an incremental rate until it became commonplace. In other words, it became a normalized policy deviation.

One day, on a busy shift, a residential burglary alarm call came in from an outlying area where no officers were working. The 

dispatcher informed the shift sergeant that no officers would be available to handle the call for at least an hour due to higher priority 

calls. Based on past evidence, the sergeant believed the alarm was probably false (and therefore “no big deal”), so, rather than holding 

the call for an extended period, he decided to handle it by himself. 

If you were casting a movie and in need of a prototypical police sergeant — big, athletic, clearly capable of leaping tall buildings in a 

single bound — this was your guy. Before joining the department, he had worked as a Customs Drug Interdiction Officer, swooping in 

on Blackhawk helicopters to stop major drug-smuggling operations. He was on the department’s SWAT team, was in great shape and 

had a dynamic, biased-for-action personality. His officers loved him because he cared about them, put their interests first, and always 

tried to make their work lives easier.

When he arrived at the residence where the silent alarm had been activated, he was confronted by a 4-foot wrought iron fence that 

surrounded the house. Spike-like tips topped the fence’s vertical bars. But this posed no obstacle to the sergeant who wanted to 

handle this “nuisance call” ASAP and get back to helping his officers with “real” police work. So, he took a short run at the fence, and, 

in his best hurdler’s form, vaulted perfectly over the fence. 

Well, almost perfectly. His leading pant leg caught on one of the fence spears, causing him to begin a tumble-like fall over the fence. 

While most of him came down on the backside of the fence, his trailing thigh was impaled on one of the spikes. He was stuck, hanging 

halfway off the fence, with a 4-inch spear holding him to the top of the fence. Although he tried to pull himself free of the spike, he 

could not — so, about to pass out from extreme pain, he radioed for emergency assistance. 

The burglary alarm was false.

The sergeant had made a bad choice and violated a policy — and this could have been a straightforward disciplinary matter. But, by 

making it clear that we were not interested in blame, I learned, through the leadership debriefing, why the policy violation had seemed 

like the right choice to the sergeant at the time. And, most important, I, as chief, and the entire department learned why it might seem 

like the right choice to another supervisor in the future unless we made some changes.

This is one of the central features of the sentinel event review process that the National Institute of Justice is exploring: not attempting 

to affix blame, but, rather, to determine why something happened in an effort to prevent it from happening again. In my department’s 

analysis of the incident with the sergeant, we concluded that the following factors had contributed to the event:

■■ The department’s crime control strategy emphasized fewer officers assigned to patrol duty and more to problem-solving units.

■■ These policies meant that, during busy shifts, patrol officers went call to call; this contributed to a widely held belief among patrol 

officers that there were never enough police officers to meet the public’s demand for service, which, in turn, resulted in sergeants 

trying to keep “nuisance calls” from their officers as much as possible.
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 41

About The Writer

Jim Bueermann is the president of the Police Foundation and a former Executive Fellow with the National 

Institute of Justice. Mr. Bueermann worked for the Redlands Police Department for 33 years, serving in 

every unit within the department, including as Chief from 1998 to 2011, during which time he developed 

a holistic approach to community policing and problem solving that consolidated housing and recreation 

services into the police department, a strategy recognized as one of the country’s 25 most innovative 

programs by Harvard Kennedy School’s Innovations in American Government. He is a graduate of California 

State University, the University of Redlands, the FBI National Academy, and California Command College. 

■■ The chief and community had high expectations of the department’s performance — and sergeants knew that 

their performance was evaluated, in part, on their shifts meeting these perceived expectations.

■■ Officers were aware of the general public’s criticism of their salaries and benefits — and knew that a key to 

retaining them was to keep service levels as high as possible.

■■ Ninety-eight percent of all silent burglary alarm calls were false alarms.

■■ There was a widely held belief among patrol personnel that rapid response to calls for service prevented crime, 

which resulted in a hypersensitivity to “holding” calls for more than just a few minutes. 

■■ The culture of policing in general, and specifically in our department, emphasized an action-oriented bias.

■■ Over time, sergeants had normalized their deviation from department’s two-officers policy for responding to 

burglary alarms to minimize hold times during busy shifts — and had done so without problems.

■■ The department lacked a mechanism for gauging the actual behavior of its members within the context of 

workloads, expectations and policy.

The incident resulted in some stitches, a bruised ego and — after it became apparent that the sergeant would 

recover fully — a lot of good-natured kidding. But it could have turned out very differently. I firmly believe that our 

open discussions allowed us to consider the facts much more strategically and, in particular, to understand how the 

unintended consequences of good intentions and high expectations were affecting decision-making. By approaching 

our debriefing of this incident — this officer-safety “sentinel event” — from an “organizational accident” perspective, 

we truly learned from this error and were able to make important changes to the department’s processes. 
National Institute of Justice | NIJ.gov

42 Mending Justice

Using Sentinel Events to Promote System Accountability

By George Gascón

The impact of a wrongful judicial outcome in a criminal case can have a reverberating impact on entire communities. Whether real or 

perceived, the wrongful conviction of an innocent person or the wrongful acquittal of a guilty defendant can shake a community’s trust 

in the criminal justice system. These long-term effects can last for years, resulting in increased apathy and cynicism and, in extreme 

cases, lead to civil unrest. It can also result in violent criminals escaping justice and sometimes victimizing others.

Although wrongful convictions and acquittals have devastating consequences for all involved, including the impacted communities, few 

systemwide solutions have surfaced to deal with the leading causes of wrongful judicial outcomes. The structure of our criminal justice 

system and the burden of proof beyond a reasonable doubt favor the value of preserving innocence — meaning that, as a society, 

we value the presumption of innocence and try to ensure that it is afforded to all defendants. Incapacitation of an innocent person is 

seen by many as more harmful than letting a guilty person go free. However, despite the best efforts of all of us who are committed 

to upholding ethical standards that protect individual liberty and public safety — both in the law and in our respective professions — 

wrongful outcomes still occur. 

Often, the face of wrongful judicial outcomes involves wrongful convictions. These cases rightly garner substantial media attention. 

Whether the case in question includes technical errors that exacerbate potential bias or new evidence emerges, the system has the 

duty to remedy the wrongful judicial outcome. Further, when the wrong person is convicted of murder or other serious crimes, the true 

assailant is out, possibly hurting others, and an innocent person’s life is ruined behind bars. Citizens are naturally in the position to 

question, “How could this happen?” And there are also cases where the person responsible is found “not guilty.” These miscarriages 

of justice can be equally troubling.

Unfortunately, in either scenario it may take years, if ever, to uncover the mistake or mistakes that led to the harmful outcome. In 

addition, corrective measures are mostly focused on the actions of those directly involved with the case and little, if any, attention is 

given to the systemic failures. These cases are mostly viewed as aberrations within what is perceived as an otherwise well-running 

system. Few want to acknowledge that the frequency of wrongful judicial outcomes is probably substantially greater than what 

is readily known. Therefore, little effort is directed toward a systemic look at the problem, which, if pursued, would increase the 

professional and lay community’s understanding of the dynamics that facilitate these mistakes. Instead, the focus is left on what went 

wrong and whom to blame on a case-by-case approach.

Making systemic improvements

To make systemic improvements, we must create a safe environment, free of blame, for all criminal justice stakeholders. Under the 

leadership of the U.S. Department of Justice, practitioners and scientists must be able to come together with a clear direction to 

review a representative number of cases and identify breakdowns in decision-making throughout the judicial system. In the same way 

that an epidemiologist would set about finding pathogens and identifying a cure or treatment for the medical condition they cause, 

criminal justice system stakeholders must explore the root causes of wrongful judicial outcomes through an independent review. 

Under the safety of this depersonalized scientific quest for knowledge, effective solutions can be developed and new systems can be 

implemented that would correct existing problems and create an effective quality assurance loop. Continuous process improvement to 

both protect innocence and hold the guilty accountable could become the mantra for this movement.

In addition to the reflective examination described above, I believe that individual criminal justice stakeholders can continue to invest 

in basic prevention efforts while being engaged in the more systemwide, all-stakeholder reviews that the National Institute of Justice’s 

Sentinel Events Initiative envisions. One essential step that I have taken as the lead prosecutor in San Francisco is to require that all 

of my staff receive wrongful conviction bias training. The training curriculum includes material on common contributing factors to 

bias and case studies from across the nation. This academic and case review is then combined with a viewing of the documentary 

film After Innocence. This film provides personal accounts that humanize those harmed by wrongful convictions, from prosecutors 

to defendants, and challenges viewers to confront the consequences of wrongful judicial outcomes. I feel that it is essential that 

prosecutors be reminded that quality case review reduces bias, and I ensure that my office brings the most appropriate charges and 

only files cases where we can prove beyond a reasonable doubt that the defendant committed the crime. The appropriate time must 

be devoted to each case to ensure that the prosecutor meets this ethical burden. Time and proper review are tools to reduce the 

possibility that a prosecutor may subconsciously filter for evidence that proves guilt while ignoring evidence that does not. 
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 43

About The Writer

George Gascón is the District Attorney for the City and County of San Francisco. He created the nation’s first 

Alternative Sentencing Program to support prosecutors in assessing risk to determine the most appropriate 

course of action, and has created California’s first Sentencing Commission in an effort to reform sentencing 

by applying evidence-based practices to prosecution. Mr. Gascón has 30 years of experience in law 

enforcement, including service as Assistant Chief of the Los Angeles Police Department, Chief of Police in 

Mesa, Arizona, and Chief of Police in San Francisco. He holds a B.A. from California State University and a 

J.D. from Western State University, College of Law. 

I believe that using a sentinel event review process to examine judicial outcomes will support prosecutors in 

maintaining public trust and ensuring public safety. 
National Institute of Justice | NIJ.gov

44 Mending Justice

Cold Case Homicides: Ideal Candidates for Sentinel Event Review

By Frank P. Tona

A sentinel event — whether an outright error or a “near miss” — can devastate the lives of those who have been involuntarily placed 

into the criminal justice system. From my perspective as a cold case detective, avoiding the types of mistakes that can result in a 

sentinel event should be at the forefront of every law enforcement agency’s policies and practices. 

One example of a sentinel event is a cold or long-unsolved case. Cases go cold for a variety of reasons: misidentifications by 

witnesses, overzealous prosecutions, ineffective counsel and poor evidence-handling practices, for example. Whether or not these 

cases are ever resolved, a longstanding cold case could provide a rich opportunity for a nonblaming, all-stakeholder sentinel event 

review. 

The investigation of cold case homicides is a deliberate, ongoing process, where every step in the process is aimed at achieving timely 

justice. No homicide investigation begins as a cold case in the making, but as good-faith decisions are made by the investigators — 

decisions intended to lead promptly to a just outcome — errors can accumulate that have unintended consequences. For example, 

an investigator may use a specific approach to interrogate a suspect or may focus on the collection and analysis of certain evidence 

from the crime scene. Certain steps in the investigative process may be assigned to less experienced investigators who may miss 

important clues or misinterpret forensic data. It is likely that these errors will go undetected — and, therefore, unquestioned — as 

long as the case remains cold and unsolved. Even in cases where a long investigation results in a wrongful conviction, decisions along 

the investigative trail rarely are subject to the kind of thorough review that would identify systemic flaws that got the investigation off 

track.

I see great value in treating a cold case as a sentinel event. An all-stakeholder, nonblaming review could examine the questions, “Why 

did the case stay cold for so long?” “Did investigators rely too heavily on practices that may have yielded positive outcomes in the 

past and fail to consider alternatives?” This kind of review would allow us to examine the role of tunnel vision in the case going cold. 

I have seen from my own investigations that it can be very difficult to drastically change momentum during an investigation once I 

have established a clear and convincing theory. Equally difficult is attempting to convince colleagues their theories are not consistent 

with the evidence or statements made by witnesses or suspects. Only later, when the direction of the investigation can be measured 

against the outcome of a correct or wrongful conviction, can we really challenge our assumptions and discover the errors committed 

along the way. And, a sentinel event review might provide a type of benchmark to guide other cold case investigations.

This is not to say that adopting a nonblaming sentinel event review process will be easy. The culture within police departments is 

frequently conservative, with specific belief systems and, often, a bureaucratic management structure. Departments can become 

victims of their own policies and antiquated practices — and police managers are sometimes reluctant to make procedural changes, 

fearing that they could result in the unnecessary expenditure of human and fiscal resources.

Operating at the sharp end

As I participated in the discussion at the National Institute of Justice roundtable in 2013, I could appreciate the potential of this type 

of learning-from-error review for officers like me who operate, as Jim Doyle puts it, “at the sharp-end of the stick.” Not only do 

police officers want to get the right person, we want to do so at the earliest possible moment before the perpetrator can claim further 

victims. But, because fallible human beings are involved in every step of the criminal justice process, error is an unavoidable reality. 

This is why I think it could be very helpful to examine errors or bad outcomes from the perspective of law enforcement’s role in the 

entire criminal justice system. By collaborating with and seeking the input of other justice stakeholders, law enforcement investigators 

— especially, perhaps, cold case investigators — may be able to learn from costly mistakes and work to improve processes and 

decision-making procedures that could help avoid similar mistakes in the future.
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 45

About The Writer

Frank P. Tona has been employed with the Charles County, Maryland, Sheriff’s Office since 2002. Currently 

a Corporal with the Patrol Division, Cpl. Tona served for seven years as a Detective. He is member of the 

department’s Cold Case Review Team, Vice President of the department’s labor union, and a member of 

the Hostage Negotiations Team. Cpl. Tona holds an undergraduate degree in criminal justice from SUNYBrockport and a graduate degree from Norwich University in public administration. 
National Institute of Justice | NIJ.gov

46 Mending Justice

Building a Learning-From-Error Culture in Policing

By John R. Firman

The International Association of Chiefs of Police (IACP) agrees with the proposition that underlies the National Institute of Justice’s 

(NIJ) Sentinel Events Initiative: errors do occur in the criminal justice system — as in any complex system or profession — and 

minimizing future errors demands a willingness to learn.

All too often, when new information comes forward about an error in our criminal justice system, we spend our time defending 

positions that are clearly untenable, rather than allowing lessons to guide improved practices. From the IACP’s perspective, systemic 

reform initiatives within the justice system — particularly ones driven by all-stakeholder reviews of sentinel events — are key.

Indeed, a professional organization like the IACP has a duty to help implement system reforms using sentinel events as markers 

for change. In that regard, the IACP is engaged in a number of projects that illustrate our organization’s commitment to innovative 

learning-from-error efforts. For example, the IACP is:

■■ Collaborating with the MacArthur Foundation and the Office of Juvenile Justice and Delinquency Prevention on ways to divert 

adolescents from long-term involvement in the criminal justice system.

■■ Working with the Office on Violence Against Women on learning from the “sentinel event” of uninformed or inappropriate response 

by the police to domestic violence, sexual assault, stalking and human trafficking crimes, which leaves victims even more 

vulnerable to further harm. 

■■ Collaborating with the Public Welfare Foundation and the Pretrial Justice Institute to prevent the release from prison of dangerous 

individuals who then commit new crimes and the equally problematic detention of individuals who pose no continuing risk to the 

community. 

■■ Working with the Department of Justice and the Innocence Project on ways to improve front-end police investigations and 

prosecutorial charging decisions to minimize focusing on the wrong suspect, which leaves the real offender free to commit new 

crimes.

Through work like this, the IACP is committed to bringing in the voices of nonpolice stakeholders — such as victims, juvenile 

corrections, prosecutors and private foundations — which, of course, is a key element of a sentinel event, learning-from-error review 

process. That said, we recognize that what NIJ’s Sentinel Events Initiative is exploring is even more ambitious in that it involves getting 

the buy-in of all nonpolice stakeholders by also including, for example, the defense bar, city risk managers, crime lab directors, and 

even political leaders or the press, where appropriate.

I believe, however, that the IACP’s ongoing work indicates that law enforcement is among the most forward-leaning of the 

stakeholders in the criminal justice system at this moment in time — a time where the understanding that human beings will inevitably 

err demands that we work toward a more systemic way to incorporate a learning-from-error process into our culture. 

Professional law enforcement organizations like the IACP — the largest body of policing professionals in the world — can play a 

crucial role. Scientific research has found that major innovations that have revolutionized police practices — community policing, 

Compstat, and problem-oriented policing, to name only three — are diffused and take hold through “peer networks.” The IACP is one 

such professional peer network with access to the best and brightest police leaders (within management and the rank and file) that 

can assist in the diffusion of NIJ’s innovative Sentinel Events Initiative.

The IACP strongly believes that sea changes in criminal justice practices must always begin with an openness to new information 

and a willingness to consider that new information with great care; indeed, this concept is what police legitimacy is all about, and it 

also lies at the core of a sentinel event review process. This is not to suggest that the work will be easy: law enforcement agencies 

have union issues, disciplinary procedures and other challenges that will have to be met head-on. But we can tackle these and other 

challenges. The IACP looks forward to helping explore the viability of this innovative learning-from-error process as a way to improve 

the reliability of the nation’s criminal justice system. 
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 47

About The Writer

John R. Firman is the director of the Research Division of the International Association of Chiefs of Police, 

the world’s oldest and largest police leadership organization. He is in charge of the development and 

implementation of a national and international policy and research and evaluation program. Mr. Firman 

helped create and currently manages the National Law Enforcement Policy Summit Series, which addresses 

current and emerging issues in the policing profession. Prior to joining the IACP in 1994, he served as 

Associate Director of the Illinois Criminal Justice Authority. He is an adjunct lecturer for the Department of 

Law, Justice and Criminology at American University’s School of Public Affairs.
National Institute of Justice | NIJ.gov

48 Mending Justice

Punishment-Based vs. Education-Based Discipline: A Surmountable Challenge? 

By Sean Smoot

In many police departments, cases of notoriety are commonly referred to as “Heater” cases. Because of the nature of the crime, 

who the victim is or who is accused of the crime, Heater cases — serial murders, serial rapes, crimes where the victim is a child, 

crimes involving celebrities or public officials — garner strong interest by the city government’s leaders, department leaders, press 

and the community. They get extra attention and scrutiny, and they remain in the spotlight unless or until they are solved. This puts 

extraordinary pressure on line officers and detectives.

Part of the sentinel event review approach that the National Institute of Justice is exploring is to get people to come out of their lanes, 

out of their stovepipes, out of a “that’s not my job” mentality and to assume greater system responsibility collectively. Even in routine 

policing cases, there is often fear of being singled out with blame landing on some poor schmuck, and everyone reverts to an “I’m just 

doing my job” mentality. Overcoming this will be a major challenge in making the sentinel events approach successful. In high-profile 

Heater cases, the fear of being singled out or blamed can become even more intense, making line officers even more reluctant to 

report operational errors or to disclose mistakes in a proactive or preventive way.

There is always the opportunity for post hoc review. But the goal — or one of the goals — of the sentinel events approach seems 

to be proactive, preemptive intervention: stopping the line, basically. But there currently is no clear mechanism for doing this. Heater 

cases, where the pressure is really on to make an arrest and get the case resolved, offer just one example of where some challenges 

lie in adopting a sentinel event review process.

Another challenge is defining what constitutes a sentinel event in the criminal justice system. Some would propose a definition that 

includes wrongful/erroneous arrest, wrongful/erroneous conviction, use of deadly force, pursuits and in-custody deaths/injuries. 

However, from the standpoint of the police officer, who is the “tip of the spear” in most, if not all, of these events, the prospect of 

adding yet another layer of conduct review is — regardless of its laudable goal — likely to receive a very negative reaction.

To understand this perspective, one needs to understand the jeopardy that officers feel each and every time they are involved in a 

significant event in the course of performing their duties. Most police departments have adopted the practice of “punishment-based” 

discipline, which is interested in placing blame and punishing for policy or procedure violations. This practice is not really compatible 

with the sentinel event philosophy, which is interested in identifying problems and preventing repetition of outcomes through policy or 

procedure modification and education.

For instance, when an officer is involved in the use of deadly force, he or she currently faces at least two, usually three and sometimes 

even more adversarial investigations and reviews:

Criminal: Typically, an officer’s conduct is reviewed rapidly by his or her own department’s homicide investigators (or 

those of a different agency) and then put before a state prosecutor for review. In many jurisdictions, state prosecutors have 

adopted a screening procedure that includes putting the case before a grand jury (even if the case is a clear-cut justified use 

of force). 

Administrative/internal affairs: The department’s own internal affairs/administrative review often occurs at the same 

time as the criminal review. Usually, the officer is interrogated by a supervisor after being ordered to answer questions under 

threat of job forfeiture. These investigations are usually adversarial in nature, as they are designed to serve as a basis for 

departmental discipline.

Civilian: Some departments have adopted a civilian review mechanism. Often referred to as civilian review boards (CRBs), 

these entities play various roles and have varying powers depending on the jurisdiction. Some CRBs review the department’s 

investigation and may make inquiries, but have no power to recommend discipline, or to approve or disapprove of investigation 

conclusions; other CRBs have subpoena powers and full authority to recommend charges or even to mete out discipline.

Civil litigation: Almost finally, the officer’s conduct is then reviewed by his or her employer’s (or his or her employer’s 

insurance carrier’s) civil defense counsel. Of course, many, if not most, of these situations wind up in civil litigation against 

the officer, his or her employer or both. 

Notice I say “almost finally” because the Department of Justice can also decide to conduct a review of the officer’s conduct, which 

could include a criminal investigation or review under the United States Code. And, of course, the officer’s conduct will also be 
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 49

scrutinized by the public and the news media, who will eventually, at the least, get a significant portion of all of the 

aforementioned investigations and, in some jurisdictions, may legally obtain the complete investigatory files.

In light of the foregoing, it should not be difficult to understand why police officers would be wary of participating in an 

additional investigation/review, such as that envisioned in a sentinel event review process.

That is not to say that officers lack interest in learning from mistakes or errors. No one wants to repeat bad outcomes 

over and over again. However, officers don’t see investigation and review of significant occurrences through an 

unmodified lens. Officers approach review processes cautiously, if not hesitantly, out of an ingrained sense of 

skepticism. Even though an honest review might, and should, prevent repeating avoidable negative outcomes, they 

often fear that the process will not be an honest one.

To help prevent police from viewing a sentinel event review as “another opportunity to fire me,” “another opportunity 

to prosecute me,” or “another opportunity to sue me,” it must not be regarded as another criminal, internal affairs or 

civil review. For sentinel event reviews to be effective and practical, they must be a cooperative effort that affords the 

types of protections provided in the medical context, where state and federal laws protect the privacy of participants 

and prevent the disclosure of information to anyone outside of the sentinel event review. Accordingly, the medical 

professionals who participate can do so without threat of prosecution or discipline and without the fear that their 

statements will later be disclosed to the press or a plaintiff’s attorney.

Unless the sentinel event process is honest and trustworthy, with adequate legal protections — including use 

immunity, privacy, confidentiality and nondisclosure, for example — police officers, who have the very best information 

about how things really work and what really happened, will not be motivated to fully participate. The sentinel event 

review approach will have a better chance of success if departments can abandon the process of adversarial/punitivebased discipline, adopting instead “education-based” disciplinary procedures and policies. 

About The Writer

Sean Smoot is the Director and Chief Counsel for the Police Benevolent & Protective Association of Illinois 

and the Police Benevolent Labor Committee, where he is responsible for providing legal services for more 

than 7,500 legal defense plan participants. He regularly represents police officers in discipline and discharge 

and civil rights cases. Since 1996, Mr. Smoot has served on the Advisory Committee for the National Law 

Enforcement Officers’ Rights Center. He was a police and public safety policy advisor to the Obama-Biden 

Presidential Transition Team. Mr. Smoot received a B.S. from Illinois State University and a J.D. from the 

Southern Illinois University School of Law.
National Institute of Justice | NIJ.gov

50 Mending Justice

Reducing Failure: A View of Policing Through an Organizational Accident Lens

By Jon Shane

It is well settled that all complex businesses — including medicine, aeronautics and transportation, petroleum, petrochemical and 

nuclear production — are at risk for an organizational accident.1 The question, then, is whether policing is also a complex business. 

In fact, research over the past seven decades has shown that it is — and never have those complexities been so eloquently stated as 

here:

“...one may well wonder how any group of men could perform the tasks required of policemen. The citizen expects police 

officers to have the wisdom of Solomon, the courage of David, the strength of Samson, the patience of Job, the leadership 

of Moses, the kindness of the Good Samaritan, the faith of Daniel, the tolerance of the Carpenter of Nazareth, and, finally, an 

intimate knowledge of every branch of the natural, biological, and social sciences. If he had all of these, he might be a good 

policeman.”

– August Vollmer, The Police in Modern Society 2

To add to this complexity, it is important to note that policing is itself an element of a larger and even more complex criminal justice 

system. In fact, this is the foundation on which a sentinel event review process rests. Although I explore here only one “stovepipe” in 

the system — the police — it is crucial to keep in mind that an all-stakeholder, nonblaming review of an error necessarily involves all 

the other criminal justice players.

The issue for the police — both in their own complex environment and as a part of a complex system — is to balance the competing 

demands of policy and practice; management and line function; intent and execution. This requires a framework that moves beyond 

individual blame and accounts for the context in which the error occurred. An individual accident results from the acts of people 

following properly established procedures. This view examines the active failure from the individual operator’s perspective and does 

not account for the contextual and precipitating factors impinging on the operator, nor does it reach the middle and upper ranks that 

are responsible for policy and supervision. It is accusatory and resides in a culture of blame, which reduces officers to secrecy and 

silence. Little is accomplished beyond affixing blame. 

When the accident is viewed through an organizational lens, however, the context reveals that the individual inherited rather than 

instigated the accident, such as when acts or omissions result from insufficient or absent policy; weak supervision; workaround 

solutions and accepted past practices that are informally adopted by employees and tolerated by management; failing to learn and 

train from prior events and precursors; or budget reductions that compromise safety. This approach — which considers the error as 

the starting (rather than the ending) point for investigation — seeks to isolate and correct the causes to ensure they are minimized or 

eliminated in the future. 

A sentinel event review process, however, considers that the individual is situated inside a larger bureaucracy that likely ensnared him 

or her through latent policy weaknesses, deviant cultural practices and poor supervision (organizational accident method). Pursuing an 

organizational accident framework focused on support, accountability3

 and professional development does not stop at the singular or 

obvious error. It always attempts to understand the behaviors, conditions and contributing circumstances behind the act or omission 

that supported the error. Only once these associated phenomena are understood in their context can future errors be avoided through 

policy development and training.4

The policing field is fraught with production hazards (e.g., arrests, traffic stops, custodial interrogations, identification procedures, 

search and seizure activities, vehicular pursuits, using force), largely because the tactical, political, social and legal environments are 

intertwined, competing and complex. One case study of a police field identification procedure (show-up) — what might be described 

as “routine surgery” in the medical context — revealed 49 errors during the investigation.1 Therefore, policing is well poised to accept 

the lead in adopting a systems approach to organizational accidents since police behavior during the initial stages of an investigation 

usually has downstream implications for the other actors in the system — prosecutors, defense attorneys and judges. Although 

the police should take a leadership role in adopting the organizational accident framework, what may at first glance seem to be an 

isolated upstream (police) error is influenced by the reciprocal downstream actions, expectations and informalities of interpersonal 

workgroup relations5

 that ostensibly offer plausible deniability for others in the system who are indisputably connected to the police. 

The downstream operators are not blameless simply because they are distant from the upstream active failure in time and space.6

One example of policing that has implications for the larger criminal justice system is wrongful conviction based on using confidential 

informants (CIs). Often seen as indispensable to police work, the CI is an accident waiting to happen if not managed properly through 
COMMENTARIES

National Institute of Justice | NIJ.gov

Mending Justice 51

policy, supervision and training. The Center for Wrongful Conviction at Northwestern University Law School cites 

informant testimony as the leading cause of wrongful convictions in capital cases.7 Previous research involving CIs 

suggests they may not be properly managed by the police,8 and preliminary findings from new research that examines 

published police policies on CIs show that those policies have latent weaknesses that relate to testing a CI’s integrity, 

supervision and training before deploying the CI9 — the very types of weaknesses that contribute to operator error. If 

the police deploy a CI without testing his or her integrity, supervising the CI and providing training for the officer and 

the CI, the chemistry for a profound accident exists. 

Death, injury, false arrest, wrongful conviction and other police errors should be valued opportunities for full disclosure 

and professional development, rather than viewed as acceptable risk or collateral damage. Police practices are on the 

precipice of legitimacy yet again, particularly given the worldwide rise in egalitarianism and leveling of hierarchical 

institutions.10 There is an opportunity to adopt a standard review methodology and embrace each error as a learning 

experience that improves transparency, strengthens community support and reaffirms the police commitment to 

professionalism.11 Or, police leaders can retreat — as medicine and aviation leaders did in the early stages of their 

learning-from-error reform movements — into a defensive posture that keeps errors and their learning value 

hopelessly locked away from progressive police managers eager to improve practices.

Notes 

1. Shane, J. (2013). Learning from Error in Policing: A Case Study in Organizational Accident Theory. New York: Springer.

2. Cited in Johnson, E. (1967). “The Sociological Interpretation of Police Reaction and Responsibility to Civil 

Disobedience,” The Journal of Criminal Law, Criminology, and Police Science 58(3):405.

3. See Waring, S. (September 2013). “An Examination of the Impact of Accountability and Blame Culture on Police 

Judgments and Decisions in Critical Incident Contexts,” Ph.D. dissertation. University of Liverpool, England.

4. Catino, M. (2008). “A Review of Literature: Individual Blame vs. Organizational Function Logics in Accident Analysis,” 

Journal of Contingencies and Crisis Management 16(1):53-62.

5. Guzzo, R.A., and Shea, G.P. (1990). “Group Performance and Intergroup Relations in Organizations,” in M.D. 

Dunnette and L.M. Hough (eds.), Handbook of Industrial and Organizational Psychology, Vol. 3, Chapter 5 (pp. 269-313), 

Palo Alto, California: Consulting Psychologies Press.

6. Failure in an organization generally occurs “...when some operation, employee, policy or process produces results 

that deviate from expectations in substantial and disruptive ways. Failure encompasses accident, non-performance, 

corrupt performance and deviant behavior.” (see Reason, J. (1998), “Achieving a Safe Culture,” Work Stress 12: 

293-306).

About The Writer

Jon M. Shane is an Associate Professor in the Department of Law, Police Science and Criminal Justice 

Administration at the John Jay College of Criminal Justice, where he focuses on police policy and practice, 

including situational crime prevention. Dr. Shane retired as a captain from the Newark Police Department 

after 20 years, where his service included many operational and administrative assignments. He has worked 

with law enforcement agencies across the country and internationally, developing policy and conducting 

performance audits and research and management studies to measure performance. Published in a number 

of professional journals, Dr. Shane currently also serves as a senior research associate for the Police 

Foundation and as a subject matter expert for the Center for Problem Oriented Policing.
National Institute of Justice | NIJ.gov

52 Mending Justice

7.Center for Wrongful Convictions. (2005). “The Snitch System,” Chicago, Illinois: Northwestern University Law School, Center for 

Wrongful Convictions, p. 3.

8. Jones-Brown, D., and Shane, J. M. (June 2011). “An Exploratory Study of the Use of Confidential Informants in New Jersey. A 

Report of the American Civil Liberties Union of New Jersey,” Newark, New Jersey: American Civil Liberties Union of New Jersey; 

Natapoff, A. (2006). “Beyond Unreliable: How Snitches Contribute to Wrongful Convictions,” Golden Gate University Law Review 37(1): 

107-129; Schreiber, A.J. (2001). “Dealing With the Devil: An Examination of the FBI’s Troubled Relationship With Its Confidential 

Informants,” Columbia Journal of Law and Social Problems 34: 301-368; Zimmerman, C.S. (1994). “Toward a New Vision of 

Informants: A History of Abuses and Suggestions for Reform, “Hastings Constitutional Law Quarterly 22: 81-178. 

9. Shane, J.M., and Jones-Brown, D. (forthcoming). “Confidential Informants and Integrity Testing: A Closer Look at Police Policy.”

10. Orren, G. (1997). “Fall From Grace: The Public’s Loss of Faith in the Government,” in Nye, J.S., Zelikow, P.D. and King, D.C. (eds.), 

Why People Don’t Trust Government, Cambridge, Massachusetts: Harvard University Press. 

11. Stone, C., and Travis, J. (March 2011). Toward a New Professionalism in Policing. Washington, D.C: U.S. Department of Justice, 

National Institute of Justice. NCJ 232359.
Mending Justice 53

National Institute of Justice | NIJ.gov

Appendix 

As part of its preliminary investigation into the feasibility 

of using a sentinel event review approach in the criminal 

justice system, the National Institute of Justice (NIJ) and 

the Office of Justice Programs looked at other learningfrom-error efforts that share certain significant features 

with a sentinel events approach to improving justice 

outcomes, including: 

■■ Do “all stakeholders” participate?

■■ Is there an emphasis on “nonblaming”?

■■ Is the approach routine and ongoing?

■■ Are the findings publicly disseminated?

■■ Is there an emphasis on being “forward-looking” or 

on future prevention?

We found a number of learning-from-error efforts that 

incorporated some of these features; brief descriptions 

are provided below. These examples — although not 

intended to be exhaustive or comprehensive — may 

provide readers with additional context for understanding 

the distinctive approach of NIJ’s Sentinel Events Initiative. 

Cambridge Review Committee on Arrest 

of Professor Henry Louis Gates, Jr.

The final report, Missed Opportunities, Shared 

Responsibilities: Final Report of The Cambridge Review 

Committee, is available at www.cambridgema.gov/

CityOfCambridge_Content/documents/Cambridge%20

Review_FINAL.pdf

The Cambridge Review Committee included a diverse 

group of stakeholders. 

With respect to a “nonblaming” element, the Committee 

stated that it “was not charged with writing an ‘afteraction’ or fact-finding report, or with assigning blame 

either to Sergeant Crowley or to Professor Gates ... 

Rather, the Committee was charged with identifying the 

lessons that can be learned from the incident and the 

implications of those lessons for the policies, procedures, 

and mission of the Cambridge Police Department and the 

city of Cambridge as well as other police departments 

and cities across the nation.”

With respect to the “routine/ongoing” element, this was 

not an ongoing, routine process but rather a singular 

review of one incident.

Child Fatality Review Teams

Child death review teams — also known as child fatality 

review teams — review child abuse, negligent fatalities 

and suspicious child deaths. They exist in most states and 

seem to have nonblaming and forward-looking principles 

at their core: 

■■ “Results of these reviews may be used to improve 

services, advocate for change, and conduct public 

awareness activities, ultimately for the purpose of 

preventing future child maltreatment deaths.” See 

the Child Death Review Teams section of the U.S. 

Department of Health and Human Services: https://

www.childwelfare.gov/responding/review_teams.cfm

■■ “Reviews focus on what went wrong and how can we 

fix it, not who is at fault and who should we blame.” 

See Examining Child Fatality Reviews and CrossSystem Fatality Reviews to Promote the Safety of 

Children and Youth at Risk, by Y. Yuan, T. Convinton 

and L. Oppenheim: http://www.childdeathreview.org/

Promo/WRMA_August2012.pdf

■■ “The purpose of fatality reviews: To conduct a 

comprehensive, multidisciplinary review of child 

deaths, to better understand how and why children 

die, and use the findings to take action that can 

prevent other deaths and improve the health and 

safety of children.” http://www.childdeathreview.org/

cdrprocess.htm

Domestic Violence Fatality Reviews

“The mission of the National Domestic Violence Fatality 

Review Initiative (NDVFRI) is to provide technical 

assistance for the reviewing of domestic violence related 
54 Mending Justice

National Institute of Justice | NIJ.gov

deaths with the underlying objectives of preventing 

them in the future, preserving the safety of battered 

women, and holding accountable both the perpetrators 

of domestic violence and the multiple agencies and 

organizations that come into contact with the parties.” 

http://www.ndvfri.org

From “Domestic Violence Fatality Reviews: From a 

Culture of Blame to a Culture of Safety,” by N. Websdale, 

Michael Town and Byron Johnson (1999), Juvenile and 

Family Court Journal, Vol. 50, No. 2 (Spring): 

“Fortunately, there are workable models in 

the fields of medicine and aviation upon which 

to draw. These models teach courts and 

communities that, with vigor, honesty, and 

candor, they can build reliable systems that 

value accountability and help prevent future 

death and injury from domestic violence. 

Because domestic violence deaths exhibit 

predictable patterns and etiologies, they are 

preventable. We argue that the establishment 

of domestic violence fatality review teams is 

one effective way of reducing domestic violence 

homicides. After briefly outlining the scope and 

extent of domestic violence related deaths, 

this article discusses the history of domestic 

violence fatality reviews and presents several 

models that appear to be both effective and fair.

“Traditionally, these tragedies [domestic 

violence fatalities] have resulted in finger 

pointing, anger, fear, frustration, and distrust. 

Sometimes, this finger pointing has found 

voice in the form of editorials, lawsuits, and 

legislative hearings. These forms of finger 

pointing, sometimes referred to as ‘tombstone 

technology’ in fields such as aviation and 

nuclear power, have not been productive. 

They can result in accusations of stonewalling 

and cover-ups. Consequently, many 

community members, including judges, court 

administrators, elected officials, prosecutors, 

law enforcement officials, and battered 

women’s advocates are looking for workable 

and fair models to review domestic violence 

fatalities, with a view to preventing future 

deaths. This search is not for the fainthearted 

since it requires a paradigm shift from a culture 

of blame to a culture of safety.”

Milwaukee Homicide Review Commission

“The Milwaukee Homicide Review Commission (MHRC) 

strives to reduce homicides and non-fatal shootings 

through a multi-level, multi-disciplinary and multi-agency 

homicide review process. The MHRC is comprised of law 

enforcement professionals, criminal justice professionals 

and community service providers who meet regularly to 

exchange information regarding the city’s homicides and 

other violent crimes to identify methods of prevention 

from both public health and criminal justice perspectives. 

The MHRC makes recommendations based on trends 

identified through the case review process. These 

recommendations range from micro-level strategies 

and tactics to macro-level police change.” http://city.

milwaukee.gov/hrc

“Partners represent key stakeholders from multiple 

levels (city, regional, county, and state), disciplines, 

and agencies (governmental and private, including 

community service providers). At each homicide review 

meeting, partners participate in an intensive discussion 

and examination of individual homicide and intentional 

crime incidents. Through this process, trends, gaps, and 

deficits within the already existing systems and programs 

designed to prevent and reduce violence are identified 

and recommendations are made to strengthen these 

systems and programs.” http://city.milwaukee.gov/hrc/

overview 

New York State Justice Task Force

“The Task Force includes representatives from all 

participants in the criminal justice system — judges, 

prosecutors, defense attorneys, members of law 

enforcement, legislators, executive branch officials, 

forensic experts, victim’s advocates and legal scholars — 

from across the State.” http://www.nyjusticetaskforce.com

“The Justice Task Force was formed with the belief that, 

while these cases of wrongful convictions are tragic, we 

can learn a valuable lesson from each of them. By closely 

examining new exonerations in New York to determine 
Mending Justice 55

National Institute of Justice | NIJ.gov

how the criminal justice system failed, the Justice Task 

Force hopes to identify any recurring patterns and 

practices that may be contributing to wrongful convictions 

in this state.” http://www.nyjusticetaskforce.com/

mission.html

Elder Fatality Review Replication Manual

This was a project of the Office for Victims of Crime, the 

National Adult Protective Services Association, and the 

American Bar Association Commission on Law and Aging. 

It promotes fatality review teams. 

From Elder Abuse Fatality Review Teams: A Replication 

Manual, by L. Stiegel, http://apps.americanbar.org/aging/

publications/docs/fatalitymanual.pdf:

■■ “Hospital physicians use the ‘Morbidity and Mortality 

Review’ process to examine what went wrong 

with a medical procedure and determine how the 

same problem could be avoided in the future. Car 

manufacturers examine accident-related deaths 

or injuries to learn how to design and build a safer 

car. Service providers in the child abuse and, more 

recently, domestic violence fields analyze deaths 

that were caused by abuse or deaths of persons who 

were known victims of abuse previously in order to 

change the system’s response to victims and avoid 

similar outcomes.” 

■■ “Other issues do not arise immediately, but they 

must be addressed before a team can start 

reviewing cases. These issues include creating a 

culture of avoiding ‘blame and shame’; preparing 

policies and procedures, protocols, or memoranda 

of understanding; deciding what to call the team; 

and, most importantly, ensuring that necessary 

confidential information can be shared and 

obtained and that confidential information and 

team deliberations and products are protected from 

voluntary or involuntary disclosure outside of the 

team.”

See also Neil Websdale, Michael Town and Byron 

Johnson, “Domestic Violence Fatality Reviews: From a 

Culture of Blame to a Culture of Safety,” Juvenile and 

Family Court Journal, Vol. 50, No. 2 (Spring 1999).

Wisconsin Criminal Justice Study Commission

This commission started with discussions between the 

Criminal Law Section of the Wisconsin State Bar and the 

University of Wisconsin Law School and the Marquette 

Law School about studying the errors that had produced 

wrongful convictions; those discussions were followed by 

a national conference hosted by the American Judicature 

Society. (See Keith Findley, “Learning from Our Mistakes: 

A Criminal Justice Commission to Study Wrongful 

Convictions,” California Western Law Review 38 (2) (July 

2005).

With respect to the “nonblaming” element, the Wisconsin 

Criminal Justice Study Commission’s charter states: 

“The goal of our commission will not be to point fingers 

or assign blame for past mistakes, as some might 

understandably fear. And while the wrongful convictions 

of Steven Avery and others are a major stimulus for 

the commission, the commission’s role will not be to 

identify specific cases of wrongful conviction. Rather, the 

overriding purpose of the commission will be to produce 

the best possible criminal justice system, one that justly 

convicts the guilty and not the innocent.” http://law.wisc.

edu/fjr/clinicals/ip/wcjsc/files/charter.pdf

Trial and Error, Center for Court Innovation

“The Center for Court Innovation, with the support of 

the U.S. Department of Justice’s Bureau of Justice 

Assistance, has embarked on a multi-faceted inquiry 

designed to promote trial and error in criminal justice 

reform. The Center is examining efforts to improve the 

criminal justice system that did not achieve the results 

that were intended in an attempt to learn lessons and 

promote innovation going forward. At its heart, this is an 

effort to encourage honest self-reflection and thoughtful 

risk-taking among criminal justice agencies.” http://www.

courtinnovation.org/topic/trial-and-error

In 2010, the Urban Institute published Trial & Error 

in Criminal Justice Reform: Learning from Failure, 

http://www.courtinnovation.org/research/trial-anderror-criminal-justice-reform-learning-failure

Daring to Fail: First-Person Stories of Criminal Justice 

Reform, a collection of interviews with leading criminal 

justice thinkers about failure, was published in 2011. 

http://www.courtinnovation.org/research/daring-fail
56 Mending Justice

National Institute of Justice | NIJ.gov

North Carolina Innocence Inquiry Commission

With respect to the “all-stakeholders” element, the 

North Carolina Innocence Inquiry Commission includes a 

Superior Court judge, a prosecuting attorney, a defense 

attorney, a victim advocate, a member of the public, a 

sheriff and others. http://www.innocencecommission-nc.

gov/index.html. For a synopsis of innocence commissions 

in North Carolina and 11 other states, see M. Tate, 

“Commissioning Innocence and Restoring Confidence: 

The North Carolina Innocence Inquiry Commission and the 

Missing Deliberative Citizen,” Maine Law Review 64 (2) 

(2012). http://mainelaw.maine.edu/academics/maine-lawreview/pdf/vol64_2/vol64_me_l_rev_531.pdf

Allegheny County Comprehensive 

Case Reviews 

With respect to the “all-stakeholders” element: “Virtually 

everyone who had been involved with these defendants 

had been invited to attend and share observations: 

judges, prosecutors, defense counsel, human service 

providers, probation, police, and staff from the courts and 

the jail.” 

With respect to the “nonblaming” element, in a discussion 

of lessons learned: “Emphasize that no one will be 

criticized. Many participants, when told that they were 

invited to a case review meeting with the presiding judge, 

figured they must have done something seriously wrong. 

Thus it was crucial to keep the discussions focused on 

how the system functioned, not on any particular person’s 

performance.”

With respect to the “ongoing/routine” element: “Allegheny 

County will plan to resume its case reviews in October 

and to continue holding them periodically until the issues 

generated are not sufficient to justify the preparation 

time. If that point is reached, it will be strong evidence 

that the county’s criminal justice system has changed for 

the better.” 

http://www.alleghenycounty.us/WorkArea/

DownloadAsset.aspx?id=35204

Other projects that may contain key elements 

of a sentinel event review approach:

■■ Eyewitness Identification Task Force: Report to the 

Judiciary Committee of the Connecticut General 

Assembly.

http://www.cga.ct.gov/jud/eyewitness/docs/

Final%20Report.pdf

■■ Florida Domestic Violence Fatality Review Team.

http://www.fcadv.org/department-children-andfamilies-and-florida-coalition-against-domesticviolence-create-statewide-do

■■ Comprehensive Operational Assessment, Criminal 

Investigative Unit, Sheriff, Will County, Illinois. 

http://www.scribd.com/fullscreen/

47496706?access_key=key-d7dvwhg1k4m4ipsr1oi

This report was commissioned by the Sheriff’s Office 

of Will County, Illinois, and prepared by a private 

consulting company. Although it reviews a case (the 

kidnapping, sexual abuse and murder of Riley Scott) 

from the perspective of only a single “silo” (policing), 

rather than all criminal justice stakeholders, it 

represents a forward-looking, learning-from-error 

review of the wrongful arrest of Riley’s father. 

■■ Hennepin County Blind Sequential Lineup Project.

This eyewitness identification project was supported 

by Grant No. 2004-IJ-CX-0044 awarded by the 

National Institute of Justice. 

https://www.ncjrs.gov/pdffiles1/nij/grants/246939.

pdf

■■ Philadelphia Women’s Death Review Team. 

http://www.phila.gov/health/pdfs/2004_2006_

PWDRT_final.pdf
Mending Justice 57

National Institute of Justice | NIJ.gov

■■ Report on the Conviction of Jeffrey Deskovic. 

http://www.westchesterda.net/Jeffrey%20

Deskovic%20Comm%20Rpt.pdf 

■■ Task Force on Eyewitness Evidence, Suffolk County, 

Massachusetts.

http://www.innocenceproject.org/docs/Suffolk_

eyewitness.pdf

■■ Vera Institute Prosecution and Racial Justice 

Program.

http://www.vera.org/centers/

prosecution-and-racial-justice-program

■■ District of Columbia Superior Court Ad Hoc 

Committee on Wrongful Convictions.

http://www.dcappeals.gov/internet/

documents/Ad-Hoc-Committee-Findings-andRecommendations_2-12-13CORRECTED.pdf

■■ National Firefighter Near Miss Reporting System.

http://www.firefighternearmiss.com/index.php/home

■■ Praxis Safety and Accountability Audit.

www.praxisinternational.org

■■ Organizational Learning in Policing.

http://www.policefoundation.org/content/

organizational-learning-policing



CRIME LAB

COMMUNITY

POLICE

VICTIMS

JUDICIARY

POLICYMAKERS

DEFENSE

RESEARCHERS

CORRECTIONS

PROSECUTION
*NCJ~247141* U.S. Department of Justice

Office of Justice Programs

National Institute of Justice

Washington, DC 20531

Official Business

Penalty for Private Use $300

PRESORTED STANDARD

POSTAGE & FEES PAID

DOJ/NIJ

PERMIT NO. G –91