U.S. Department of Justice

Office of Justice Programs

National Institute of Justice

NATIONAL INSTITUTE OF JUSTICE

NOVEMBER 2015

PAVING THE WAY: 

Lessons Learned in 

Sentinel Event Reviews 
U.S. Department of Justice 

Office of Justice Programs 

810 Seventh St. N.W. 

Washington, DC 20531 

Loretta E. Lynch 

Attorney General 

Karol V. Mason 

Assistant Attorney General 

Nancy Rodriguez, Ph.D. 

Director, National Institute of Justice 

This and other publications and products of the National Institute of Justice 

can be found at: 

National Institute of Justice 

Strengthen Science • Advance Justice 

http://www.nij.gov 

Office of Justice Programs 

Innovation • Partnerships • Safer Neighborhoods 

http://www.ojp.usdoj.gov 

The National Institute of Justice is the research, development and evaluation agency 

of the U.S. Department of Justice. NIJ’s mission is to advance scientific research, 

development and evaluation to enhance the administration of justice and public safety. 

NIJ is a component of the Office of Justice Programs, which also includes the Bureau 

of Justice Assistance; the Bureau of Justice Statistics; the Office for Victims of Crime; 

the Office of Juvenile Justice and Delinquency Prevention; and the Office of Sex 

Offender Sentencing, Monitoring, Apprehending, Registering, and Tracking (SMART). 

Opinions or points of view expressed in this document are those of the authors and do 

not reflect the official position of the U.S. Department of Justice. 
NATIONAL INSTITUTE OF JUSTICE

NOVEMBER 2015
 


PAVING THE WAY: 

Lessons Learned in 

Sentinel Event Reviews 

SENTINEL EVENTS INITIATIVE TEAM 

Katharine Browning 

Thomas Feucht 

Nancy Ritter 

Katherine Darke Schmitt 

Maureen McGough 

Scott Hertzberg 

NCJ 249097 

PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 1 

PAVING THE WAY:
 


LESSONS LEARNED IN 

SENTINEL EVENT REVIEWS 

When bad things happen in a complex 

system, the cause is rarely a single act, 

event or slip-up. More often, bad outcomes 

are “sentinel events.” 

A sentinel event is a negative outcome that:

 • Signals underlying weaknesses in a 

system or process.

 • Is likely the result of compound errors.

 • May, if properly understood, provide 

important keys to strengthening the 

system and preventing similar adverse 

outcomes in the future. 

Sentinel event reviews (SER) were 

initially developed as a process for better 

understanding the causes of industrial 

accidents. They were later adapted in 

aviation and in medical environments. The 

SER process brings all stakeholders to the 

table on a regular basis to discuss, in a 

nonblaming way, why a negative outcome or 

event happened. 

The overarching goal of SER is to mobilize 

a routine, culture-changing practice 

that can lead to increased system reliability 

and, hence, greater public confidence in a 

system’s legitimacy. 

T

his report reviews the lessons 

learned by three forward-leaning 

teams in their review of a negative 

criminal justice outcome (“sentinel 

event”) in their jurisdiction. The 

project represents the next step in 

NIJ’s exploration of the feasibility of 

using sentinel event reviews as a way 

to learn from errors in the criminal 

justice system. 

In 2014, NIJ published Mending 

Justice: Sentinel 

Event Reviews, 

which discusses 

the concept of 

sentinel event 

reviews in depth. 

U.S. Department of Justice 

Office of Justice Programs

National Institute of Justice SPECIAL REPORT

NATIONAL INSTITUTE OF JUSTICE 

MENDING JUSTICE: Sentinel Event Reviews 

SEPTEMBER 2014

National Institute of Justice | NIJ.gov 
 

 

2 PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 

Sentinel Event Reviews in the Criminal Justice System 

The National Institute of Justice (NIJ) — the scientific research arm of the 

Department of Justice — has, since 2011, been investigating the feasibility of using 

SER as a way to learn from errors in the criminal justice system. NIJ’s Sentinel Events 

Initiative borrows extensively from medicine, aviation and other high-risk fields where 

a blame-placing, backward-looking review of errors is yielding to a more forwardthinking, nonblaming, problem-solving approach. 

In criminal justice, a sentinel event might be a police shooting, the exoneration of a 

wrongly convicted person, the release from prison of a dangerous offender, or even a 

“near miss” that could have led to a bad outcome had it not been caught. 

Although most criminal justice agencies already have error-detecting procedures in 

place — police internal affairs reviews, for example, or prosecutors’ professional 

ethics boards — these often become “gotcha” processes that focus on assigning 

individual blame. This can drive the reporting of errors underground, making future 

errors even harder to detect and correct. 

NIJ’s Sentinel Events Initiative seeks to answer three empirical questions about using 

SER in the justice system:

 • Can it be done?

 • Does it provide a means to achieving desired outcomes, such as increased 

effectiveness or fewer errors or other public safety dividends, such as greater public 

perception of the integrity of the nation’s justice system? 

• Can it be sustained over time and incorporated into the routine activities of state and 

local justice processes? 

NIJ’s Sentinel Events Initiative Beta Project 

In 2014, NIJ asked jurisdictions from around the country to volunteer for an 

experiment — or beta test — of the SER process. Through a competitive process, 

we selected three cities: Milwaukee, Philadelphia and Baltimore. NIJ provided very 

minor logistical support, but no funding, to the beta sites. We commend the people 

who served on the beta SER teams for their courage, innovation and commitment 

to improving the nation’s criminal justice system. They are true pioneers. 

In exchange for their willingness to experiment with us, we promised the beta SER 

teams as much anonymity as possible, including details of the sentinel event they 

chose to review. Therefore, this summary of lessons learned in the three beta sites 

does not attribute statements to any individual or jurisdiction. 

National Institute of Justice | NIJ.gov 
PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 3 

Each of the three sites designed and conducted its own review of a justice error (a 

sentinel event) that had occurred in their jurisdiction. In their successful completion 

of an SER, the beta sites provided the first empirical evidence of the feasibility of 

adopting SER in the justice system — that is, an answer to NIJ’s first research 

question: Can it be done? The answer is “yes,” although each site emerged with 

important differences in how the SER team was structured and how they conducted 

their review. This demonstrated that, at least at this point, there is no single recipe for 

a successful SER in the criminal justice system. Therefore, we present this summary 

not as an implementation manual but, rather, as a guide to questions that jurisdictions 

should ask — and factors they should consider — in designing an SER. Subsequent 

research efforts will look at the second and third questions that NIJ is exploring 

regarding the sustainability of the SER process in criminal justice and the extent to 

which it achieves measurable public safety and system improvements. 

The lessons learned presented in this report were derived in two ways:

 • Interviews with individual team members at each site.

 • A two-day discussion with all three sites in January 2015. 

Where — and How — Do You Start? 

It could be helpful to determine whether your jurisdiction has any other review systems 

already in place, such as a conviction integrity unit or homicide review group. If it does, 

explore what you might learn from them about facilitating discourse among agencies. 

Consider how you’re going to get key leadership buy-in; for example, a high-ranking 

official may be generally supportive of an idea but not supportive enough to properly 

staff and dedicate resources to executing it. To succeed, SER must come to be regarded 

as a good use of people’s time. How will you gain the support of upper management in 

using the time and energy of personnel to participate on an SER team? 

How Do You Choose the ‘Right’ Event for SER? 

When it comes to picking an event to review, three overarching considerations 

emerged in the beta project:

 • Mitigation of legal risk.

 • “Age” of the event.

 • Broad system participation. 

Mitigation of legal risk. Are criminal and civil actions pending? Resolved? Is there 

any potential for appeals that would affect the SER process once it is started? This 

may be the single most important procedural question facing SER in the criminal 

National Institute of Justice | NIJ.gov 
4 PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 

justice system. It doesn’t mean an SER can’t be done; it only means that actions may 

be needed to mitigate these risks. 

“Age” of the event. The three beta sites chose noncurrent (older) events 

for their reviews. It should be noted, however, that some beta team members 

expressed the desire to review a more current event the next time, so that real-time 

recommendations could be developed and applied systemwide. Here are some pros 

and cons of selecting an older event: 

Selecting an Older Sentinel Event for Review 

Benefits Challenges 

Legal exposure is minimized if all civil and 

criminal legal proceedings have concluded. 

Older records may be lost or otherwise not available. 

Older records may not be automated; review of paper 

files is more time-consuming. 

Challenges of having to incorporate persons 

who were involved in the original event may be 

avoided if they are no longer part of the system. 

Persons involved in the event may not be available to 

participate. 

Luxury of having more, varied events to select 

from if recency is not a factor. 

The review may not have obvious and immediate 

beneficial outcomes if system changes have already 

been made in the interval since the event occurred. 

The perspective of hindsight regarding an older 

event/case may add some clarity to the review 

process. 

Personal recall of the event by persons involved may 

dim over time, decreasing their utility for the review 

process. 

The review dynamic may be easier and less 

tense if the event was “long ago and far away.” 

The older event may not have much current interest 

or traction among team members — or among 

policymakers or the public, who need to be engaged 

for change to take place. 

Broad system participation. Choose an event/case in which multiple agencies 

have a stake so that the SER can have the broadest potential impact on the system. 

Consider “high-frequency, low-impact” events, not just bigger, high-profile events. 

The beta sites were designed to conduct a single SER in order to test the fundamental 

question about feasibility: Can justice agencies successfully complete an SER? 

Lower-impact events that occur frequently — problems that everyone recognizes 

but has become inured to because they are so commonplace — can provide an 

opportunity for significant system improvement through an SER process. 

Are there “bad outcomes” that are not amenable to an SER? Almost certainly. 

The goal of an SER is to discover — and address — underlying system flaws and 

National Institute of Justice | NIJ.gov 
PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 5 

weaknesses. Many bad outcomes signal system flaws, and these events are ripe for 

SER. Other bad outcomes may be more limited in their system implications, such as 

events involving a single actor or a single cause. (Sometimes, a bad apple is simply a 

bad apple.) However, these distinctions regarding bad outcomes are not always easy 

to discern at the outset. Events where participants seem too quick to assert “whose 

fault this is” may be exactly the kinds of events that would benefit from an SER. 

Who Should Be on the SER Team? 

With respect to considering the 

composition of a sentinel event review “Members of the sentinel event review 

team bring with them their own biases to 

what is meant to be a non-biased process. 

Consider the impact of key factors like 

trauma, race and poverty on the actors in 

the system and on the persons who are 

the subject of the review.” (Site A) 

team, the three beta sites offered this 

insight: 

• Decision makers who can help maximize 

the opportunity to make systemwide 

changes. 

• Sharp-end-of-the-stick practitioners 

with front-line knowledge; sometimes 

executives don’t have the boots-on-the­


ground experience that is important to
 


understanding the event and the various
 

agencies’ responses to it.


 • Determine ahead of time how you will mediate situations where line staff feel 

constrained by having their bosses at the table. Consider recruiting line staff for 

the SER team in pairs, so they can support each other in what might be perceived 

as a “scary” process and empower them to contribute as individuals, both when 

leadership is present and when they are not.

 • Think broadly, including beyond criminal justice. For example, does the event touch on 

public health or mental health stakeholders? The education or social service systems?

 • Consider the pros and cons of including members of the community, who can 

ask the basic questions and help provide a larger community perspective. Would 

having representatives from grassroots organizations, faith-based groups, victim 

advocacy groups or other community organizations help make the SER process 

more sustainable in your jurisdiction? The medical model, for example, suggests 

that there are good reasons to include patients in a review of an error, including the 

principle that people don’t sue because they are harmed; they sue when they are 

not included.

 • Will you include people with firsthand knowledge of the event, including those whose 

decisions and actions at the time influenced the outcome of the event? If they are 

National Institute of Justice | NIJ.gov 
6 PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 

not available, are there other ways to gain their perspectives? If they do participate, 

how will you make the review environment safe and comfortable for them?

 • Who will not be on the review team? Families of victims or police officers or others 

who were involved in the actual event? Think about the pros and cons of making 

such individuals a part of the review team versus having them come and talk to the 

review team.

 • Bottom-line desirable traits for SER team members: people who are engaged in the 

process and people who see themselves as change agents.

 • The SER should be staffed with a notetaker who is responsible for taking minutes, 

compiling documentation, and tracking “homework” assigned to team members, 

but who does not participate in the review itself.

 • Determine your “substitute” policy. Will you allow principals to send substitutes to 

meetings? What are the implications for the team environment if substitutes drop in 

and drop out? How will you mediate those implications if they arise?

 • Is there “history” (contention) among SER 

team members before the process even 

“People in the process are not starts? How will you manage that? 

interchangeable ... you get to core • Determine how you will operate if you need 

issues because the relationships 	 to consider expanding the SER team at 

some point. How will members of the team have been developed.” (Site A) 

weigh in on the decision to include new 

members? What will happen if the team 

cannot agree? 

Who Should Lead the Sentinel Event Review? 

The leader — or facilitator — of the SER team acts like a project manager. Do you 

want an independent, neutral convener or someone from one of the stakeholder 

agencies? Either way, it will be a challenge to overcome the significant, but not 

surprising, reality that people are busy and it’s difficult to meet regularly. Pick a 

facilitator who can (and wants to) make the time to lead the process. 

The three beta sites agreed that the role of facilitator took significant time and energy 

(managing all the logistical details of setting up and planning the review meetings) 

and a commitment to and understanding of the philosophical underpinnings of the 

SER process. Where they differed was how they operationalized the leadership role. 

At one of the sites, the facilitator was a member of the senior staff of a participating 

agency. In the two other sites, the facilitator was the research partner: One facilitator 

National Institute of Justice | NIJ.gov 
PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 7 

was embedded in one of the participating agencies but was not an agency employee; 

the other researcher-facilitator was from a nongovernmental institution. 

The facilitator should be someone who is intellectually curious and well-informed 

about the philosophy of SER. The facilitator’s social capital, personal relationships 

with SER team members, personality and facilitation skills are important. 

The facilitator must be someone the others can trust; he or she must also have the 

means to hold others accountable for their participation in the SER. 

Be aware of any perception that the facilitator may be aligned with one agency or one 

point of view. 

The beta project experience suggests that there are a variety of workable alternatives 

for selecting a facilitator, as long as she or he is capable, dedicated, and respected by 

team members and understands the leadership role. 

What Is the Role of a Researcher in SER? 

The role of a researcher on a sentinel event review team is not that of a traditional 

academic research partner. A researcher may be a valuable member of the team, 

not a passive observer or data collector but, rather, a fully participating member. The 

researcher should avoid being perceived as a process evaluator as that could have a 

dampening effect on the review. 

Some of these roles would offer opportunities for a researcher to make a positive 

contribution to an SER:

 • Serves as the empiricist who weighs the evidence and challenges the team on their 

thinking.

 • Checks fidelity of the SER model as it unfolds.

 • Operates as a neutral team member who is not aligned with any particular agency 

or perspective regarding the “facts.”

 • Could provide a neutral environment for the team by hosting the SER process in an 

academic environment.

 • May be most useful if he or she has one foot in the practice world and one foot in 

the research world (an action-research model).

 • Can help establish the credibility of the SER process within the criminal justice 

system and with community stakeholders.

 • Brings qualitative research expertise, such as content analysis of documents, 

interviewing, and software applications for qualitative coding. 

National Institute of Justice | NIJ.gov 
8 PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 

• Can help identify key lessons learned and recommendations, including possible 

suggestions regarding which recommendations should be addressed first. 

Could the researcher serve as the leader or facilitator of the SER process? Yes. 

Because the researcher can be more neutral, she or he may be better able to bring 

the criminal justice professionals from various agencies together. 

How Do You Structure the SER and Set Ground Rules? 

At this stage in the development of an SER process within criminal justice, it is 

important to distinguish between the “local ground rules” that each jurisdiction’s 

SER team can set and the more requisite components — a ‘Model A’ template, as 

it were — that are emerging as NIJ continues its exploration of using this innovative 

process to learn from errors. Just as the first Model A automobile had key, necessary 

components such as an engine and tires, it is becoming clear that certain Model 

A features are necessary to a successful SER in the criminal justice system: There 

must, for example, be a nonblaming 

atmosphere, a trusted facilitator, shared 

“The process can’t work if people come information, protected deliberations, and 

all stakeholders at the table, striving 

toward consensus, and no substitutes 

in late to the process and don’t have the 

history and don’t know the ground rules.” for the team members during reviews. 

(Site A) (See the next section, “What Does a 

‘Nonblaming’ Review Really Mean?”) 

In addition to the necessary components of SER, the beta sites’ experiences revealed 

local ground rules that should be addressed:

 • Decide where and how often the SER team will meet. Will you use teleconferencing 

when face-to-face contact is not critical? Consider scheduling meetings over the 

lunch hour. Publish a meeting schedule in advance.

 • Determine how long the SER will last. Take the time to map out how the review will 

unfold. Creating shared expectations around who will be responsible for delivering 

what information — and when — may help with follow-through and encourage 

team members to hold each other accountable for the process.

 • Make setting ground rules a team-building exercise at the outset of the process. 

Have participants explicitly agree to ground rules and to hold each other 

accountable for following them. Ground rules should address issues like information 

sharing, confidentiality, shared goals, engagement and participation, accountability, 

respect, and other issues important to the team, such as ensuring that everyone’s 

concerns get airtime. 

National Institute of Justice | NIJ.gov 
PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 9 

• Establish clear expectations for the purpose of the SER and how members will 

participate. 

• Be clear about the safety gauge. How will people communicate that they do not feel 

comfortable during the review process? Make sure individuals are willing to voice 

their misgivings about the process, and address those explicitly.

 • Create a plan to work through conflicts or tension in the group. Guard against the 

process collapsing because participants choose to withdraw rather than work out 

their differences.

 • Plan time for the SER team members to share information about their agency roles 

and processes. Even people who have worked in the criminal justice system for 

many years can be ignorant of (or have inaccurate assumptions about) how other 

agencies operate. Have each member of the review team describe his or her work to 

help SER members start thinking outside their own “silos.”

 • Devote time to building and reviewing a timeline to fill in gaps in individual 

knowledge of what happened and when, even if all or most of the SER team 

members have firsthand knowledge of the event — or even if the event was widely 

covered in the media.

 • Allow for periodic check-ins between individual team members and the facilitator to 

discuss how the process is going.

 • Decide whether you will seek the perspectives of people who do not participate on 

the team — for example, through interviews.

 • Try to ensure that the SER team stays intact and that the same people attend each 

meeting (that is, no “substitutes”); this will facilitate a mutual understanding of the 

ground rules.

 • To be a constructive SER member, it may be necessary to be both a critical 

observer of the process and also a supporter of it.

 • Based on the event, what expertise might the SER team need to consult or tap into? 

If, for example, the event being reviewed 

is the conviction of an innocent person 

and one of the factors in the wrongful 

conviction was a confession, an expert in 

false confessions may provide valuable 

input.

 • Recognizing that one of the necessary 

features of the SER ‘Model A’ is striving 

for consensus in understanding the 

“Charting an accurate and understandable 

timeline for the events that stretched 

out over a couple of years and involved 

a lot of people in a lot of agencies was 

challenging.” (Site C) 

National Institute of Justice | NIJ.gov 
 

 

10 PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 

system weaknesses that may have contributed to the event, how will you determine 

when the review is complete if you can’t reach a consensus? What provisions can 

be made for concluding a review when consensus isn’t achieved?

 • Decide as a group what the outputs and outcomes will be and the timeline for 

producing any work products, including final reports or recommendations. (See 

“What Happens When an SER Is Complete? Measuring ‘Impact’ and ‘Outcomes’”.)

 • Be mindful of emotional consequences for the SER members. Don’t forget 

you’re human: Reviewing a high-impact event with a bad outcome can be 

emotionally difficult. 

What Does a ‘Nonblaming’ 

Review Really Mean? 

“... holding each other accountable 

A number of the members of the beta [for maintaining the nonblaming tone teams noted that we seem to be in an era of 

of the review] has a purpose so no one 	 organizational fatigue, which is a challenge to 

experimenting with any new review process. leaves the meeting feeling angry or 

Therefore, it’s crucial to understand — and struggling with the process.” (Site A) demonstrate — that SER is different from 

other reviews of critical incidents, such as 

after-action reports, task forces, independent 

monitors, commissions, inspector general reports, internal affairs investigations and 

performance management systems like CompStat. One significant way in which 

SER is different: It is not about assigning blame. 

Here are other lessons learned:

 • SER is a process that promotes a culture of continual self-improvement.

 • One of the greatest challenges is to get people who operate within a highly 

adversarial system to avoid reflexive defensiveness. What can you do to instill a 

culture of collaboration and nonblaming in an SER? How can SER team members 

“model” openness, especially when it means stepping outside of traditional 

adversarial roles? Consider volunteering/admitting errors as a way to overcome 

defensiveness.

 • In an effort to not cast blame, one beta participant found it helpful to try to 

understand what the quarterback sees when he throws an interception. Others 

said it was helpful to refer to the event as a “problem” or “issue,” rather than as a 

“mistake.”

 • Be aware that there may be trust issues among team members during the first 

SER, but this likely will improve with the second review, and onward, as people start 

National Institute of Justice | NIJ.gov 
PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 11 

putting more trust in a blame-free
 


approach to learning from error.
 
 “Remind team members that everyone 

Here is some additional advice [each agency] operated with what 

for achieving a nonblaming SER information they had at the time. This may environment: 

help alleviate some blaming.” (Site C) 

• Make the meeting room a “safe 

space” where each participant has 

the freedom to discuss the event and share their information.

 • Make sure individuals are willing to voice their misgivings about the process and 

address those explicitly. 

How Do You Deal With Data-Sharing Issues? 

With respect to records regarding the event being reviewed, a number of factors 

should be considered:

 • What records are you likely to need? Are they still available? Where are they and 

who “owns” them? Will you need data specialists to access or retrieve them?

 • Will SER team members be responsible for bringing their own agency’s records to 

the table, or will there be a different arrangement for obtaining them?

 • Is every SER team member able to see all of the records? What are the recordsharing implications if one of your review team members is not a public employee? 

If reviewing the event will require going through lots of records and files, consider 

splitting the work up among team members.

 • How might public access and sunshine laws be triggered when data are shared 

among team members?

 • Are some records more sensitive than others? If so, are separate data-handling 

procedures necessary? How will you address issues of disclosure? A signed 

confidentiality agreement may help, although it may not survive a subpoena. 

Consider whether a Memorandum of Agreement or a court order could be useful in 

facilitating data-sharing and maintaining protections for sensitive data.

• If you encounter a situation where data 

you need are unavailable, are there 

alternative sources or records? 

“One bad experience [with a sentinel 

event review] could derail this before 

• Do you need copies of records and files, people see the benefit of it.” (Site C) 

or will you work with notes abstracted 

from the originals? Consider the cost of 

photocopying and scanning and who will be responsible for those costs. 

National Institute of Justice | NIJ.gov 
12 PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 

Where Will Challenges Likely Arise? 

The following graphic summarizes the logistical, interactional and structural challenges that the beta sites 

most frequently reported in performing a sentinel event review. 

Logistical Challenges 

Factors regarding the mechanics of implementing the SER process 

Finding enough time to commit to the process, both the monthly meetings and the “homework” 

information-gathering, whether done by team members or assigned to staff. 

Coordinating everyone’s calendars to keep a consistent group coming to the meeting. 

The amount of work involved in doing a thorough review. 

Feeling as if the team is repeating the same discussions over and over, but with so much information (and 

with new players coming into the meetings), it is hard not to repeat. 

Participants who come to the table because they are required to but do not have the time or will to devote 

to conducting a review the way it “should” be done. 

Having the wrong people at the table could kill the process. 

Withholding records or information. 

Participants who are too busy to participate. 

Interactional Challenges 

Factors regarding how SER team members relate to each other during the process 

Defensive or close-minded attitudes of participants who are resistant to considering their own agency’s 

role in the circumstances that lead to the event. 

Getting a consistent level of buy-in from all participants; ensuring everyone understands the importance 

of the SER process. 

Convincing participants that the SER process is about trying to make things better, not about pointing fingers. 

People who are only concerned about their own goals and what they want and are unable to value others’ 

goals and viewpoints. 

Lack of honesty among participants. 

Breaches of group confidentiality. 

Reluctance to be critiqued, scrutinized or exposed to civil liability. 

Lack of teamwork or cooperation. 

National Institute of Justice | NIJ.gov 
PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 13 

Structural Challenges 

Factors regarding the use of SER in a criminal justice context 

The professions represented are natural adversaries in their “day jobs” and had to overcome that to work 

as a team. 

Having to confront the lack of empirical evidence about an older event where the records are inadequate. 

Getting all the information to the table because the event involves a lot of systems/agencies. 

Understanding other agencies and systems and how they operate. 

Facing legal barriers related to data confidentiality, which dictates what can be discussed by the SER team. 

Elected officials may be unwilling to “stick their necks out” to participate in the process, but their buy-in 

is critical. 

Creating a safe environment in which to conduct the SER. 

Making decisions based on unique outlier events/cases; hard to draw larger conclusions from a single 

case- or event-specific example. 

Recommendations may go beyond policy and practice changes that are within the scope of agency 

leadership to implement. Some change that comes from the SER may require legislative action. 

Picking a high-profile event/case may make it difficult to ensure that people approach the SER as 

objectively as possible; they may already have opinions about what happened and why. 

The inclusion of people who do not work within the criminal justice system may be a challenge because 

of confidentiality issues surrounding nonpublic data. 

What Happens When an SER Is Complete? Measuring 

‘Impact’ and ‘Outcomes’ 

Although it’s unlikely — especially in the review of a closed or older case — that 

critical new information will come to light, the SER team should think about how this 

would be handled if it occurs. 

Even if the SER team agrees on desired outcomes, it is still important to think about 

outputs. What products, reports or recommendations will be produced? Who will be 

responsible for producing them? On what timeline? What process will be used to help 

ensure that the team achieves whatever level of group buy-in they deem necessary? 

At the end of the process, when recommendations are being formulated, test the 

recommendations with the team to determine whether they address universal issues 

or are related only to anomalies in the event reviewed. 

National Institute of Justice | NIJ.gov 
14 PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 

“You can do the process of sentinel event 

review well and not achieve its purpose at 

all if [no change] is implemented.” (Site B) 

As noted earlier, NIJ’s third research 

question is whether the SER approach 

provides measurable outcomes in terms 

of system functioning, avoiding errors in 

the future, and achieving greater public 

safety dividends. It may be challenging 

to measure the success of an individual 

SER, let alone to measure the culture 

shift that may accrue over time, based on using the new learning-from-error process. 

That said, here are some possible metrics that the beta SER teams raised:

 • Were recommendations for changes in policies or procedures made?

 • Were the recommendations presented to decision makers?

 • Did SER team participants value the process — as measured, for example, by 

surveys of team members regarding perceptions of the candor, information-sharing 

and participation of themselves and their fellow SER team members.

 • Were similar “potential” sentinel events averted? 

Do all (or any) SER recommendations have to be “revolutionary?” Consider that 

the process itself may be part of what’s revolutionary ... and publishing even 

nonrevolutionary recommendations will help improve accountability and transparency. 

Will the SER recommendations be publicly released or kept confidential? Public 

dissemination may help SER achieve its goal of becoming an important tool in helping 

criminal justice professionals learn from errors in their own and other jurisdictions. 

Consider what follow-up mechanisms are in place (or might be put in place) to help 

promote the adoption of any SER team recommendations or action items. Here is 

some guidance:

 • Be strategic about how you present your findings or recommendations. Make sure 

decision makers are kept informed of your progress throughout the review so they 

don’t lose sight of the end-goal.

 • Be prepared to talk about the SER process as you present your findings to reinforce 

its utility (without violating confidentiality rules agreed to by the team).

 • Build toward sustainability (see the next section, “How Can a Jurisdiction Sustain 

Sentinel Event Reviews?”). Conduct a review of your SER process to determine what 

lessons you learned. What would you do the same next time? What would you do 

differently? 

National Institute of Justice | NIJ.gov 
PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 15 

The SER process is challenging because it is a major paradigm shift from how the 

criminal and juvenile justice systems typically operate. Therefore, it is important to 

have realistic expectations of this process. Instilling a nonblaming atmosphere with 

an emphasis on transparency and accountability (what other fields call a “culture of 

safety”) takes work, and it does not happen with a single review of an event. Even 

one SER can be a very labor-intensive process, although this may decrease once the 

SER process becomes more routine. One beta participant put it this way: “It is very 

hard to step back and take a 30,000-foot view, but incredibly important that we do 

so.” Said another: “These types of case studies are where the organizational learning 

takes place.” 

By its very nature, the criminal justice system is adversarial and the SER process 

provides an opportunity for people to get together on a regular basis outside of their 

usual roles. 

SER can actually be enjoyable. It can 

provide the opportunity to take a break 

from the usual, generally overwhelming 

production pressures. It can provide a rare 

opportunity (that would become less rare 

if it produces measurable benefits and is 

institutionalized) to step back and take a 

30,000-foot view of errors and discuss
 


improvements. 

“The audience for the recommendations 

(agency leadership, political leadership, 

whomever it is) must see the credibility 

and objectivity in the process, so the 

process must be conducted in a way that 

undergirds that.” (Site C)
 


SER can be a personally and professionally 
 

gratifying process that provides a chance 
 

to learn from other people in the criminal and juvenile justice bureaucracy and 
 

potentially other stakeholder agencies and organizations. Millions of people work in 
 

individual agencies — law enforcement, crime labs, the prosecution and defense 
 

bars, judges, corrections, victim advocates and service providers — and never have 
 

the opportunity to participate in this sort of review of error.
 


How Can a Jurisdiction Sustain Sentinel Event Review? 

In an era of having to do more with less, the sustainability and institutionalization of 

SER requires demonstrable results — not only in policy and practice but eventually 

in the workplace and systemwide culture. That said, how SER is sustained and 

institutionalized will undoubtedly be unique in every jurisdiction — but the beta 

project suggests that it will require the political will from leadership to make it a part 

of the culture of the agencies involved. 

National Institute of Justice | NIJ.gov 
16 PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 

Some members of the beta teams said that implementing SER would likely require 

resources (money and extra support staff) to permit already busy people to participate 

in what may be a time-consuming review process. Others expressed the sentiment 

that this should not be necessary if SER was considered just a part of their job. 

An investment in resources for team-building on the front end of the process 

could be helpful. 

One challenge to the sustainability of SER is that the criminal justice system is not 

driven by the scientific method; it is driven by laws. Still, it is possible to examine 

causes of error and do some proactive prevention. This could argue for the role of 

an academic in representing the scientific method in this kind of nonblaming, allstakeholder review process. 

Routine, multiple reviews can help refine our response to persistent systemic 

problems. Sometimes, a recommendation does not “stick” the first time around — 

and, even when a recommendation is adopted, it may quickly come under attack 

in the form of workarounds that circumvent the recommended fix, often as a result 

of pressures to “keep up production” and not slow the process. A commitment to 

conducting multiple reviews may help to mitigate this. 

Think about where the SER process will be “housed” — it could be housed in a 

university or other academic setting. Another option might be to house SER in a 

criminal justice advisory board (if a jurisdiction has this), which might have more 

“ownership” in reviewing public-agency errors. 

Find SER champions or supporters — that is, agencies and people who are not 

members of the SER team but are supportive of the goals and the innovative process. 

Multiagency commissions or coordinating councils (such as local criminal justice 

commissions) could be champions or supporters. 

Could risk managers play a crucial role? Could risk managers serve as the logical 

counterpart to medical insurance companies with respect to how the medical field 

adopted SER as a way to learn from errors? Could they serve as change agents if 

they believe that cost savings could accrue from SER over time? What challenges 

exist in making risk managers supporters? If they’re change-averse or not interested 

in potential reform, how can they be encouraged to see their stake in the process? 

Is the mayor’s office a possible supporter? Or a jurisdiction’s office of management 

and budget? 

National Institute of Justice | NIJ.gov 
PAVING THE WAY: Lessons Learned in Sentinel Event Reviews 17 

The SER process must have clear goals, which may be unique to each jurisdiction. 

It’s important for each stakeholder to understand what’s in it for them and what they 

will gain from the review. It is crucial that leadership, balancing competing demands, 

develops confidence that SER is worth the investment. 

To sustain and institutionalize SER, it will be necessary to convince the public — the 

ultimate stakeholders in the criminal justice system — of its value. 

National Institute of Justice | NIJ.gov 

 

Learn more about NIJ’s Sentinel Events Initiative — which aims to improve 

the administration of justice in our nation’s criminal justice and juvenile 

justice systems — in Mending Justice: Sentinel Event Reviews 

(www.NIJ.gov, keyword: Mending Justice). 

Stay tuned to NIJ.gov (keyword: Sentinel Events) for more about this 

initiative. 

Or join the conversation on social media: 

Follow us on twitter.com/OJPNIJ 

Like us on facebook.com/OJPNIJ 
*NCJ~249097*