U.S. Department of Justice

Office of Community Oriented Policing Services

COPS COMMUNITY ORIENTED POLICING SERVICES

U.S. DEPARTMENT OF JUSTICE

www.cops.usdoj.gov 

Early Intervention Systems for

Law Enforcement Agencies:

A Planning and Management Guide

Prepared for

Office of Community Oriented Policing Services

1100 Vermont Ave., N.W.

Washington, D.C. 20005

www.cops.usdoj .gov

Prepared by

Samuel Walker, PhD

Department of Criminal Justice

University of Nebraska at Omaha

2003

This report was funded by Grant #2000-CK-WX-K124 from the Office of Community

Oriented Policing Services of the U.S. Department of Justice. The views expressed in

this report do not necessarily reflect to views or the official policies of the 

U.S. Department of Justice.

ISBN# 1-932582-26-6

Early Intervention Systems for

Law Enforcement Agencies:

A Planning and Management Guide

Table of Contents

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .i

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .iii

Chapter 1. Introduction to Early Intervention Systems . . . . . . . . . . . . . . . . . .3

Chapter 2. The Components of an Early Intervention System . . . . . . . . . . . .27

Chapter 3. Background: The Origins and Development of Early

Intervention Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47

Chapter 4. Lessons From the Field: Police Managers' Perceptions and

Experiences with Early Intervention Systems . . . . . . . . . . . . . . . . . . . . .77 

Chapter 5. Planning and Implementing an EI System . . . . . . . . . . . . . . . . . .99

Chapter 6. Policy and Research Needs for Early Intervention Systems . . . . .115

Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123

Table of Contents

Executive Summary

i

• Early Intervention (EI) systems are an effective mechanism for enhancing accountability

within law enforcement agencies.

• Police managers experienced with EI systems report that they have a positive impact on

officer performance and that they strengthen supervision.

• EI systems are extremely complex administrative tools that require close attention by

police managers.

• In creating an EI system, careful decisions need to be made about the number of

performance indicators to be used and the thresholds to be used to identify potential

problem officers.

• For an EI system to be effective, sergeants and other supervisors need extensive training

about their new responsibilities.

• EI systems are consistent with the goals of Community Policing and can help to improve

police-community relations problems.

• Careful planning is essential in the development of an effective EI system. The key to the

planning process is the involvement of officers from all ranks of the department.

• EI systems are a "work in progress." Existing systems need to engage in a process of

continuous review and fine-tuning.

• This report focuses on EI in larger agencies: the exact size and scope of EI systems for

small and medium-sized law enforcement agencies is not known at this time.

• The law enforcement profession needs to sponsor continuing policy development related

to EI systems. 

• Additional research is required on the impact and effectiveness of EI systems.

Executive Summary

Acknowledgments

iii

This report was supported by Grant #2000-CK-WX-K124 from the U.S. Department of Justice

Office of Community Oriented Policing Services (COPS). The views expressed in this report do

not necessarily reflect the views or official policies of the U.S. Department of Justice.

I would like to express my special thanks to Dr. Ellen Scrivner of the COPS Office for her

support of this effort and for her on-going commitment to the issue of police accountability.

Pam Cammarata picked up the project after Dr. Scrivner left the COPS Office and has been

very supportive of it over the course of the last year.

Professor Geoffrey P. Alpert of the University of South Carolina served as a major consultant

to this project, and I would like to express my thanks for all of his help.

Dawn Irlbeck, my Research Assistant at the University of Nebraska at Omaha, made a

tremendous contribution to this report. She had major responsibilities for developing and

administering the PERF survey and for analyzing the results that are presented here in

Chapter Four. She also proofread the final report and saved me many embarrassments. 

I would like to thank her for all of her work.

Chapter Two and the Appendix report the findings of a survey administered in collaboration

with DiscoverWhy and the Police Executive Research Forum (PERF). I would like to thank 

Dr. Lorie Fridell, Research Director at PERF, and both Brad Ashford and Linda Ordoukhanian at

DiscoverWhy for agreeing to collaborate on this survey.

This project also involved a series of workshops hosted by four Regional Community Policing

Institutes (RCPIs) across the country. This included the RCPIs in Tampa, Florida; Baltimore,

Maryland; Chicago, Illinois; and Sacramento, California. I would like to thank the directors

and staff of those agencies for their assistance. Sergeant Ed Hudak of the Coral Gables (FL)

Police Department assisted Professor Alpert and myself in those efforts.

The author of this report has had the benefit of tremendous cooperation from officials at

both the Los Angeles County Sheriff's Department (LASD) and the Phoenix Police Department.

At the LASD I would like to thank Commander Neal Tyler and Commander William J.

McSweeney. At the Phoenix Police Department I would like to thank Lieutenant Dan Olsen,

Detective Toye Nash, and Sergeant Ron Snodgrass. 

The Phoenix Police Department hosted the EI System State of the Art Conference in February

2003. I would like to thank Chief Harold Hurtt, Lieutenant Olsen, Detective Nash, and

Sergeant Snodgrass for all of their help with that valuable event.

The author of this report provided technical assistance to the Douglas County (NE)

Department of Corrections in developing its Performance Enhancement Program (PEP)

beginning in the year 2000. That experience provided valuable insights into one agency's

planning process.

Acknowledgments

Chapter One

Introduction to Early Intervention Systems

The Concept of Early Intervention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

A Word About Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

A Recommended Best Practice in Police Accountability . . . . . . . . . . . . . . . . . . . . . . .8 

EI Systems and Other Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

Goals and Impacts of EI Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

Individual Officers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

Supervisors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

The Department . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

Early Intervention and Community Policing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16

Early Intervention and Problem-Oriented Policing . . . . . . . . . . . . . . . . . . . . . . . . . . .17

Early Intervention and COMPSTAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17 

Early Intervention and Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18

Early Intervention and Police-Community Relations . . . . . . . . . . . . . . . . . . . . . . . . . .18

Early Intervention and Racial Profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

One Size Does Not Fit All Departments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20

A Work in Progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22

In an EI system, performance data are entered into a computerized

database. These data include departmental use-of-force reports,

citizen complaints, officer involvement in civil litigation, resisting

arrest charges, and other performance indicators. Some current EI

systems use a dozen or more performance indicators while others

use a smaller number.

An EI system is early in the sense that it helps to identify officer

performance problems that do not warrant formal disciplinary

action but suggest that an officer is having problems dealing with

citizens. The major contribution of an EI system is its capacity to

spot patterns of performance and to intervene before problems

lead to a serious incident such as a lawsuit, a citizen complaint

over excessive force, or some other public crisis involving the

department. An EI system warns an officer to the extent that it

sends an informal but nonetheless clear message that his or her

performance needs improvement. 

Introduction to Early Intervention Systems

3

The Concept of Early Intervention 

An Early Intervention (EI) system is a data-based management tool designed

to identify officers whose performance exhibits problems, and then to provide

interventions, usually counseling or training, to correct those performance

problems. EI systems have emerged as an important mechanism for ensuring

police accountability.1

EI systems do more than just focus on a few problem officers. As a 1989

report by the International Association of Chiefs of Police (IACP) explains, an

EI system is "a proactive management tool useful for identifying a wide range

of problems [and] not just a system to focus on problem officers."2

1
An EI system is officially separate from the formal disciplinary

system. It is designed to help officers improve their performance

through counseling, training, or coaching. No record of

participation in an EI program is placed in an officer's personnel

file, although a separate record of participation is usually

maintained by the internal affairs or professional standards unit.

One or more of the incidents identified by an EI system may

warrant formal disciplinary action that is officially recorded, but

identification by the EI system remains separate from the

disciplinary process.

EI systems are data-driven mechanisms of accountability,

providing systematic data as a basis for performance evaluations

of officers. They differ from traditional performance review

systems that rely heavily on subjective assessments such as

"works well with people" or "demonstrates initiative."3 The

database can identify specific areas of performance that need

correcting (e.g., a pattern of citizen complaints alleging rudeness).

Bob Stewart, former Executive Director of the National

Organization of Black Law Enforcement Executives (NOBLE),

says that "If I could choose only one accountability mechanism, it

would be an early intervention system." Stewart has seen police

performance problems from several perspectives: he was a police

officer in Washington, DC, the chief of police in Ormond, Florida,

and is now a consultant to communities and law enforcement

agencies around the country. Based on this long experience, he

now regards EI systems as the single most valuable tool for

achieving accountability, primarily because of their capacity to

monitor a wide range of officer activities and to spot performance

problems at an early stage.4

EI systems are retrospective performance reviews. They do not

attempt to predict officer performance based on background

characteristics or other factors, rather they indicate that current

performance levels, while not warranting disciplinary action, still

warrant improvement. Past efforts to develop a methodology for

predicting which applicants for police employment will perform

well and which are not likely to perform well have not proven

successful.5 An EI system provides a basis for counseling or

4

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 1
5

warning an officer that his or her performance needs to improve

in the future, and in successful cases, for documenting that the

improvement has occurred.

EI systems are designed to help officers improve their

performance. The names of several EI systems reflect this

orientation. The New Jersey State Police EI system is known as

the Management Awareness Program (MAP), indicating its

purpose of assisting management. In this respect, EI systems

represent a significant departure from traditional police

disciplinary practices. Law enforcement agencies have been

punishment-oriented bureaucracies, with innumerable rules and

regulations that can be used to punish an officer, but with few

procedures for either rewarding good conduct6 or helping officers

with problems. Apart from employee assistance programs (EAP)

designed to address substance abuse or family problems, police

departments have done relatively little in a formal way to correct

problem behavior.7 An EI system helps to identify specific

performance problems that need to be addressed (e.g., a tendency

toward verbal abusiveness, frequent charges of resisting arrest,

etc.).

EI systems represent a problem-solving approach to officer

performance. The problem in this instance involves questionable

officer performance. The problem-solving involves identifying

officers in need of assistance and providing that assistance

through counseling or training. (The EI problem-solving process

is discussed in detail later in this chapter.) As one commander

with EI system experience explained, his agency's EI system

"provides a way for the department to provide non-disciplinary

direction and training before the officer becomes a liability to

citizens, the department, and him/herself" (See Chapter Four).

EI systems are very similar to COMPSTAT programs.

COMPSTAT is one of the most important innovations in police

management. It is a data-based system designed to help law

enforcement agencies respond effectively to crime and disorder

and to hold their managers accountable for their performance.8

Both EI systems and COMPSTAT programs rely on the analysis

of systematic and timely data–in the case of COMPSTAT the data

Introduction to Early Intervention Systems

Chapter 1
Chapter 1

involve crime and disorder, with EI systems the data involve

individual officer performance.

EI systems are consistent with the basic principles of personnel

management and human resource development.9 Employers

recruit, select, and train employees to effectively serve the goals

and objectives of their organizations. Effective personnel

management assumes that employee performance is assessed and

evaluated on a regular basis, and that the organization takes steps

to correct unsatisfactory performance.

Contrary to the expectations of many people, EI systems have not

encountered significant opposition from police unions

representing rank and file officers.10 To be sure, there has usually

been grumbling and fear of the unknown, but in practice unions

have not succeeded in blocking the operation of an EI system

once it is in place. The police managers' survey reported in

Chapter Four of this report found that only 16 percent of

managers had encountered serious opposition from police unions.

Potential opposition from unions is best overcome by involving

union representatives in the planning of an EI system (see Chapter

Five).

EI systems are consistent with the goals of community oriented

policing (COP). Community policing creates demands for

responsiveness to community residents and in the process new

measures of police performance. The traditional measures of the

crime rate and the clearance rate are no longer adequate. EI

systems have the capacity to quickly document the performance

of officers who are not effectively serving the community and

provide a basis for remedial action by the department.

The available evidence indicates that EI systems are successful in

achieving their goals of reducing officer misconduct. An NIJ

study of EI systems in three police departments found significant

reductions in use-of-force and citizen complaints among officers

following EI intervention.11 Commanders with EI system

experience, meanwhile, are able to report specific examples of

individual officers whose performance improved as a result of

intervention. One commander, for example, has described an

6

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide
7

officer who had on-the-street problems because of an excessive

fear of being struck in the face. Intervention counseling identified

the problem and retraining in tactics helped the officer overcome

the problem. Commanders report many other such success stories.

In the survey of managers which is summarized in Chapter Four

of this report, about half (49 percent) reported that the system has

had a positive impact on the on-the-street performance of their

officers, while almost a third (28 percent) reported a mixed

impact. No commanders reported a negative impact.

Chapter Three of this report discusses in detail the research

findings regarding the effectiveness of EI systems. Chapter Four,

meanwhile, reports the findings of a survey of police managers

with EI system experience. Generally, managers have had very

positive experiences with the impact of their systems of

supervision and accountability.

A Word About Terminology

Early Intervention

This report uses the term early intervention. It does not use the

more widely used term early warning. Early warning has a

negative connotation, suggesting that the system is primarily

oriented toward discipline. One department with a comprehensive

EI system found through interviews with officers that they did not

like the phrase early warning because of its "big brother"

Introduction to Early Intervention Systems

Chapter 1

• Data-based management information system.

• Capacity for identifying and correcting performance problems.

• Recommended police accountability "best practice"

• Separate from the formal disciplinary system

• Consistent with the goals of Ccommunity Ppolicing

• Consistent with the process of Pproblem-Ooriented Ppolicing

• Rely on systematic and timely data

• Consistent with the process of COMPSTAT

• Careful planning is needed for development and implementation

BOX 1.1 EI Systems: An Overview
connotation. EI systems are evolving in the direction of more

comprehensive personnel assessment systems, for the purpose of

examining a broad range of performance issues. For this reason,

the more positive early intervention term is used here. Other

terms are also appropriate. These include Personnel Performance

Index, as used by the Los Angeles County Sheriff's Department,

or the Personnel Assessment System (PAS), as used by the

Phoenix Police Department.

Officers With Performance Problems

This report uses the term officers with performance problems

rather than the commonly used term problem officers. The latter

terms unfairly labels officers and suggests that their performance

cannot change. The term officers with performance problems

focuses on behavior without labeling an officer and conveys the

message that performance can improve.

A Recommended Best Practice in Police

Accountability 

EI systems have been recommended by a wide range of

organizations as a best practice in police accountability.

• A January 2001 report by the U.S. Department of Justice

(2001), Principles for Promoting Police Integrity, includes EI

systems among its recommended best practices.12

• The Commission on Accreditation for Law Enforcement

Agencies (CALEA, 2001) has adopted Standard 35.1.15

mandating EI systems for large agencies. The CALEA Standard

states that:

A comprehensive Personnel Early Warning System is an

essential component of good discipline in a well-managed law

enforcement agency. The early identification of potential

problem employees and a menu of remedial actions can

increase agency accountability and offer employees a better

opportunity to meet the agency's values and mission

statement.13 

8

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 1
9

The CALEA Standard warns that a department should not be

faced with a situation where an officer is alleged to have

committed a serious act of misconduct where "there was an

escalating pattern of less serious misconduct, which could have

been abated through intervention." The Standard suggests but

does not require the use of such performance indicators as use-offorce incidents, citizen complaints, and disciplinary actions.

• A 1989 report by the International Association of Chiefs of

Police (IACP) (1989:80) recommends EI systems as a means of

controlling corruption and building integrity in police

departments.14

• The U.S. Civil Rights Commission was the first agency to

recommend EI systems, in its 1981 report, Who is Guarding the

Guardians?15

• Since 1997, EI systems have been included in consent decrees

and memoranda of understandings settling law suits brought by

the Civil Rights Division of the U.S. Justice Department under

the "pattern or practice" clause of the 1994 Violent Crime

Control Act. EI systems are mandated in the agreements related

to the Pittsburgh Police Bureau; the New Jersey State Police;

the Metropolitan Police Department of Washington, D.C.; the

Los Angeles Police Department; and the Cincinnati Police

Department.16

The consent decree settling the Justice Department suit against the

New Jersey State Police, for example, requires the development

of a Management Awareness Program (MAP) for the purpose of

"maintaining and retrieving information necessary for the

supervision and management of the State Police to promote

professionalism and civil rights integrity, to identify and modify

potentially problematic behavior, and to promote best

practices..."17 The consent decree with the Los Angeles Police

Department (LAPD) sets requirements for an EI system, known as

TEAMS (Training Evaluation and Management System). The

consent decree specifies seventeen categories of data to be

collected and entered into the system.18 

Introduction to Early Intervention Systems

Chapter 1
EI Systems and Other Best Practices

EI Systems are the centerpiece of an emerging package of best

practices designed to enhance police accountability. The other

elements in the package of best practices, as recommended by the

U.S. Department of Justice report Principles for Promoting Police

Accountability (2001), include a comprehensive use-of-force

reporting system and an open and accessible citizen complaint

system. Data on officer traffic enforcement can also be entered

into an EI system and can be used to address the issue of potential

racial bias in traffic stops.19 The EI system is the central

repository of data on use-of-force and citizen complaints (and

other indicators). In this new paradigm of accountability,

particular issues such as use-of-force performance are no longer

treated as separate items but are linked to all areas of officer

performance in a systematic fashion. 

A Vera Institute evaluation of the implementation of a consent

decree in Pittsburgh commented that "the early warning system is

the centerpiece of the Police Bureau's reforms in response to the

consent decree."20 It is the mechanism by which these

performance data become useful to managers committed to

enhancing accountability and reducing officer misconduct.21

Goals and Impacts of EI Systems 

The goals and potential impacts of EI systems are broader than

generally understood. When the EI concept originally developed

more than twenty years ago, EI systems had a narrow focus on

"problem" police officers. The implicit idea was to "catch" the

"bad" cops. As EI systems have developed over time, however,

10

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 1

• U.S. Department of Justice

• U. S. Civil Rights Commission

• Commission on Accreditation for Law Enforcement Agencies

• International Association of Chiefs of Police

BOX 1.2 EI Systems Recommended By:
11

police managers increasingly understand that they have broader

goals and potential impacts, including individual officers,

supervisors, and the department as a whole.

Individual Officers

EI systems have a major focus on individual officers. The goal of

the system is to correct the performance of officers who appear to

be having performance problems. The NIJ study found that the EI

systems investigated did succeed in reducing citizen complaints

and officer use of force among officers subject to intervention.

This project and the findings are discussed in detail in Chapter

Two.22

One important change that has occurred is that there is now a

broader definition of a "problem" employee. Originally, problem

officers were defined primarily as those who frequently used

excessive force and received a large number of citizen complaints.

Reflecting a broader approach, however, the commander

responsible for the Pittsburgh Police Bureau's PARS reports that

the system allows them to identify their "top performers" and

their "under-performers" as well as their "problem officers".

Another police department with a comprehensive system, for

example, identified an officer who had made no arrests, no

pedestrian stops, and no traffic stops in a given time period. The

data also indicated that this officer was working the maximum

number of permissible hours of off-duty employment. These

indicators allowed the department to take the appropriate

corrective action. 

The capacity to identify top performers as well as officers with

performance problems requires a comprehensive EI system such

as the Pittsburgh PARS. Most of the early EI systems are not able

to identify under-performers, however, because they do not collect

the necessary performance data, such as arrest activity. The

difference between comprehensive and more limited EI systems is

discussed in detail in Chapter Two.

Introduction to Early Intervention Systems

Chapter 1
Supervisors

An EI system also reorients the role of supervisors in a

department. An EI database provides first-line supervisors with

documentary evidence of officers' performance and permits both a

detailed analysis of individual officers and comparisons among

officers. In the survey of police managers (See Chapter Four) one

manager reported that the EI system as "A useful tool to involve

supervisors and lieutenants in non-traditional models of problem

solving. It has served to enhance their management skills and help

round out their people interaction skills."

At the same time, however, orienting supervisors to their new role

is one of the major challenges facing EI systems. Engaging an

officer with performance problems and helping that person to

recognize his or her performance problems is a role that many

supervisors are reluctant to undertake. Some prefer to define their

role as friend and supporter of their officers. Some do not have

the skills for the task. Most have not been properly trained for this

new role. Supervisor training tends to emphasize the formal

disciplinary aspects of the job. The intervention phase of an EI

system requires supervisors to be a combination of coach and

trainer, providing a delicate mix of support, criticism, and help.23

As already mentioned, the Pittsburgh PARS takes a special

approach to reorienting the role of supervisors. Sergeants are

required to access the system's database and review the

performance of officers under their command. In addition, as

mentioned earlier, the PARS requires sergeants to conduct "rollbys" of officers who have been identified by the system. In this

way, PARS creates a structured and systematic form of intensive

supervision for officers who have been identified as having

performance problems. (In this and other systems, a supervisor

has a password granting access to all officers under his or her

command but no other officers. A district commander at the rank

of captain, for example, would have access to the records of all

lieutenants, sergeants, and officers in that district.) 

12

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 1
13

Some EI systems record each supervisor's access of the system.

Thus, the performance review of a sergeant can include a review

of the extent to which he or she accessed the system during the

period under review. 

An evaluation of the consent decree in Pittsburgh concluded that

the department's PARS "marks a sweeping change in the duties of

the lowest level supervisors." Supervisors log on to the PARS

each day prior to roll call. The screen notifies each supervisor of

any critical incident involving an officer under his or her

command since the last time that supervisor logged on.24

The consent decree regarding the Los Angeles Police Department

requires that "on a regular basis, supervisors review and analyze

all relevant information in TEAMS II about officers under their

supervision...."25 Additionally, the consent decree requires that

"LAPD managers on a regular basis review and analyze relevant

information in TEAMS II about subordinate managers and

supervisors in their command...and that managers' and

supervisors' performance in implementing the provisions of the

TEAMS II protocol shall be taken into account in their annual

personnel performance evaluations."26 The consent decree

involving the Cincinnati Police Department contains a similar

requirement that supervisors be evaluated on the basis of "they're

to use the risk management system to enhance department

effectiveness and reduce risk"27

Introduction to Early Intervention Systems

Chapter 1

The Individual Officer

• Improved Performance

• Higher Standards of Accountability

Supervisors

• Improved Supervisory Practices

• Higher Standards of Accountability as Supervisors

The Department

• Higher standards of accountability

• Reduction in litigation

• Improved Community Relations

BOX 1.3 Goals and Impacts of EI Systems
EI systems can also address the problem that, with regular shift

changes, sergeants often find themselves responsible for officers

who they do not know. As one commander reports, "There is a lot

of movement of personnel, so supervisors often do not know the

histories of their officers. The EWS report brings them up to

speed in a much more timely fashion." Chapter Four of this report

contains the findings of a survey of police managers with EI.

The San Jose Police Department has taken EI systems to a new

level by developing a Supervisor's Intervention Program (SIP)

addresses the performance of supervisors as well as rank and file

officers. Whenever the team of officers under a supervisor's

command receives three or more citizen complaints within a sixmonth period, the supervisor is required to meet with his or her

chain of command and the head of Internal Affairs. (The San Jose

EI system relies only on citizen complaints.) In the first year of

the SIP program, four supervisors met the thresholds and were

counseled by the department.28

Some departments are exploring ways to use their EI system to

hold sergeants accountable. The San Jose Police Department, for

example, identifies sergeants when officers under their command

receive a certain number of citizen complaints. This approach

recognizes that many officer performance problems are the result

of inadequate supervision.29

The Department

An EI system also has the potential for considerable impact on the

department as a whole. The system defines standards of conduct

and provides a database for measuring officer performance and

identifying substandard performance. At the same time, a fully

operational system redefines the role of supervisors, giving them

specific duties related to supervision of officers with performance

problems. And as mentioned in the previous section, the more

sophisticated EI systems have the capacity to monitor supervisors'

use of the system and to grade their performance accordingly. 

14

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 1
15

Problems identified by the EI system can also lead to changes in

departmental policy. The consent decree between the Justice

Department and the New Jersey State Police, for example,

requires that "Each supervisor shall, consistent with his or her

authority, implement any appropriate changes or remedial

measures regarding traffic enforcement criteria, training, and

enforcement practices for particular units or subunits..."30

Over the long run, the cumulative effect of the changes in the role

of supervisors and changes in departmental policy has the

potential for changing the organizational culture of a police

department and establishing new standards of accountability. In

the police commander survey (See Chapter Four), one commander

explained that over time rank and file officers came to accept the

goals of the EI system: "Most no longer believe this system is out

to 'get' them, but rather to assist them." A top commander in the

Pittsburgh Police Bureau claims that PARS has begun to change

the culture within the department. Two impacts are particularly

notable. First, the system forces a more intensive form of

supervision on the part of sergeants. Second, by identifying

officers who are underperforming because they are devoting too

much time to off-duty employment, and then correcting that

problem, PARS helps to develop a greater commitment by officers

to their primary employment.31

The positive impact on the organizational culture of a department

is not guaranteed, however. If there is little effective use of the EI

system–as in failure to conduct meaningful interventions with

officers–the impact could easily be negative: reinforcing officer

cynicism about the gap between stated ideals and actual practice

Learning from the Corrections Profession

The corrections profession has already begun to incorporate the

EI concept into its accreditation process. The American

Correctional Association (ACA) is adopting a "performancebased" approach to accreditation. The indicators for agency

performance are essentially the same as those used in law

enforcement EI systems. The Performance-Based Standards for

Introduction to Early Intervention Systems

Chapter 1
Adult Community Residential Services, for example, includes the

following standard:32

"Number of offenders' grievances filed alleging inappropriate

use of force in the past 12 months, divided by [the] Average

daily offender population in the past 12 months.”

The approach used by the ACA focuses on the agency, with

organizational-level performance measures, while law

enforcement EI systems focus on individual officers. There are

good reasons for suggesting that the ACA approach is the better

of the two. It properly focuses on organizational-level issues of

management and supervision where corrective action can prevent

future misconduct. The current law enforcement focus on

individual officers tends to make scapegoats of a few officers,

generate hostility from the rank and file, and ignore important

issues of management and supervision.

Early Intervention and Community Policing

EI systems are fully compatible with the goals of community

oriented policing (COP). In particular, the community policing

movement has created demands for new measures of police

performance.33 Because COP reorients the goals of policing, new

measures of are needed for both individual police officers and

entire departments. The traditional measures of the crime rate and

the clearance rate are no longer adequate.

Traditional police performance measures are inadequate in several

regards. They emphasize crime, to the neglect of quality-of-life

issues; they fail to take into account perceived community needs;

and they fail to reward adequately good police performance.34 By

systematically identifying and attempting to control inappropriate

behavior, an EI system can potentially reduce the number of

incidents that alienate communities from the police. 

One of the basic goals of COP is to establish closer ties to the

communities receiving police services. COP places particular

emphasis on being more responsive to community concerns about

the quality of police services, particularly with respect to racial

16

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 1
17

and ethnic minority communities concern about excessive force or

other forms of inappropriate behavior by officers. 

In this regard, Alpert and Moore recommend "the development of

statistical evidence on the use of force and the incidence of

brutality, discourtesy, and corruption..." This is precisely what an

EI system does. Indeed, Alpert and Moore's approach suggests

that a law enforcement agency's personnel data system should

transcend a narrow focus on suspected officers with performance

problems and include all current sworn officers.35

Early Intervention and Problem-Oriented Policing

EI systems are also consistent with the principles of problemoriented policing (POP). In this case, the problem is not graffiti or

public drunkenness, but those officers with performance

problems. As initially formulated by Herman Goldstein, POP

holds that law enforcement agencies should desegregate the

various aspects of their role and, instead of attempting to address

"crime" and "disorder" as global categories, should identify

particular problems within each category and develop narrowly

tailored responses appropriate to each. In addition, they should

develop the appropriate performance measures for each problem.36

The POP process of scanning, analysis, response, and assessment

(SARA) is, for all practical purposes, the way an EI system

operates:

• Scanning: review of data in the EI system database. 

• Analysis: identification and selection of potential officers with

performance problems. 

• Response: intervention with selected officers.

• Assessment: post-intervention review of officers' performance. 

Early Intervention and COMPSTAT 

EI systems are very similar to COMPSTAT programs in terms of

both purpose and process. Both rely on the analysis of systematic

data for the purpose of addressing problems and holding police

managers accountable for problems under their command.

Introduction to Early Intervention Systems

Chapter 1
An evaluation of the Pittsburgh PARS (its EI system) described

one key element as modeled after COMPSTAT. The Quarterly

COMPSTAT meetings in Pittsburgh involve a review of officers

who have been identified by PARS. Area commanders make

presentations about any officers under their command who have

been identified and conclude with a recommendation regarding

formal intervention. After a discussion, the chief of police makes

a final decision on what course of action to take.37

COMPSTAT is one of the most notable police innovations in

recent years, and has been widely adopted across the country. A

COMPSTAT program involves a computerized database of timely

and systematic data on criminal activity. Analysis of the data

provides a means of developing rapid and narrowly targeted

responses designed to reduce crime. Regular COMPSTAT

meetings are designed to hold police managers accountable by

requiring them to describe crime trends in their areas of

responsibility and to explain what responses they have developed. 

Early Intervention and Risk Management

EI systems can be a key component in a risk management system

(RMS).38 Risk management is a process for reducing an

organization's exposure to financial loss due to litigation. In law

enforcement, the major civil litigation costs arise from fatal

shootings, excessive physical force, and high-speed pursuit

incidents. An EI system database can readily identify patterns of

individual officers and situations that represent actual or potential

risks. It also provides a structured process for correcting those

problems and reducing the potential financial risk. In fact, the

expanded EI system in the Cincinnati Police Department, as

mandated by a memorandum of understanding with the Justice

Department, is called the Risk Management System.

Early Intervention and Police-Community

Relations

EI systems represent a potentially effective response to the

historic problems of officer misconduct and tensions between law

enforcement professionals and the communities they serve. For

18

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 1
19

many decades, alleged police officer misconduct, including

misuse of deadly force, use of excessive physical force, and

discourtesy, has been a major cause of tensions between the police

and racial and ethnic minority communities. Civil rights leaders

have alleged that minority citizens are not only the targets of

police misconduct at a rate disproportionate to their presence in

the population, but that police departments have failed to

investigate citizen complaints about misconduct and discipline

guilty officers.39 

To an unfortunate extent, police abuse of citizens has been

explained in global terms, typically labeling all police officers

with a broad brush. Abusive behavior for example, has been

attributed to a general police subculture, with the implicit

assumption that certain attitudes and behaviors are common to all

officers in all police departments.40 Other observers have

attributed abusive behavior to race, arguing that abuse reflects

racist attitudes and behavior on the part of white police officers

toward citizens of color.41 Some observers have attributed overly

aggressive or abusive behavior to gender, holding that reflects

male norms of behavior.42 Still other observers have attributed

police misconduct to organizational dysfunction, arguing that poor

leadership and low standards of professionalism have tolerated

many different forms of police officer misconduct.43

The great virtue of an EI system is that it can pinpoint specific

actions that are creating conflict with the community and the

particular officers responsible for them. For example, if there are

complaints in one neighborhood about frequent use of force, an EI

system can help to confirm or refute the allegations, and if

confirmed, identify the officers involved. At the same time, it can

help to identify those officers who are engaged in active police

work (e.g., a high volume of arrests) without resorting to

inappropriate behavior. 

Early Intervention and Racial Profiling

A comprehensive EI system has the potential for addressing the

issue of racial profiling. The Pittsburgh and Phoenix EI systems

collect data on traffic stops, including data on the race and gender

of drivers.

Introduction to Early Intervention Systems

Chapter 1
These data make it possible to identify officers who are stopping a

suspiciously high number of racial or ethnic minority drivers. As a

discussion paper by Samuel Walker argues, an EI system solves

many of the problems associated with interpreting traffic stop

data. Rather than use resident population data, as many reports do,

an EI system permits comparisons among officers working similar

assignments.44

In a predominantly Latino neighborhood, for example, it is to be

expected that a high percentage of all stops would involve Latino

drivers. The EI system database can help identify officers who are

stopping far more Latino drivers than their peers. The data alone

would not prove that racial profiling exists. Rather, it would be

the starting point for supervisory review that would determine

whether or not an officer's activities involve racial or ethnic bias.

The peer officer comparison approach is discussed in Chapter

Two of this report. It is the methodology currently used in the

Pittsburgh PARS.45

In one department, the EI system identified an officer who had a

practice of stopping female drivers for questionable purposes. The

department received a formal citizen complaint from one female

driver who alleged that the officer made improper sexual

advances during a traffic stop. A review of the EI database

revealed a very high rate of traffic stops of female drivers.

Combined with the original complaint, these data were sufficient

to cause a formal departmental intervention.

Using peer officers as the comparison group solves the most

vexing problem with regard to the analysis of traffic stop data.

Most current data collection efforts use the racial and ethnic

composition of the resident population as the baseline or

denominator.

One Size Does Not Fit All Departments

There is no single model for EI systems. The outline of the basic

components of a system described above represents a single

framework. Each department can develop an EI system

20

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 1
21

appropriate to its needs. These needs will vary significantly

according to the size of the department.46 

Medium-sized and small agencies do not need as complex and

sophisticated a system as do large departments. Most of the

available information about EI systems at present, however, is

derived from large law enforcement agencies. Consequently, at

this time it is not possible to specify the exact needs of mediumsized and small departments. Developing those needs is on the list

or priorities for future research and program development (See

Chapter Five).

A Work in Progress

Although EI systems are increasingly popular and have been

recommended by a wide range of professional groups, they are far

more complex than is generally realized. They require making a

number of difficult choices regarding their design, and once

operational require close, ongoing administrative attention. 

EI systems are in a state of continuing development. There is no

solid consensus regarding a single best way to operate a system.

Local police departments have found that they need to continually

revise and fine-tune their EI systems. As one police manager

explains, "this is a new practice that may, and has, changed over

time. It continues to evolve." Another explains that an EI system

"needs to continuously evolve and expand" (See Chapter Four).

Additionally, our understanding of EI systems is experiencing a

paradigm shift. First, they are no longer seen only as a means of

identifying officers with performance problems, but instead are

recognized as an effective data-based tool for general personnel

assessment. Second, instead of focusing exclusively on rank and

file officers they are seen as a tool for promoting the

accountability of supervisors.

Introduction to Early Intervention Systems

Chapter 1
Conclusion

Early Intervention systems are an important new tool for police

accountability. They are data-based programs for identifying

officers with performance problems and providing the basis for

departmental intervention to correct those problems.

EI systems have emerged as a recommended best practice in

police accountability. They have been recommended by a wide

range of professional associations. In addition, EI systems are

consistent with the goals of both community policing and

problem-oriented policing.

Although they are growing in popularity, EI systems are still in

the early stages of development. Departments are still wrestling

with a number of difficult and important issues. These issues are

addressed in the remaining chapters of this report.

22

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 1
Chapter Two

The Components of an Early

Intervention System

The Components............................................................................................25

Size and Scope: Two Approaches to EI Systems .......................................................25 

Performance Indicators....................................................................................26

The Number of Indicators.................................................................................26

Department-Specific Indicators ..........................................................................29

Identification and Selection Process ....................................................................30

Identifying Problem Performance: Three Examples..................................................30

The Proper Thresholds ....................................................................................31

Mandatory vs. Discretionary Referral to Intervention ...............................................33

Feedback to Officers.......................................................................................35

Intervention .................................................................................................35

Action Alternatives.........................................................................................35

Training Supervisors for Intervention...................................................................37

Holding Supervisors Accountable ........................................................................38

Post-Intervention Monitoring.............................................................................39

An EI System Appropriate for Each Department ......................................................40

The Need for Careful Planning...........................................................................41

Conclusion ...................................................................................................41

Size and Scope: Two Approaches to EI Systems 

While all EI systems will have the same basic components, the

size and scope of different systems will vary according to their

goals.48 Two basic types of EI systems currently exist,

representing differently sized systems with different goals. Some

systems, such as the Pittsburgh PARS and the Phoenix PAS, are

comprehensive personnel assessment systems. They collect a very

wide range of data and are able to conduct complex data analyses

that can include identifying top-performing and underperforming

officers as well as officers with performance problems. Systems

of this sort not only require a sophisticated technological

infrastructure but also an enormous amount of administrative

commitment.49

25

The Components

An Early Intervention system consists of four basic components: performance

indicators, identification and selection process, intervention, and postintervention monitoring. An earlier report identified three components of EI

systems.47 This report, reflecting the maturation of EI systems, utilizes four

components, because the identification and selection process is a separate

component from the list of performance indicators.

The Components of an Early Intervention System

2

• Performance Indicators

• Identification and Selection

• Intervention

• Post-Intervention Monitoring

BOX 2.1 Components of an EI System
Many other systems, such as the Miami-Dade and the Tampa,

Florida systems, are more narrowly focused and may be described

as performance problem systems. They collect a smaller range of

data and focus on a more limited range of performance issues. As

a result, they are also less expensive and require a smaller

commitment of administrative resources.

Neither system is necessarily better than the other. They simply

have different goals and different approaches to police

accountability. The goals of the more comprehensive systems are

more ambitious, but they are also more demanding of the

department. An evaluation of Pittsburgh's PARS, for example,

pointed out that its comprehensive approach comes at a price. By

imposing greater administrative responsibilities on sergeants for

monitoring officer behavior, it necessarily takes time away from

traditional field supervision and other duties.50

Performance Indicators

The performance indicators involve those aspects of officer

performance that are the subject of official departmental reports

and are entered into an EI system database. Almost all experts

argue that an EI system should utilize a broad range of

performance indicators and not rely on just one indicator as many

of the first EI systems did (and some still do). Relying on a single

indicator such as citizen complaints is not recommended. There

are a number of problems related to official data on citizen

complaints and they are not consistently reliable indicators of

officer performance.51 Using a broader range of indicators is more

likely to identify officers whose behavior requires departmental

intervention.

The Number of Indicators

There is no consensus of opinion regarding the exact number of

performance indicators to use. The Pittsburgh PARS uses 18

performance indicators (although the department is considering

adding five more). The Phoenix PAS utilizes 24 indicators. The

consent decree between the U.S. Department of Justice and the

Los Angeles Police Department specifies 17 categories of data to

26

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 2
27

be entered into the department's TEAMS II (See Appendix A).52

The Memorandum of Understanding involving the Cincinnati

Police Department requires the collection of ten performance

indicators. 

Box 2.2 represents a list of performance indicators in a model

comprehensive personnel assessment system. Box 2.3,

meanwhile, lists the performance indicators in a model problem

officer system.

The Components of an Early Intervention System

Chapter 2

Comprehensive Personnel Assessment Model

1. All non-lethal uses of force

Citizen injuries requiring medical attention 

Disciplinary outcomes

2. All officer-involved shooting incidents

On-duty/off-duty discharges, citizen injuries, citizen fatalities

Disciplinary outcomes

3. All officer-involved vehicular pursuits

Accidents

Injuries

Fatalities

Disciplinary outcomes

4. All citizen-initiated complaints

Use of force

Racial bias

Investigative outcomes

5. All citizen-initiated commendations or compliments

6. All departmental commendations and awards

7. Criminal arrests and investigations of subject officer

BOX 2.2 Performance Indicators
28

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 2

8. Civil suits or an administrative claim in which subject 

officer is named

Nature of claims

Case outcomes

9. All arrest reports, crime reports, and citations made by

officers

10. All motor vehicle stops and pedestrian stops made by

subject officer

Race, ethnicity, gender, age of drivers stopped

11. Performance evaluations for each officer

12. Training history 

Any failure of an officer to meet weapons qualification

requirements

13. All management and supervisory actions, including

non-disciplinary actions, related to each officer

14. Sick leave or family leave record

15. Canine unit deployment involving subject officer

Deployments

Bites

Citizen injuries

Complaints or civil suits arising from deployments

16. Failure to appear in court

BOX 2.2 Performance Indicators (Continued from page 27)
29

The choice of the number of indicators to use involves a trade-off.

The greater the number of indicators used, the greater the capacity

of the system to evaluate the full range of officer performance.

The more comprehensive Pittsburgh PARS, for example, allows

commanders to identify top-performers and under-performers, as

well as officers with performance problems. Because the system

includes arrest data, commanders can identify officers who have

several use-of-force reports but who are also making a very large

number of arrests and compare them with officers who have

slightly fewer complaints but who make very few arrests.

At the same time, however, utilizing a greater number of

indicators imposes greater administrative demands in terms of

data entry and data analysis.53 A smaller problem officer system,

with fewer performance indicators, will be much less expensive

and easier to create and maintain.

Department-Specific Indicators

Some performance indicators have been adopted because of

specific controversies surrounding a particular department. The

memorandum of understanding between the U.S. Department of

Justice and the City of Cincinnati requires that the Risk

Management System include drawing and pointing a weapon at a

citizen. This indicator is included because there had been many

complaints from citizens about such action by police officers.54

The Components of an Early Intervention System

Chapter 2

1. All officer-involved use-of-force incidents

2. All officer-involved shooting incidents

3. All citizen-initiated complaints

4. All officer-involved civil suits or administrative claims

5. All departmental disciplinary actions

6. All citizen-initiated or department-initiated commendations or

awards.

BOX 2.3 Early Intervention System Focused Performance Model
The Phoenix PAS includes High Stress Radio Calls (calls

involving actual or intended suicides, or child abuse, molestation,

or death) as a performance indicator. This indicator is being

included because of three incidents involving suicide or attempted

suicide by officers following a series of calls to incidents of this

sort. In one case, an officer mentioned to his supervisor that he

had seen enough of these calls, and expressed suicidal thoughts.55

Individual departments may have other special problems they are

aware of and want to incorporate into their EI systems.

Identification and Selection Process

Given the existence of a performance database, a strategy is

needed to identify officers for performance review and to select

officers for referral to intervention. These are separate stages. In

many EI systems an officer who is identified will automatically

be referred to intervention. Others, however, utilize a two-stage

process through which officers who are identified are subject to a

full-scale performance review with the result that only some are

referred to formal intervention. 

Several participants at the Early Intervention State of the Art

Conference56 vigorously emphasized the point that a computerized

database only gives you numbers. It does not make decisions

about what to do with identified officers; only supervisors can do

that. These statements only highlight the point that there is no

magic in a sophisticated computerized system. To be effective, an

EI system requires responsible management by the command

staff. 

Identifying Performance Problems: Three Examples

The following examples are based on actual cases adapted from

the EI system of one large police department.57 

Officer A

The Indicators: Officer A had five use-of-force reports during one

reporting period. Relative to other officers in the same unit, this

30

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 2
31

was a very low figure. But, he made only eight arrests during this

period. 

The Analysis: Officer A used force in more than half of the arrests

he made. The ratio of force to arrests is cause for departmental

concern and probable intervention.

Officer B

The Indicators: A female driver filed a citizen complaint against

Officer B alleging an inappropriate sexual advance during a traffic

stop. An examination of Officer B's EI file indicated a very high

number of traffic stops involving females.

The Analysis: Officer B appears to be abusing his law

enforcement powers to harass female drivers.

Officer C

The Indicators: Officer C had no citizen complaints or use-offorce reports for the reporting period. He also had made no

arrests, no traffic stops, and no field stops. An examination of his

EI file indicated that he was working the maximum number of

permitted hours of off-duty employment.

The Analysis: Officer C. was devoting all his energy to his offduty employment rather than fulfilling his responsibilities to the

department.

The Proper Thresholds

The central issues with regard to identifying and selecting officers

for intervention the question of the proper performance thresholds

to be used. (Some systems use the term "triggers" but that word

has inappropriate connotations and is not recommended.) The

issue can be framed in the following terms:

** How many use-of-force incidents over what time period

should qualify an officer for intervention? 

The Components of an Early Intervention System

Chapter 2
** What combination of use-of-force reports, citizen complaints,

and other indicators over what period of time should qualify

an officer for intervention? 

At present, there is no consensus regarding the best set of

thresholds. Three different formulas are currently being used:

department-level thresholds, peer officer thresholds, and ratiobased formulas.

1. Department-Level Thresholds

In the Miami-Dade EIS, an officer is identified if in any quarter

he or she has two or more citizen complaints or three or more

use-of-force reports in the previous 90 days (or quarter). A list of

the officers so identified is circulated to commanders who then

must take one of four actions: refer the officer to the

Psychological Services Bureau for counseling, refer the officer to

the department Stress Abatement Program, refer the officer to

some other form of corrective action, or conclude and document

that no action is required. 

The Miami-Dade EIS also operates at an earlier and less formal

manner than the formal process described above. Monthly reports

are circulated listing officers who have two or more citizen

complaints or two or more use-of-force reports (or both).

Commanders are not required to take any formal action, but are

expected to review an officer's performance and perhaps conduct

an informal meeting with him or her.

2. Peer Officer Averages

The Pittsburgh PARS utilizes an approach that compares officers

performances with peer officers, meaning officers with the same

assignment (e.g., night shift, high-crime area, daytime traffic unit,

etc.). Some experts have described this approach as comparing

apples with apples. Officers assigned to high-crime areas are more

likely to generate use-of-force reports and citizen complaints

because of the nature of the work environment. It is unfair to

expect them to meet the same numerical standard as officers

working low crime areas.58 

32

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 2
33

The Memorandum of Understanding mandating the EI system in

the Cincinnati Police Department (officially called the Risk

Management System) also requires a peer officer formula. The

RMS is required to report data on the average level of activity for

each data category by individual officer and by all officers in a

unit (See Appendix D). Finally, the Phoenix PAS is developing a

peer-officer-based formula.

3. Performance Indicator Ratios

A third approach to thresholds analyzes performance data in terms

of the ratios between different performance indicators: e.g., the

ratio of use-of-force reports to arrests. In one example, (See Box

2.4, Officer A), an officer did not use force very often, but used

force in more than half of all arrest situations. One of the virtues

of this approach is that it allows commanders to identify topperforming officers (e.g., high arrest activity, low arrest-tocomplaint ratio) and not unfairly subject them to intervention. The

Phoenix PAS has been testing this approach prior to

implementation.59

Mandatory vs. Discretionary Referral to

Intervention

Some systems use a mandatory approach for referral to

intervention while others use a two-stage discretionary approach. 

In the initial Minneapolis Police Department Early Warning

System, an officer was automatically referred to intervention if he

or she received three complaints in a twelve-month period.60 Not

only did this system (which has since been replaced by a more

sophisticated performance review system) rely only on citizen

complaints but it did not take into account the nature and context

of the complaints.

The Miami-Dade EIS generates monthly reports that list officers

who have received two or more citizen complaints or have made

two or more use-of-force reports (or both) in the last 60 days.

This list is for informational purposes only, and supervisors are

not required to take any action. It is understood, however, that

The Components of an Early Intervention System

Chapter 2
supervisors will review any listed officer's performance and

possibly speak to individual officers. Quarterly reports list officers

who have two or more personnel (e.g., citizen) complaints and

involvement in three or more use of force incidents (or both)

within the previous 90 days. Supervisors must take action with

regard to these officers. The alternatives include referring the

officer to the stress abatement program or to the Psychological

Services Program for counseling, or requiring some other

corrective action, or concluding (and documenting) that no action

is required. In short, commanders are provided information about

officer performance but are left the discretion to decide whether

any formal intervention is required, and if so what action is

appropriate.

The Los Angeles Sheriff's Department's PPI system utilizes a

discretionary two-stage approach. Only some of those officers

initially identified by the system are placed on Performance

Review. Between 1996 and 2002, a total of 1,213 employees were

identified, but only 235 (or 19 percent) were placed on

Performance Review.61 The Performance Review Committee

screens officers twice. First, the committee solicits the views of

an officer's captain about whether placement on Performance

Review is appropriate. A second screening, taking into account

the captains' views, is conducted to select officers for a detailed

Employee Profile Report (EPR) (known informally as a "Blue

Book").62 The EPRs are prepared by sergeants assigned to the

department's Risk Management Bureau.

The performance review is designed to place an officer's

performance in the context of his or her assignment and

performance history. It considers evidence that might exculpate an

officer along with evidence indicating a problem that is believed

to be worse than suggested by the performance data. For example,

an officer with a number of use-of-force reports might be excused

because of particularly intensive law enforcement activity during

the period under review and an otherwise exemplary record. At

the same time, another officer might have in addition to a number

of complaints and use-of-force reports a reputation for a bad

temper and inability to work well with colleagues. In the end,

some of the identified officers are selected for intervention.

34

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 2
35

Pittsburgh's PARS also generates information about officer

performance for the benefit of supervisors but leaves them the

discretion to decide what action to take. An evaluation of the

PARS found that formal intervention is a rare outcome for officers

initially identified by the system. The most common response

involves monitoring of the officer through supervisory field

checks (also referred to as "roll-bys").63 Part of the reason for the

few formal interventions is that, unlike other EI systems, the

PARS has a formal process for pushing responsibility for

monitoring officer behavior down to first line supervisors. As

explained elsewhere in this chapter, the PARS requires sergeants

to log on to the system every day, a process that forces daily

consideration of performance data.

Feedback to Officers

One issue that has been discussed among experienced EI system

commanders is the nature of feedback to officers who are

identified but not selected. There may be some value in informing

such officers that their performance was identified by the EI

system but that upon review, it was determined that there was no

problem requiring formal intervention. This situation is likely to

arise with high-performing/low-ratio officers. Informing them of

the decision not to refer them to intervention may be a valuable

way of communicating departmental approval and support for

their performance.

Intervention

The intervention phase consists of some form of counseling or

retraining for officers who have been selected. In some EI

systems the counseling is done by an officer's immediate

supervisor. In other departments, the counseling involves other

command officers.

Action Alternatives

The Commission on Accreditation for Law Enforcement Agencies

(CALEA) Standards state that a departmental Personnel Early

Warning System should have a menu of remedial actions for

The Components of an Early Intervention System

Chapter 2
officers selected for intervention.64 In the Miami-Dade Police

Department, the action alternatives include referral to the

departmental Psychological Services Program or participation in

the Stress Abatement Program. Another alternative is a

determination that no problem exists and that no formal action is

necessary.

One of the virtues of the data-driven nature of EI systems is that

the performance data provide the basis for discussing specific

aspects of an officer's performance. Chapter Four of this report

includes comments by police managers explaining how their EI

systems strengthen the hand of supervisors in dealing with

officers with performance problems. Where citizen complaints

about rudeness are frequent, referral to Verbal Judo might be the

proper response. Where problems in handling arrest situations are

the problem, specific retraining can be offered. Depending upon a

department's existing personnel procedures, the counseling or

retraining can be recommended or mandated.

In the Los Angeles Sheriff's Departments PPI system, the

Performance Review Committee develops a formal performance

plan for each subject officer.65

Intervention Group Training Classes

Some EI systems conduct intervention through classes for groups

of selected officers.66 There are a number of problems with this

approach, however. First, the subjects covered in a class are

general in nature and do not necessarily address the particular

problems of individual officers. Second, they are difficult to

schedule and considerable time may pass between the time an

officer's problems are identified and he or she is enrolled in the

intervention class. Third, there is evidence that bringing a group

of selected officers together creates a dynamic of group solidarity

that reinforces inappropriate attitudes. Such classes have been

referred to as "bad boys" classes.67

36

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 2
37

Training Supervisors for Intervention

The most important issue identified at the Early Intervention State

of the Art Conference involved preparing supervisors to conduct

interventions.68 Lack of adequate orientation and training was also

one of the most serious problems reported in the police managers

survey (See Chapter Four). One explained that as a result of

inadequate training, the system is misunderstood by the first line

supervisors, and therefore retraining is needed at the application

level. EI systems represent a new role for supervisors, especially

sergeants, in which they are expected to coach and help officers

under their command. The traditional role of sergeants has

involved formal disciplinary actions, and training has emphasized

the procedural aspects of the process. 

In an EI system it is essential that first-line supervisors actually

conduct intervention sessions, and do so in a manner consistent

with the system's goals. It is possible for a particular sergeant to

simply tell an officer not to worry about it. This would undermine

the EI system and breed cynicism among officers. 

A related problem is the potential for favoritism, in which one

supervisor excuses the performance of one officer under his or her

command while taking a tough stand on another officer with

similar performance problems.

A final problem is the possibility of inconsistent interventions

across a group of supervisors. Since interventions are designed to

be confidential, it is not possible to require documentation of the

actual substance of intervention sessions.

It is essential that departments provide proper training for

supervisors regarding their role in interventions and that this be

done well before an EI system become operational. Several steps

are possible to ensure that interventions are delivered as desired.

First, in some departments the intervention is conducted in a

meeting with more than one supervisor. This helps to ensure that

one supervisor will not inappropriately excuse the performance of

an officer. At least one system in operation is programmed to

require supervisors to notify their own supervisors that the

The Components of an Early Intervention System

Chapter 2
intervention has occurred. Failure to do so will result in an

automatic email notice to the higher-ranking supervisor. This

procedure, of course, does not guarantee anything about the

content of the session. 

Holding Supervisors Accountable

Some EI systems have begun to address the issue of inadequate

first-line supervisors. If one or more officers in a unit are

identified by an EI system, part of the problem may be inadequate

supervision. 

The San Jose Police Department has developed a Supervisors

Intervention Program (SIP). A supervisor is identified by the

system whenever officers under his or her command receive a

combined total of three or more complaints in any six-month

period. The supervisor is then required to meet with his or her

immediate supervisor, the head of the Professional Standards and

Conduct Unit (PSCO), and an assistant chief. One limitation of

the system is that it relies on only one performance indicator,

citizen complaints.69 The San Jose Police Department is perhaps

the first to institute a formal process for using the EI system to

evaluate the performance of supervisors. More discussion and

program development on this issue is warranted in the immediate

future.

The Pittsburgh PARS takes a different approach, integrating the

EI system into routine supervision. Sergeants are required to

access the PARS database every day and to review the

performance of all officers under their command. This approach

socializes them into use of the system and the habit of data-driven

supervision. Sergeants are able to intervene with an officer even

38

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 2

• Referral to department employee assistance program

• Referral to professional family counseling service

• Referral to credit counseling services

• Retraining on traffic stop tactics

• Verbal judo training

BOX 2.4 Intervention Action Alternatives
39

before that officer crosses the system thresholds. Through this

process, supervisors are expected to regard performance analysis

and intervention as regular parts of their jobs and not some

special and onerous task reserved only for serious problems.70

The Pittsburgh PARS also attempts to ensure that interventions

are conducted through a process of computerized notifications.

When an officer is identified by the system, his or her sergeant is

notified by email that an informal counseling session is required.

The sergeant must then report by reply email when the counseling

session has been completed. Failure to provide this notification

automatically generates an email notice to that sergeant's

lieutenant. Failure of the lieutenant to take the proper action

generates an email notice to the next higher commander.

Post-Intervention Monitoring

Following an official intervention, supervisors need to follow-up

by monitoring the performance of an officer for a specified period

of time. At least three different approaches to follow-up exist in

current EI systems. These approaches vary by their degree of

formality and record-keeping.

In many departments, the monitoring is informal and conducted

by immediate supervisors. Some departments keep officers on a

list for a specified time period following intervention and

continue to monitor their performance. In the Pittsburgh PARS,

sergeants are required to do "roll-bys" of officers who have been

identified by the system. A roll-by is a direct observation of an

officer at work. In effect, this is a form of intensive supervision.

An EI system such as Pittsburgh's has the effect of structuring and

systematizing such intensive supervision.

In the Los Angeles Sheriff's Department, an officer remains on

Performance Review for two years after initial identification. An

officer may be removed only upon recommendation of his or her

captain and approval by the Performance Review Committee. In

other departments, however, the monitoring involves an elaborate

system of observation, evaluation, and reporting. The New

Orleans PPEP involves a six-month follow-up in which

The Components of an Early Intervention System

Chapter 2
supervisors are required to observe subject officers while they are

on duty and to file a signed evaluation of officers' performance

every two weeks. The resulting documentation ensures that

monitoring occurs.71

The post-intervention follow-up component of EI systems has

received the least attention by officials involved in developing

and managing EI systems. It is an issue that needs additional

attention in the immediate future.

In determining the exact nature of the post-intervention follow-up,

agencies also encounter the problem of balancing the demands of

accountability with the need for efficiency. As we indicated in our

discussion of this issue related to the performance database, there

is no clear answer to the problem at present. It needs to be the

subject of continued discussion and experimentation within the

law enforcement profession.

An EI System Appropriate for Each Department

The range of requirements EI can address prohibits the

development of any single definitive model of an EI system. The

abovementioned basic system components can provide only a

general framework. The most effective EI systems are developed

specifically to address the needs of the personnel they serve, and

those needs are as unique as the agencies that produce them. 

One of the most influential variables in EI system development is

the size of the agency it will serve.72

Small- to mid-sized agencies do not require the same levels of

complexity and sophistication as their larger peers. Large agencies

not only have much different problem sets, however, they also

have significantly different levels of resources available for EI

system development. Due in large part to this discrepancy in

resources, the vast majority of information currently available

about EI system development and implementation is derived from

work with larger law enforcement agencies. 

40

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 2
41

This makes for an unfortunate scarcity of information regarding

the requirements for effective EI systems in smaller and mid-sized

agencies. Developing this information is high on the list for future

research and program development (See Chapter Six).

The Need for Careful Planning

It is the consensus of opinions among police commanders with

extensive EI system experience that careful planning is necessary

for the effective development and implementation of an EI

system. One of the main points of this report is that EI systems

are extremely complex administrative tools, with several critical

components and several different goals. Several of the established

EI systems have gone through significant revision since they were

first implemented. The Phoenix Police Department took over two

years to develop and implement its PAS. Other experts believe

this was a wise decision. One important aspect of an effective

planning process is to involve officers from all ranks in the

process. This helps to ensure that officers fully understand and are

committed to the goals and procedures of the EI system.

Conclusion

EI systems are extremely complex administrative tools. They

consist of four basic components, each of which requires

decisions about the exact design. Careful planning and

consideration of these decisions is necessary for an EI system to

be implemented effectively. A planning process is described in

Chapter Five.

The Components of an Early Intervention System

Chapter 2

Chapter Three

Background: The Origins and Development of

Early Intervention Systems

Evidence About Officers With Chronic Performance Problems .....................................45

The Early History of EI Systems..........................................................................48

The First Permanent Systems: Miami and Miami Dade ..............................................50

An Evolving Concept .......................................................................................53

An Emerging Best Practice................................................................................54

Implementing EI Systems: A Major Challenge .........................................................55

The Los Angeles Sheriff’s Department PPI System....................................................55

The Los Angeles Police Department.....................................................................57

Pittsburgh Police Department............................................................................58

Research on the Effectiveness of EI Systems ..........................................................59

(1) The NIJ Study of Early Warning Systems...........................................................60

(2) Evaluation of Pittsburgh PARS .......................................................................67

(3) Assessment of the LASD PPI System.................................................................68

(4) Anecdotal Evidence From Other Departments ....................................................69

(5) The Police Managers’ Survey.........................................................................69

Conclusion ...................................................................................................70

Evidence About Officers With Chronic

Performance Problems

Early Intervention (EI) systems originated out of a growing

recognition that a few officers are responsible for a

disproportionate number of any police department's problems

related to use-of-force and citizen complaints. The existence of

the so-called "problem officer" was known informally for a long

time. The evidence was largely anecdotal, however, and

departments did not take any official action to address the

problem. Police chiefs increasingly commented that "90 percent

of our problems are caused by ten percent of our officers."73

Professor Herman Goldstein, one of the leading experts on

policing and the founder of problem-oriented policing, was the

45

T

he purpose of this chapter is to describe the origins and development

of EI systems. The first section covers the basic rationale for EI

systems: the argument that a small number of police officers are

responsible for a disproportionate number of problematic incidents

such as citizen complaints, use-of-force incidents, and civil suits against law

enforcement agencies. The second section provides a brief overview of the

history of EI systems. The final section examines the development of EI

systems in a few selected jurisdictions for the purpose of illustrating the

administrative difficulties some of them have had in implementing their EI

systems. The final section summarizes the findings of a National Institute of

Justice-sponsored evaluation of the EW systems in three major police

departments.

Background: The Origins and Development

of Early Intervention Systems

3
first expert to discuss the possibility of "Identifying Officers with

a Propensity for Wrongdoing," and then doing something to

improve their performance.74 He cited an experimental (but shortlived) program by Hans Toch in the 1970s in which peer officers

counseled Oakland, California, police officers with records of

use-of-force incidents.75

A 1981 report by the U.S. Civil Rights Commission, Who is

Guarding the Guardians?, provided the first data documenting the

phenomenon of a few officers receiving a disproportionate

number of all citizen complaints. It published a table indicating

that officers who had received five or more complaints over a

two-year period represented only 12 percent of all officers

receiving complaints but accounted for 41 percent of all

complaints (Table 3-1). One officer had 12 complaints while two

others had 11 complaints during this two-year period. In response,

the Commission recommended that "A system should be devised

in each department to assist officials in early identification of

violence-prone officers."76

46

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3

Source: U.S. Commission on Civil Rights, Who is Guarding the

Guardians? A Report on Police Practices (Washington, DC: Government

Printing Office, 1981), p. 166.

• 2 complaints 298 officers

• 3 complaints 136 officers

• 4 complaints 70 officers

• 5 complaints 33 officers

• 6 complaints 18 officers

• 7 complaints 8 officers

• 8 complaints 6 officers

• 9 complaints 1 officers

• 10 complaints 0 officers

• 11 complaints 2 officers

• 12 complaints 1 officers

Number of Complaints per Officer

BOX 3.1 Houston Complaint Data
47

The aftermath of the 1991 Rodney King beating heightened

national awareness of the phenomenon of the problem officer. 

The Christopher Commission, appointed to investigate the Los

Angeles Police Department following the King incident,

identified 44 problem officers in the Los Angeles Police

Department (LAPD) with extremely high rates of complaints.77

In a particularly notable observation, the Christopher Commission

pointed out that these 44 officers were "readily identifiable" on

the basis of existing departmental records. Yet, the LAPD

appeared to have made no effective use of these records to

identify the officers and take any corrective action. The

Commission found, for example, that citizen complaint data were

not used in making routine performance appraisals.78

A year after the Christopher Commission report, the Kolts

Commission investigated the Los Angeles County Sheriff's

Department (LASD), and using a similar methodology identified a

group of 62 deputies with significant performance problems in the

department. These 62 deputies were responsible for almost 500

use-of-force/harassment complaint investigations. Seventeen of

these deputies were responsible for 22 civil lawsuits that resulted

in damage awards or settlements of about $3.2 million against the

County. The Kolts Commission concluded that the LASD had

"failed to deal with officers who have readily identifiable patterns

of excessive force incidents on their records." 

Not only were nearly all of the 62 deputies still on patrol duty at

the time of the Kolts report, but–even more alarmingly–many

served as field training officers (FTOs). The Kolts Commission

warned that in that capacity they were "imparting their 'street

wisdom' to patrol deputies."79

The report recommended the creation of an EI system within the

LASD to identify these deputies. The Kolts report staff "were

struck that the LASD conducted no regular or meaningful analysis

of lawsuits and administrative investigations" that might help to

avert further misconduct and save the county millions of dollars.

It concluded that "the concept of 'risk management' was largely

unknown and unpracticed in the LASD."80

Background: Origins and Development of Early Intervention Systems

Chapter 3
Meanwhile, the Christopher Commission report spurred

journalistic investigations of the problem officer phenomenon in

other cities. The New York Times reported that in Kansas City,

two percent of the sworn officers were responsible for 50 percent

of all citizen complaints.81 A Boston Globe investigation found

that 11 percent of Boston Police Department officers were

responsible for 61.5 percent of all complaints.82 And in

Washington, D.C., investigative journalists identified certain

officers with long records of complaints and other problematic

behavior.83

The first formal evaluation of EI systems confirmed the anecdotal

evidence regarding problem officers. Sponsored by the National

Institute of Justice (NIJ) (and directed by the author of this

report), this study evaluated the EI systems in the Miami-Dade,

Minneapolis, and New Orleans police departments. It found that

officers who had been selected for EI system intervention had

substantially more serious performance records than did peer

officers in their original recruit classes. Officers selected by the

Miami-Dade Employee Information System (EIS), for example,

averaged twice as many use-of-force incidents per year as their

peers and were twice as likely to have been reprimanded and

three times as likely to have been suspended.84 A full discussion of

the findings of this study appears at the end of this chapter.

The Early History of EI Systems

Initial Experiments

EI systems began to emerge in the 1970s as a crisis management

response to public concern over police abuse of force. The

histories of most of these programs are lost and may not be

recoverable.85 In its 1981 report, the U.S. Civil Rights

Commission cited with favor the EI systems in Oakland, New

York City, and Kansas City.86 Little is known about those systems

except for the fact that none appear to have survived very long.

The Mollen Commission report on corruption and violence in the

New York City Police Department in 1984, for example, offers no

evidence of a functioning EI system.87

48

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
49

Because of a Police Foundation evaluation, information about the

original Kansas City program is available. The initial Kansas City

early warning system was authorized in early 1972 and began

operating in November, 1973, as part of an experimental Peer

Review Panel Program.88 The Panel was part of the same effort to

stimulate innovation in the Kansas City Police Department that

included the well-known Kansas City Preventive Patrol

Experiment.89 The Panel had the general goal of reducing tensions

between the KCPD and the public and, to that end, sought to

effect change in police officer attitudes and behavior through peer

support and pressure. The program was designed to "identify

those patrol officers with high frequencies of negative encounters

with citizens." It used "no single fixed formula" for identifying

officers with performance problems but utilized a combination of

four separate criteria. These included: (1) three or more "negative

encounters" with citizens, defined as both official citizen

complaints and "interfering with an officer" charges; (2) referral

by an immediate supervisor or other high-ranking department

official; (3) an officer's "voluntary request for appearance;" (4)

involvement in an incident of "public or departmental notoriety."90

The Police Foundation evaluation found that the Peer Review

Panels had no significant effect on reducing either citizen

complaints or interfering with officer charges among those

officers who were subject to the program. The ultimate fate of the

Kansas City Peer Review Panel is not known. Given the fact that

it was an experimental program, funded through a grant from the

Police Foundation ($73,000 in 1973 dollars), and had no positive

impact, it appears that the program lapsed.91 A 1991 New York

Times article on the Kansas City EWS portrayed it as a new

program, with no reference to the one cited by the Civil Rights

Commission.92

In Oakland, an experiment in utilizing police officers as change

agents within their own department included one component

designed to reduce citizen-officer conflict among officers with

performance histories of frequent conflict. These officers were

counseled by peer officers with exemplary performance records.

The experiment concluded with some modestly optimistic

Background: Origins and Development of Early Intervention Systems

Chapter 3
conclusions about the effectiveness of peer counseling to deal

with officers with performance problems.93 It appears, however,

that once the formal experiment ended, the peer-counseling

program ended as well.

The 1981 report of the U. S. Commission on Civil Rights, Who is

Guarding the Guardians?,

94 which was based on extensive

hearings on police misconduct in three major cities, took note of

these early EW systems and recommended that all departments

establish similar systems. Specifically, the Commission concluded

that:

The careful maintenance of records based on written

complaints is essential to indicate officers who are frequently

the subject of complaints or who demonstrate identifiable

patterns of inappropriate behavior. Some jurisdictions have

"early warning" information systems for monitoring officers'

involvement in violent confrontations.95

The First Permanent Systems: Miami and MiamiDade 

Two early EW systems that have had continuous operation from

their creation to the present day, and about which detailed

information exists, were developed in the City of Miami Police

Department and the Miami-Dade (formerly Metro-Dade) Police

Department in the late 1970s and early 1980s. The histories of

these two systems reflect the process of development that has

occurred in most of the early systems: the intersection of growing

public pressure to curb misconduct and improve policecommunity relations and creative leadership by a few key

individuals inside the department, operating without the aid of any

external models.

City of Miami

In 1979, the City of Miami Police Department became concerned

with its officers' behavior that generated citizen complaints in the

late 1970s. In a May 29, 1979 memorandum to the chief, the

commander of the Internal Security Unit suggested an early

warning system based on concepts of organizational development.

50

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
51

This involved a "cyclical model where the problem is diagnosed,

external professionals are consulted, strategies are developed,

programs are implemented and evaluated, and results are fed back

to begin the cycle again."96

To illustrate his idea, the commander identified a list of officers,

by assignment, who had two or more citizen complaints during a

two-year period (1976-1978). He also compiled a list of officers

who had received five or more civilian complaints during that

period. Armed with these data and the Internal Security Monthly

Activity Reports, the commander computed statistics indicating

that five percent of all officers on the force accounted for 25

percent of all citizen complaints. He noted that "if this group were

suddenly removed from our department, our complaint picture

could be reduced by as much as one-fourth. Obviously, this group

should warrant some special attention, if we are to reduce our

complaint incidence."97 The data also indicated that the small

groups of officers with the most complaints were also more likely

to receive complaints regarding excessive force.

The result of this analysis was a proposal that the Miami Police

Department should systematically provide supervisors with

information "that can be used to identify problem officers."98 The

IA commander also noted that off-duty employment, including

rock concerts, wrestling matches, and football games generated a

high number of citizen complaints. He reasoned that fatigue may

"heighten an officer's opportunity to react in an aggressive

manner."99 He suggested that the department should respond to

these officers before they become involved in self-destructive

activities or develop a trend of violating departmental orders. His

proposal included more intensive supervision, counseling by

outside professionals and training in tactics and strategies:100

The Miami EW system eventually developed selection criteria

including four categories of behavior: (1) citizen complaints, (2)

control of persons (i.e., use-of-force) incidents, (3) official

reprimands, and (4) firearms discharges.101 Once an officer is

identified by the system, his or her supervisor is notified and is

expected to meet with the officer and to determine if he or she

needs any assistance, such as counseling, training, or other

Background: Origins and Development of Early Intervention Systems

Chapter 3
intervention. Officers identified by the EW system are then

closely monitored. Supervisors are expected to investigate any

problem incident and follow-up with a memorandum containing a

recommendation. Internal Affairs provides the supervisor with a

report of each incident that must be reviewed as well as the

officer's assignment when the incident occurred. After evaluating

the reports, the supervisor makes a recommendation that may

include 1. Reassignment, 2. Retraining, 3. Transfer, 4. Referral to

an Employee Assistance Program, 5. Fitness for Duty Evaluation,

or 6. Dismissal Pursuant to Civil Rules and Regulations. The

memorandum goes to the Commander of Internal Affairs through

the chain of command. Each reviewing supervisor must agree or

disagree with the recommendation.102

The success of the Miami EW system, however, is uncertain. In

2001-2002, the department faced a series of problems, including

the indictment of a number of police officers on federal charges of

abuse of authority. News media accounts indicated that some of

the officers indicted had records of serious performance problems

–which presumably should have been identified by the EW

system. In late 2002, the chief of police resigned in the wake of

these events.103

Miami-Dade 

The EW system in the Miami-Dade police department was

developed at about the same time as the City of Miami's response

to a number of racial incidents. (It is not clear what role the City

of Miami's EW system played in influencing the development of

the Miami-Dade system.) The beating of an African-American

school teacher and the highly publicized fatal beating of Arthur

McDuffie, an African-American insurance agent, by Miami-Dade

officers, heightened racial tensions in the Miami area. On May 17,

1980, the acquittal of four officers accused of McDuffie's death

provoked three days of riots that resulted in civilian deaths and

millions of dollars in property damage.104 As a result of these

problems, a local ordinance directed the Miami-Dade Police

Department to develop an Employee Profile System (EPS) that

provides detailed information on each employee. In addition to

demographic data, the EPS includes all complaints, use-of-force

52

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
53

incidents, commendations, discipline, and disposition of all

internal investigations. As an adjunct to the EPS, the department

implemented an Early Identification System (EIS) under the

supervision of the Internal Review Bureau. 

The EIS was operational by 1982. Officers are identified on the

basis of two or more "personnel complaints" and/or three use-offorce incidents in the previous three months, and then listed on

the next Quarterly Report. The reports are disseminated to

supervisors to be used as they deem appropriate with regard to

each officer. The "Action Alternatives" include referral to

counseling for stress, referral to a 40-hour training assistance

program, or counseling by supervisory personnel. The EIS has

remained basically unchanged for the past two decades.105 The NIJ

evaluation of the Miami-Dade EIS found that intervention

succeeded in substantially reducing use-of-force among subject

officers.106

An Evolving Concept

Without any guidance from recommended models, EI systems

developed in an ad-hoc fashion. Over the years, a rough

consensus began to emerge about the major system components.

The most important change was an expansion of the number of

performance indicators used by a system. Some initial systems

(e.g., Minneapolis)107 used only citizen complaints. Others used

citizen complaints and use-of-force reports. As time passed, there

was increased recognition that one or two indicators are not

adequate for identifying officers with potential performance

problems. (This issue is discussed in greater detail in Chapter

Four). Citizen complaints, for example, are filed in only a small

number of potential cases, making them a very arbitrary indicator

of officer performance.108 At the same time, it is increasingly

recognized that other indicators–such as resisting arrest charges,

use of sick leave–are very useful for identifying officers with

potential performance problems.

Background: Origins and Development of Early Intervention Systems

Chapter 3
An Emerging Best Practice

By the year 2001, EI systems had emerged as a recognized best

practice in police accountability. A series of actions by the U.S.

Department of Justice contributed to this development. A 1996

DOJ conference on Police Integrity recommended that police

departments explore EI systems as an accountability

mechanism.109 A 2000 conference sponsored by the COPS Office

included a discussion of EI systems.110 Finally, the Justice

Department produced the 2001 report entitled Principles for

Promoting Police Integrity. Other recommended best practices

include a comprehensive use-of-force reporting system, an open

and accessible citizen complaint process, and data collection on

traffic stops.111

In 1997, the Civil Rights Division of the DOJ entered into the

first consent decree under Section 14141 of the 1994 Violent

Crime Control Act. The consent decree with Pittsburgh included a

requirement that the police department implement an EI system,

which became known as the Performance Assessment and Review

System (PARS). Subsequent consent decrees and memoranda of

understanding have contained similar requirements. The consent

decree with Los Angeles, for example, requires the LAPD to

include seventeen performance indicators in its revised TEAMS II

system.112 The memorandum of understanding with Cincinnati

requires the police department to include ten performance

indicators in its Risk Management System.113

Finally, in 2001, the Commission on Accreditation for Law

Enforcement Agencies (CALEA) adopted Standard 35.1.15

requiring a Personnel Early Warning System.114 This

recommendation by CALEA is particularly significant since the

Commission represents the leaders of the law enforcement

profession. 

In short, by 2001 a consensus of opinion had emerged among the

U.S. Justice Department, federal courts, the law enforcement

accrediting agency, and civil rights agencies that EI systems were

a necessary part of a comprehensive approach to police

accountability.

54

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
55

Implementing EI Systems: A Major Challenge

Although the concept of EI systems has become increasingly

popular, the history of particular systems indicates that they are

often extremely difficult to implement and maintain. When law

enforcement agencies began to develop the first EI systems they

were forced to do so without guidance from any published

literature on the subject. There were no descriptions of systems,

recommended models, or evaluations. Some departments simply

devised their own systems, while others borrowed from other

early systems.115

With the benefit of hindsight, it is now clear that implementation

is a major issue with respect to EI systems. (This issue is

discussed in detail in Chapters Three and Four of this report.) The

NIJ evaluation of EI systems in three cities concluded that EI

systems extremely complex administrative tools that require

considerable administrative attention.116 An effective EI system

requires careful planning and close, ongoing administrative

attention once it is established. The problems associated with the

development and implementation of an EI system are illustrated

by the histories of several large departments.

The Los Angeles Sheriff's Department PPI System

The Personnel Performance Index (PPI) in the Los Angeles

Sheriff's Department (LASD) is widely regarded as the most

sophisticated EI system in the country. One report described it as

"without question, the most carefully constructed and powerful

management tool for control of police misconduct currently

available in the United States."117 Representatives from the LASD,

for example, have been invited to describe the PPI at national

level conferences on police accountability. The history of the PPI,

however, indicates the difficulties in establishing a first-rate EI

system.

The idea for the PPI originated with the Kolts Report on the

LASD in July 1992. The report pointed out that the LASD did not

have an effective mechanism for identifying officers who posed a

high risk to the department with regard to civil liability claims.

Background: Origins and Development of Early Intervention Systems

Chapter 3
One of the main reasons was that the department did not

systematically collect and retain relevant performance data. Many

important items, for example, were maintained on paper 3x5

cards, and some of these records were periodically discarded. In

order to achieve an effective risk management mechanism, the

Kolts Report recommended the creation of what eventually

became the PPI.118

The LASD responded favorably to the Kolts recommendation and

planned to have the PPI operational by 1993. Because of a

number of delays–similar to those that have affected other

agencies-the target date for a fully operational PPI was pushed

back to 1997. During this period, Los Angeles County settled a

civil claim against the Sheriff's Department by promising to pay

an additional $500,000 if the PPI was not fully operational by

March 31, 1997. When that date arrived, the Special Counsel to

the Sheriff's Department served as reviewer and concluded that

the PPI system was indeed fully operational.119

In short, it took almost five years for the LASD to implement its

EI system. This occurred in the context of strong support from

both the Sheriff and the County Board of Supervisors, and in an

agency that is technologically sophisticated. 

Nonetheless, in early 2003 the Special Counsel to the LASD

delivered a very critical report on the PPI. A number of

administrative problems had weakened the effectiveness of this

potentially outstanding system. In particular, staff cutbacks had

made it difficult for the LASD to enter performance data into the

PPI database, with the result that commanders were not able to

conduct timely analyses of officers' performance. The Special

Counsel also found that many supervisors in the department were

not familiar with the PPI and its capabilities. Some district

commanders had spent time and money developing their own

local systems because they were not aware that the data they

wanted was readily available through the PPI. The Special

Counsel's report clearly indicated that for an EI system to be fully

effective, a department needs to maintain adequate staffing for the

system and to engage in a continuous process of training

officers–and commanders in particular–on the nature and

capabilities of the system.120 

56

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
57

The Los Angeles Police Department

The story of the TEAMS in the Los Angeles Police Department

(LAPD) represents a more disturbing picture of implementation

problems associated with EI systems.121 It begins with the highly

publicized beating of Rodney King on March 3, 1991. The

Christopher Commission, appointed to investigate problems

within the LAPD, found both that a small group of 44 officers had

particularly serious performance records that were readily

identifiable through existing departmental records, and that the

department took no effective action to either discipline or correct

the behavior of these officers. To address this problem, the

Commission recommended the creation of an EI system.122

The LAPD's EI system became known as TEAMS (Training

Evaluation and Management System). As a report by the Los

Angeles Police Commission documents, development of the

system over the next few years was erratic at best. In 1996, the

Police Commission hired Merrick Bobb (then also the Special

Counsel to the LASD) to conduct a five-year update on

implementation of Christopher Commission recommendations.

Bobb reported that the LASD did not have an operational EI

system as intended by the Christopher Commission and that

TEAMS was at that point "weak and inadequate."123

The president of the Los Angeles Police Commission then

initiated a request for a federal grant to facilitate the

implementation of an expanded version of TEAMS. The U.S.

Department of Justice awarded a grant of about $175,000 in

September, 1997, and the new EI system became known as

TEAMS II. In 1999, the Rampart scandal embroiled the LAPD in

further controversy, prompting three separate investigations of the

department (one by the LAPD itself,124 one by the L.A. Police

Commission,125 and one sponsored by the police union126). The

Police Commission's report documented the history of the

TEAMS II system and the fact that it was not currently

operational. In fact, the city had not even drawn on the federal

grant awarded to implement TEAMS II. 

Background: Origins and Development of Early Intervention Systems

Chapter 3
Meanwhile, the U.S. Justice Department sued the City of Los

Angeles and the LAPD under the 1994 Violent Crime Control

Act, alleging a pattern or practice of abuse of citizens. The suit

was settled through a consent decree that closely resembled

similar decrees and memoranda of understanding negotiated by

the Justice Department. The consent decree mandated the LAPD

to complete the planned implementation of TEAMS II and

specified that it should incorporate 17 separate performance

indicators.127

In short, eleven years after the 1991 Christopher Commission

recommendation, the LAPD did not have a fully operational EI

system.

Pittsburgh Police Department

The Performance Assessment and Review System (PARS) in the

Pittsburgh Police Bureau represents a case of a successful

implementation. The system originated with a 1997 consent

decree settling a U.S. Department of Justice suit against the

department for failure to address excessive force problems. The

Pittsburgh suit was the first to be settled under the provisions of

Section 14141 of the 1994 Violent Crime Control Act authorizing

the DOJ to bring suit where there is a "pattern or practice" of

violating citizens' rights.128

The Pittsburgh consent decree has served as a model for

subsequent agreements in Section 14141 litigation. Later

agreements are very similar in overall content, but tend to be

much longer and more detailed than the Pittsburgh agreement.129

The Pittsburgh PARS includes data on eighteen performance

indicators. Officers are identified by the system if their number of

indicators is one or more standard deviations greater than their

peer officers–thus, night shift officers are compared with other

night shift officers. One of the most significant aspects of PARS is

that it is incorporated into the routine supervision process in the

department. Supervisors are directed to log on to the system each

day prior to roll call. As a result, each supervisor is able to note

each and every activity that might require supervisory attention.

58

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
59

The incidents are not limited to those officers who have been

previously identified by PARS. A Vera Institute evaluation

characterized the PARS process in terms of a "sweeping change in

the duties of the lowest level supervisors."130

Implementation of the Pittsburgh PARS has been subject to two

separate monitoring and evaluation procedures. The consent

decree required the appointment of an independent monitor to

oversee implementation of the decree and to file semi-annual

reports with the court. The monitor found the Pittsburgh police

department to be in compliance with the decree, including the

provisions related to PARS, and in the summer of 2002 the court

lifted the decree (for the police department but not for the

independent Office of Municipal Investigations which is

responsible for handling citizen complaints).131

Meanwhile, the consent decree and its implementation were

evaluated by the Vera Institute of Justice. The evaluation,

completed in 2002, found not only that the consent decree was

successfully implemented, that it had a positive effect on many

aspects of the police department, and that a number of alleged

unintended consequences did not in fact occur. The Vera

evaluation found that the EW system was functioning effectively

as intended.132

The successful implementation of the EW system in Pittsburgh

was due in large part to what the Vera Institute described as a

"dogged determination" on the part of the police chief and other

city officials to comply with the terms of the consent decree.133

The lesson of the Pittsburgh experience is that a sophisticated EI

system can be successfully implemented in a large police

department in a relatively short period of time, given sufficient

leadership and resources.

Research on the Effectiveness of EI Systems 

There are five sources of evidence on the effectiveness of EI

systems. They include: (1) the NIJ study of three major police

departments, (2) an evaluation of the consent decree mandating

the creation of PARS in the Pittsburgh Police Bureau, (3) the

Background: Origins and Development of Early Intervention Systems

Chapter 3
reports of the Special Counsel to the Los Angeles Sheriff's

Department which include periodic assessments of the LASD PPI,

(4) anecdotal evidence from other departments, and (5) the

UNO/PERF survey of police managers. The first three are

discussed below. The UNO/PERF survey is reported in detail in

Chapter Four.

(1) The NIJ Study of Early Warning Systems

The National Institute of Justice (NIJ) study was conducted by the

author of this report and involved the EW systems in three large

police departments: Miami-Dade, Florida; Minneapolis,

Minnesota; and New Orleans, Louisiana. The study found all

three EI (the earlier NIJ report used the term Early Warning, but

this report uses the now-preferred term Early Intervention)

systems to be effective in reducing citizen complaints and use of

force among officers subject to intervention.134

The NIJ study included both qualitative and quantitative

components and a number of previously unacknowledged issues

were identified in the course of the evaluation. In many respects,

the qualitative findings on the administration of EI systems are

more significant that the quantitative findings on the impact of EI

system intervention on individual officers. Most important, the

study found that EI systems are extremely complex administrative

tools that require close ongoing administration. Additionally, the

authors concluded that it was difficult to separate the process and

impact of an EI system from the larger climate of accountability

within a police department. 

Qualitative Analysis of the Three EW Systems

Each of the three police departments had very different histories

with respect to accountability in the years immediately preceding

the evaluation, and these histories affected the nature of their

respective EI systems.

The Miami-Dade police department had been in the process of

enhancing accountability for almost twenty years. The department

had a comprehensive personnel database in place that served the

60

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
61

EIS but was independent of it. The Minneapolis Police

Department, meanwhile, was in the process of raising its

standards of accountability, particularly with respect to the use of

force. The current chief was appointed in large part because of

criticism of the lax disciplinary policies of the previous chief. The

New Orleans police department, meanwhile, was also under the

direction of a new chief executive who had been appointed to

reform the department following a number of force and corruption

scandals that generated considerable national publicity. The NIJ

study concluded that the EI systems in each of the three

departments were integral parts of ongoing departmentwide

accountability efforts. In such a context, it is difficult to specify

the independent effect of the EI system separate from other

reforms that occur in the department generally. 

As has already been pointed out, the qualitative research also

found that EI systems are far more complex administrative

mechanisms than is generally recognized. The study found that EI

systems require enormous administrative attention if they are to

be implemented and maintained effectively. The study dispelled

the notion that an EI system is simply a tool (e.g., an alarm clock)

that can be purchased off the shelf, plugged into the department,

and then left to run by itself. (Chapter Three of this report

provides even stronger documentation on the importance of

planning and administrative follow-through.) 

Evidence of the administrative complexity of an EI system is

illustrated by several examples drawn from the three departments

studied. One of the EI systems had actually ceased operation for a

period of about one year because of disruptions related to

administrative changes in the department. One of the EI systems

had undergone considerable revision and improvement since it

was first launched. The officials in charge called their efforts "a

work in progress." One of the EI systems studied had serious

problems related to inputting data in a timely fashion. Two of the

departments studied had rudimentary computer systems for their

EI systems, such that it was possible to retrieve the files for

individual officers but not to generate data on larger patterns and

relationships between incidents and officers.

Background: Origins and Development of Early Intervention Systems

Chapter 3
Quantitative Evaluation

The quantitative component of the evaluation attempted to assess

the impact of each EI system on officers who were subject to

departmental intervention. Because of the nature of the individual

EI systems and the data available, two of the EI systems were

subject to one mode of evaluation and the third subject to a

different mode.

In Miami-Dade and Minneapolis, the evaluation methodology

consisted of an analysis of the performance records of all officers

hired in certain years (1990-1992 in Miami-Dade; 1991-1992 in

Minneapolis). Each cohort of officers was partitioned into those

officers who had been subject to the EI system and their

colleagues who had not been subject to it (non-EI). The two

groups were then compared in terms of background characteristics

and performance records. The performance records of the EI

group were analyzed in terms of pre-intervention and postintervention.

The nature of the New Orleans PPEP, which involved a four-day

class with a group of officers, offered a unique opportunity for a

different methodology. First, one PPEP class was directly

observed for the purpose of collecting data on the content of the

intervention and the observed responses of officers in the class.

Second, because all officers subject to PPEP classes are invited to

complete a course evaluation form, it was possible to analyze the

responses of all officers who had ever been subject to the PPEP

intervention.

Findings

(1) Officer Backgrounds

The data in Miami-Dade and Minneapolis found that race or

ethnicity was not a factor in with regard to those officers subject

to EI intervention. White, African American, and Hispanic

officers were selected for EI in proportions closely related to their

presence in the larger cohort. This finding is particularly

significant because it does not support the popular view that the

62

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
63

problem of police use of excessive force is primarily a matter of

white police officers mistreating racial and ethnic minorities. It is

consistent with other research indicating that race or ethnicity is

not a determining factor with regard to law enforcement officer

behavior.135

Female officers, meanwhile, were only about half as likely to be

subject to EI intervention, as their presence in the larger cohort

would have predicted. This finding is consistent with other

research indicating that female officers are less likely to be

involved in use of force incidents and are rarely identified as

officers with patterns of repeated use of excessive force.136

(2) Performance Histories

Officers selected for intervention by the EI systems in MiamiDade and Minneapolis did in fact have significantly worse

performance histories prior to intervention when compared to

their colleagues.

In Miami-Dade, for example, officers selected by the department's

Early Intervention System (EIS) averaged twice as many use-offorce reports per year as non-EIS officers. EIS officers were twice

as likely to have received four or more citizen complaints than

their peers, were twice as likely to have received a formal

reprimand, and almost three times as likely to have been

suspended. In Minneapolis, officers selected by the department's

EI system averaged twice as many complaints as their non-EI

peers, were slightly more likely to have been reprimanded, and

were three times as likely to have been suspended. In one

troubling finding, EI officers in both Miami-Dade and

Minneapolis were slightly more likely to be promoted than their

non-EI peers, although in Miami-Dade the difference was not

statistically significant. This issue merits further research. The

data from this study does not permit investigation into the reasons

for this finding. It is possible that for some officers, the negative

aspects of assertive behavior that results in EI system

identification also includes positive aspects that help earn

promotion. Future research and policy development can,

hopefully, permit EI systems to identify and curb the negative

Background: Origins and Development of Early Intervention Systems

Chapter 3
aspects of assertive behavior while retaining the positive aspects

of officer assertiveness. 

This finding lends support to the anecdotal view, supported by

investigative journalists' reports that a small group of officers in

any department do have significantly worse performance records

than their colleagues.

(3) Impact of Intervention

The evaluation found that in Miami-Dade and Minneapolis the EI

systems had significant and positive impacts on officers'

performance histories. Among officers subject to EI system

intervention, citizen complaints and involvement in use-of-force

incidents declined by about two-thirds. In Miami, 27 of the 28

EIS officers had at least one use-of-force report; following

intervention only 14 had a use-of-force report. Twenty of the 28

officers had four or more complaints prior to intervention, while

only nine of the 28 had four or more complaints following

intervention. In Minneapolis, EI officers received an average of

1.9 complaints per year prior to intervention and 0.65 per year

after intervention.

One curious aspect of the findings is that the improvements in

officer performances were virtually identical in Miami-Dade and

Minneapolis despite the fact that the two EI systems had very

different administrative histories. The Miami-Dade system had

been in place for over 15 years and was well-integrated into the

department. The Minneapolis system, on the other hand, was far

less sophisticated (with regard to performance indicators) and had

experienced some significant disruptions. One possible

explanation is that an EI system works effectively with many (and

perhaps most) officers simply by putting them on notice (e.g.,

warning) and that an elaborate intervention is not necessary.

(4) Officers' Responses to Intervention

The nature of the New Orleans PPEP made it possible to both

observe officers in the PPEP class and to analyze responses to the

course evaluation. This component of the evaluation yielded

extremely important findings.

64

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
65

First, the direct observation of the class found that officers

responded in very different ways to different units and teaching

methodologies. During the class, most of the officers expressed

open hostility to the PPEP, referring to it as "politeness school,"

or "bad boys class." Officers were visibly disengaged from

aspects of the course that involved traditional lectures and in

particular lectures about general topics such as the role of the

police, stress management, and the problem of substance abuse in

American society. Some officers appeared to be close to falling

asleep in the class during these units. Some expressed their

disinterest through obvious body language. 

In sharp contrast, however, they were actively engaged in the unit

of the course that asked them to critique specific police-citizen

encounters. The case studies were drawn from the department's

files, and in a few cases the officers claimed to remember the

incident (none involved officers in the class). In this exercise, the

officers appeared to be knowledgeable and professional police

officers, fully aware of proper procedure in difficult situations,

and proud of their own knowledge about how to control situations

effectively. It should be noted that this engagement was exhibited

by the same officers who had earlier voiced extremely hostility to

the PPEP.

The lesson of this finding is that interventions that involve

traditional lectures are not likely to be effective in engaging

officers about performance issues. The use of active learning

techniques that allow officers to draw on their experience and

knowledge, on the other hand, appear likely to engage their

attention. In the observed case, however, it is not known whether

the officers in the class internalized the lessons from the case

studies and altered their own behavior accordingly.

With respect to the evaluations of the PPEP classes, officers who

completed the evaluations gave the class an extremely positive

rating. On a scale of 1-10, they gave it an average rating of 7.0.

When the four officers who gave it a rating of one (the lowest

possible rating) are omitted, the average rating rises to 8.1. These

responses are extremely significant in light of the fact that the

officers in the observed class openly disparaged the PPEP

program.

Background: Origins and Development of Early Intervention Systems

Chapter 3
The positive responses on the numerical evaluation were

supported by the narrative comments on the evaluation forms. All

of the officers made at least one positive comment about the class.

Comments included favorable references to the verbal

judo/complaint reduction component and the stress reduction

component. About three-quarters of the officers also made

negative comments. Significantly, however, all the negative

comments were about the PPEP in general or the department and

its leadership. There were no negative comments about the PPEP

class per se.

The positive evaluations of the PPEP classes are an extremely

important finding. It suggests that most officers appreciate

practical assistance designed to help improve their performance.

This finding is reinforced by the PERF survey of police managers

(See Chapter Three) which found no major hostility to EI systems

from rank and file officers.

Limitations of the Study

While extremely encouraging, the evaluation was subject to a

number of limitations that need to be noted: 

First, this was a retrospective study utilizing available official

data. Consequently, it was subject to the limitations of that official

data. While the study sought to control for important variables,

the investigators are not 100 percent certain that all the data are

accurate. One particularly important issue involves officers'

assignment. Reassignment to a low-crime area could account for a

significant drop in citizen complaints and use-of-force incidents.

Official records with regard to assignment histories, while good,

are not necessarily perfect. 

Prospective evaluations in which investigators can monitor

assignment patterns and other relevant factors can reduce or even

eliminate these problems.

Second, the design of this study does not make it possible to

determine exactly what aspect of the EI intervention accounts for

the positive results. It is entirely possible that for many officers

66

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
67

simply being selected by the EI system was a sufficient warning

that led to improved performance. A number of police officials

involved with EI systems believe that this is true. The effect here

is independent of the actual content of the intervention. It is

equally possible, however, that some officers benefited from

specific aspects of the intervention: advice on how to de-escalate

confrontations, or to handle groups of troublesome citizens. It is

entirely likely that different officers respond differently.

Future research on EI systems should focus on specifying what

kinds of intervention are appropriate and effective for different

kinds of officers. It may also be useful to think in terms of triage:

that some officers may need only a warning, that others may need

help with specific performance issues, and that some may not

respond at all to intervention.

(2) Evaluation of the Pittsburgh PARS 

The Pittsburgh PARS has been evaluated in two different ways.

First, the consent decree included a court-appointed monitor to

oversee implementation of the decree. The reports of the monitor

found that the Pittsburgh Police Bureau successfully complied

with the terms of the decree, including implementation of the

PARS. The decree was lifted in the summer of 2002. 

Second, the Vera Institute and PARC conducted a general

evaluation of the impact of the 1997 consent decree settling a

U.S. Justice Department pattern or practice suit against the

department. The evaluation found that PARS had a generally

positive impact on the department. Several supervisors testified

that it did enhance the quality of supervision in the department. At

the same time, they expressed concern about the burden of paper

work required by PARS. One supervisor offered an unverified

estimate that the amount of time sergeants devoted to traditional

street-level supervision declined from 70-80 percent of total time

to 25-30 percent.137 

An important aspect of the Vera/PARC evaluation was an analysis

of some departmentwide activity data (although trends in these

data are not necessarily attributable to PARS alone). The data

Background: Origins and Development of Early Intervention Systems

Chapter 3
found, for example, that despite anecdotal reports of a serious

decline in officer morale because of the consent decree, the

number of sick days taken by officers actually declined from 1995

to 1999. Also, there was no upsurge in separations from the

department between 1996 and 1999, despite anecdotal reports to

the contrary. The data indicated a decline in arrests beginning in

1994, but this trend paralleled a decline in reported crime and

could not be attributed to the consent decree. Finally, there was no

decline in traffic tickets issued following the 1997 consent decree

(a sharp decline occurred in 1996 as a result in a change in the

rules for overtime pay).

(3) Assessment of the LASD PPI System

The Special Counsel to the Los Angeles Sheriff's Department

reports data indicating that the PPI is very effective in achieving

its goals. The Special Counsel a form of external citizen oversight

for the LASD and operates through a contract with the Los

Angeles County Board of Supervisors. The Special Counsel

reported monthly average data on officers subject to Performance

Review (PR) for periods three years prior to PR, the period during

PR, and the period after PR. The rate of officer-involved

shootings dropped from an average of .50 per month before PR to

zero following PR. Use-of-force incidents dropped from an

average of 7.11 per month before PR to .98 per month after PR.

And personnel complaints dropped from an average of 3.86 to .74

in the same time periods.138 

One interesting finding was that 11 percent of all officers placed

on Performance Review left the department within a short period

of time. This included ten retirements, six discharges, one

resignation, and one departure for reasons that were not clear. In

short, two-thirds were voluntary departures.139 To the extent that

the PPI encourages problem officers to leave the department

voluntarily, it saves the department both the direct expense of

conducting a full two-year Performance Review and the risk that

an officer will engage in other serious misconduct.

The 16th Semiannual Report of the Special Counsel, however, was

very critical of the PPI. As noted earlier, it found that the PPI had

not been fully staffed and as a result performance data were not

68

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
69

being entered into the database in a timely fashion. Also, many

commanders were not aware of the nature and capabilities of the

system.140 As a result of the Special Counsel's report, LASD

commanders made a public commitment to make the necessary

changes to ensure that the PPI was fully effective and capable of

achieving its basic goal of reducing litigation costs to the county.

The Special Counsel's report highlighted the administrative

demands of an EI system and the need to continuously oversee

the operation of a system.

(4) Anecdotal Evidence From Other Departments

At the Early Intervention State of the Art Conference in early

2003, commanders from several EI systems were able to describe

case studies of individual officers who had been helped by EI

intervention. One commander, for example, described the case of

a female officer who was not engaging citizens in a properly

assertive manner, a practice that encouraged aggressive behavior

on the part of some citizens (and resulting uses of force by the

officer). Intervention counseling determined that the officer had a

great fear of being struck in the face. Training helped the officer

become skilled in the proper techniques for engaging potentially

aggressive citizens. As a result, the officer's performance

improved dramatically. The commander from another department

described the case of an officer whose performance was being

affected by serious personal financial difficulties. Intervention

counseling helped the officer obtain professional financial advice

and his performance subsequently improved.141

The two cases cited above are anecdotal and not sufficient to

validate the overall effectiveness of an EI system. Discussions at

the EI State of the Art Conference explored the possibilities of

developing more systematic ways of documenting individual

success stories (with personal identifiers eliminated to preserve

privacy).

(5) The Police Managers' Survey

The findings of the police managers' survey, which includes both

managers experienced with EI systems and managers without

such experience, is reported in Chapter Four of this report. 

Background: Origins and Development of Early Intervention Systems

Chapter 3
Conclusion

EI systems originated as administrative tools for identifying a few

officers with performance problems. They have evolved into more

elaborate tools for reviewing officer performance and achieving

higher standards of accountability. Research to date indicates that

EI systems can be effective in achieving their goals. At the same

time, the history of EI systems in several departments indicates

that EI systems are difficult to implement effectively. Careful

planning and continuous administrative oversight is necessary.

70

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 3
Lessons From the Field: Police Managers

Perceptions and Experiences With Early Intervention Systems

The PERF Survey............................................................................................73

Part One: Experiences With Early Intervention Systems ............................................74

Source of their EI Systems .............................................................................76 

Overall Assessment of the System....................................................................76

Impact on the Quality of On-the-Street Police Service...........................................76

Positive Contributions to Supervision and Accountability........................................76

Strengthens the Role of Supervisors .................................................................77

Early, Non-Disciplinary Intervention .................................................................78

Ability to Track Officers Who Change Assignments................................................78

Problem-Solving Supervision...........................................................................78

Timely, Objective Data .................................................................................79

Lack of Opposition From Rank and File and Unions ...............................................79

More Positive Than Negative Impact on Officer Morale ..........................................80

Changes in Officers’ Attitudes ........................................................................80

Differential Impact on Officers .......................................................................81

No Problems Regarding Confidentiality .............................................................81

Problems With EI System...............................................................................81

Chapter Four
Implementation Problems..............................................................................82

Lack of Commitment by the Deparment ............................................................83

Infrastructure Needs....................................................................................83

Successful Planning .....................................................................................83

Need for Continued Revision and Improvement....................................................83

What Needs to be Changes or Improved.............................................................84

Part Two: Departments Without Early Intervention Systems .......................................84

Managers Knowledgeable About EI Systems.........................................................84

Managers Overwhelmingly Anticipate Positive Impact of an EI System........................85

Managers Anticipate Objective Systematic Data...................................................85

Managers Anticipate Positive Impact on Supervisors..............................................86

Managers Anticipate a Reduction in Lawsuits Against the Department........................86

Managers Anticpate Multiple Benefits...............................................................86

Anticipate a Positive Impact on Community Relations ...........................................87

Overestimate the Opposition From Within the Department.....................................87

Understand Opposition Will Come From the Officers With Performance Problems .........87

Overestimate Union Opposition.......................................................................88

Underestimate Administrative Problems ............................................................88

Selling the Idea: Benefit to Subject Officers.......................................................88

Selling the idea: Makes the Supervisor’s Job Easier ..............................................89

Recognize Limitations of Current Accountability Mechanisms ..................................89

Able to Track Officers Who Change Assignments ..................................................90

Reactive Versus Proactive..............................................................................90

Prefer Information From Other Law Enforcement Agencies .....................................90

Conclusion ...................................................................................................91

Lessons From the Field: Police Managers

Perceptions And Experiences With Early Intervention Systems

Chapter Four
The PERF Survey

This survey represents collaboration between the University of

Nebraska at Omaha, the Police Executive Research Forum

(PERF), and DiscoverWhy. Funding was provided by the Office

of Community Oriented Policing Services (COPS) of the U.S.

Department of Justice.

DiscoverWhy is a private survey research firm developing

innovative online technologies for surveying public opinion. This

project represents an experiment in applying these technologies to

law enforcement and criminal justice research. PERF is a

professional association of police managers, with a membership of

nearly 600 commanders and supervisors across the country. PERF

has been involved in cutting edge research, training and

After more than a decade of experimentation and development (and in

some departments almost two decades), law enforcement agencies have

a considerable body of practical experience with Early Intervention

systems.

To tap into this experience, we surveyed law enforcement managers. As

described in detail below, this survey used innovative online techniques,

surveying members of the Police Executive Research Forum (PERF). The

methodology and principal findings are reported in the following sections. The

survey used the term Early Warning, and consequently that term is often used

in this chapter.

Lessons From the Field:

Police Managers’ Perceptions and Experiences

With Early Intervention Systems

4

73
demonstration projects since 1975. The COPS Office provided a

grant to the Criminal Justice Department of the University of

Nebraska at Omaha to provide technical assistance to law

enforcement agencies regarding Early Warning (EW) systems.

This survey represents one component of that project.

PERF provided a list of email addresses for its members. Each

member then received an email invitation to participate in the

survey. Those individuals indicating they wished to participate

then received a password allowing them access to the survey

instrument. Participants could then log on and participate at their

convenience. The DiscoverWhy technology permits an electronic

survey, with both closed and open-ended questions, that may be

accessed and completed at the convenience of each participant.

The open-ended questions are particularly valuable for tapping

qualitative assessments of the subject (in this case, EI systems)

and in the process eliciting valuable data embedded in mixed and

ambiguous responses. PERF submitted a mailing list of 831

names. Email invitations were sent to 521 members with currently

active email addresses. A total of 135 members participated in the

survey, for a response rate of 26 percent. About 40 percent of the

respondents work in departments with EI systems, while 60

percent work in departments without EI systems. This distribution

is consistent with the findings of an earlier survey of EI

systems.142

Part One: Experiences With Early

Intervention Systems

Law enforcement managers who have had practical experience

with an EI system report very positive assessments of its impact

in their departments. They report that it has had a positive impact

on supervision and accountability and on the quality of on-thestreet police work. Especially important, they have not

experienced serious opposition from rank and file officers or their

police unions. Many, in fact, report that the attitudes of rank and

file officers changed from suspicion to understanding and support

for the EI system.

74

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 4
75

Experienced managers also report a number of problems with EI

systems, however. The most important problems fall into two

general categories: a failure to communicate the nature and

purpose of these systems during the planning stage and a failure

of some command officers to follow through on their

responsibilities once the system begins operating. In short, these

are management problems and not problems inherent in EI

systems.

Lessons From the Field: Police Managers’ Perceptions And Experiences With Early Intervention Systems

Chapter 4

64%

32%

4%

Positive Mixed Negative

0%

10%

20%

30%

40%

50%

60%

70%

49%

28%

0%

23%

Positive Mixed Negative No Impact

0%

10%

20%

30%

40%

50%

General Assessment of EI System

Impact of EI System on 

Quality of Police Service
Source of their EI System

Most departments (70 percent) developed their own EI systems,

while 20 percent borrowed theirs from other departments.

Overall Assessment of the System

Experienced police managers report overwhelmingly positive

experiences with their EI systems: 64 percent reported a positive

general assessment and 32 percent reported a mixed general

assessment. The nature of these mixed assessments is explored in

responses reported below.

Impact on the Quality of On-the-Street Police

Service

Managers overwhelmingly reported that the EI system had at least

some positive impact on the quality of on-the-street police

service. Almost half (49 percent) reported a positive impact, while

28 percent reported a mixed impact. This represents a combined

total of 77 percent of respondents who report at least some

positive impact. Perhaps most important, no respondents reported

a negative impact on the quality of on-the-street police service. In

particular, no manager reported that the EI system caused officers

to back off and reduce their activity level. 

Positive Contribution to Supervision and

Accountability

Managers report that their EI systems have made positive

contributions to supervision and accountability in their

departments. Comments from these managers include:

• The process assisted us in identifying officers or non-sworn

personnel who have field responsibilities who began to

demonstrate behavior not consistent with our policies and

standards.

76

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 4
77

• As command staff members, we are constantly looking for ways

to pick up on officers in need of assistance. The Early Warning

System has picked up on several officers in the department who

were headed down the wrong path

Strengthens the Role of Supervisors

Police managers report that their EI systems strengthen the role of

supervisors. Some of their comments include:

• A useful tool to involve supervisors and lieutenants in a nontraditional model of problem solving. It has served to enhance

their management skills and help round out their people

interaction skills.

• Sergeants have been able to evaluate the strengths and

weaknesses of their squads even before meeting with them, and

therefore are able to develop proactive strategies to address

personnel and leadership issues.

• When an officer is placed in the program, for whatever reason,

his or her supervisor is also involved in the process. The Early

Warning System Board also looks at the actions of the

supervisor to determine if the supervisor is providing the

supervision needed to correct the problem.

Lessons From the Field: Police Managers’ Perceptions And Experiences With Early Intervention Systems

Chapter 4

65%

21%

2%

12%

Positive Mixed Negative No Opinion

0%

10%

20%

30%

40%

50%

60%

70%

Impact on Management 

and Supervision
• Raises the awareness of lack of proper supervision.

• Supervisors pay more attention to what is going on, document

all activities more thoroughly, and talk with officers when things

might look like a problem.

Early, Non-Disciplinary Intervention

Mangers report that, consistent with the objectives of EI systems,

the EI system has been an effective instrument for early, nondisciplinary intervention with officers who exhibit performance

problems. Two managers, for example, reported that:

• "We have been able to intervene with help before misconduct

occurs that requires discipline."

• "[The] program provides a way for the department to provide

non-disciplinary direction and training before the officer

becomes a liability to citizens, the department, and him/herself"

Ability to Track Officers Who Change

Assignments

One problem in police departments is that because of regular

changes in assignment many supervisors do not know the

performance histories of new officers under their command. An

EI system database overcomes this problem.

• There is a lot of movement of personnel, so supervisors often do

not know the histories of their officers. The EWS report brings

them up to speed in a much more timely fashion.

• No one falls through the cracks any longer.

Problem-Solving Supervision

Some managers understand that an EI system facilitates a

problem-solving approach to supervision. 

• A useful tool to involve supervisors and lieutenants in nontraditional model of problem-solving. It has served to enhance

their management skills and help round out their people

interaction skills.

78

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 4
79

Timely, Objective Data

Managers report that having objective performance data helps to

eliminate charges of favoritism, levels the playing field since no

one can be accused of playing favorites, and makes it much easier

to deal with the police union.

Lack of Opposition From Rank and File and

Unions

The most surprising finding is that managers report that they have

not experienced opposition to the EI system from their local

police union. This finding is particularly significant because

managers who do not presently have an EI system in place

anticipate such problems (see below). Eighty-four percent of the

managers with EI systems report having had no problems with

their unions regarding their systems.

Lessons From the Field: Police Managers’ Perceptions And Experiences With Early Intervention Systems

Chapter 4

16%

84%

Yes No

0%

20%

40%

60%

80%

100%

Problems With Police Union?
One of the few managers who encountered problems with the

union stated: 

"The union believes that the EWS is a witch hunting tactic,

used by the administration to target disliked officers. In fact,

no one has been disciplined because of EWS, it just wasn't

explained very well."

More Positive Than Negative Impact on Officer

Morale

Twice as many managers (12 percent) reported that the EI system

had a positive impact on morale among rank-and-file officers as

reported a negative impact (six percent). Slightly more than half

(52 percent) reported that the EI system had a mixed impact on

officer morale. Finally, 30 percent reported that the EI system had

no impact on rank and file morale. 

Explaining the positive impact on morale, one manager pointed

out that "nothing hurts morale like no action being taken against

problem officers. It shows supervision that problem people can be

dealt with." 

80

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 4

12%

52%

6%

30%

Positive Mixed Negative No Impact

0%

10%

20%

30%

40%

50%

60%

Impact on Officer Morale
81

Changes in Officers' Attitudes

Managers report changes in rank and file officers attitudes, from

suspicion or hostility to growing support for EI systems.

• The system was met with much cynicism and distrust when first

introduced. Experience and education has reduced those

attitudes.

• Most no longer believe this system is out to "get" them, but

rather to assist them.

Differential Impact on Officers

Some mangers reported that the impact of the EI system varied

from officer to officer. They reported, for example, that:

• Some officers take it to heart, others see it as unfair discipline.

• The solid officers of the department are supportive of the system

both conceptually and practically.

No Problems Regarding Confidentiality

Virtually no problems were reported with regard to breaches of

confidentiality, such as improperly leaking information about

which officers had been identified by the EI system. In fact, the

only reported case involved a situation where the officer subject

to intervention disclosed that information himself.

Problems With EI System

The overwhelming majority (85 percent) reported that they had

experienced some problems with their EI system. Only 15 percent

said they had had no problems.

Lessons From the Field: Police Managers’ Perceptions And Experiences With Early Intervention Systems

Chapter 4
Implementation Problems

Mangers did report a number of problems with their EI systems.

Virtually all of these problems were related to implementation

issues, in particular a lack of communication about the nature of

the EI system and a lack of follow-through on the part of

responsible officials.

The greatest number of problems involved a failure to

communicate the nature and purpose of the EI system to officers

in the department. Commanders identified this problem in

response to several different questions on the survey. Some

commanders explained, for example:

• [Top management] did not explain the purpose of the program

well [to mid-level supervisors].

• [Departments needed to deliver better] training and introduction

to the first line supervisors.

• [The agency] could have done a better job of pre-selling and

training on the benefits of an EWS. 

• While the system was crafted appropriately, it was not explained

to the officers or the first line supervisors to the extent

necessary to make it an understandable and viable system.

• The system is misunderstood by the first line supervisors, and

therefore retraining is needed at the application level.

82

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 4

85%

15%

Yes No

0%

20%

40%

60%

80%

100%

Problems With EI System?
83

Lack of Commitment by the Department

Another problem reported by experienced managers is that while

the EI system is good in principle, the department as a whole and

some mid-level supervisor did not follow through with their

responsibilities, thereby undermining the effectiveness of the

system:

• The program does not hold a prominent place in the

organization. As a result, no results are provided to commanders

as to its effectiveness or benefits.

• Our program lacks real involvement from the commanders.

Typically the officer's actions are justified or rationalized by the

commanders and rarely is there a consequence for the officer.

• [There was] irregular review and lack of consistent follow up. 

Infrastructure Needs

The lack of an adequate technological infrastructure was also a

major problem. About 43 percent of the managers reported that

their department did not have the necessary infrastructure in place

when their department began planning its EW system.

Successful Planning

Many managers reported that their EI planning effort was

successful. The key to success was the involvement of all relevant

stakeholders:

• Gathering input from all involved, officers through management

as well as the union.

• We involved Risk Management, the Deputies Union, and

management.

• Involving a wide range of personnel and viewpoints from the

start.

Need for Continued Revision and Improvement

Several managers acknowledged that their EI systems were not

perfect and that further revision and improvement was necessary.

Lessons From the Field: Police Managers’ Perceptions And Experiences With Early Intervention Systems

Chapter 4
"The system has worked well. EWS needs to continuously

evolve and expand."

What Needs to be Changed or Improved?

When asked what they thought needed to be changed in their EI

systems, managers most commonly reported that communication

problems needed to be addressed: 27 percent indicated that

improved communication between command staff and rank-andfile was their most pressing need, while 14 percent said that it

would be most important to improve communication among

command staff. 

The need to clarify certain aspects of the EI system was also

mentioned: 22 percent stated that it would be most important to

clarify the relationship of the EI system to the department's formal

disciplinary system, while 19 percent report that it would be most

important to clarify the purpose of the EI system. These problems

are also essentially communication problems.

Part Two: Departments Without Early

Intervention Systems

About 60 percent of the managers surveyed work in departments

that do not currently have operational EI systems. Their responses

indicate their knowledge about EI systems and their perceptions

of both potential contributions and problems.

Managers Knowledgeable About EI Systems

Generally, police managers in departments without EI systems are

very knowledgeable about them. Only 13 percent of respondents

said that they had no knowledge of EI systems, while another ten

percent, based on specific comments they made, were

misinformed about them. Thus, 77 percent of all managers had at

least some knowledge about EI systems. 

84

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 4
85

Managers Overwhelmingly Anticipate Positive

Impact of an EI System 

Managers overwhelmingly (94 percent) have a positive view of EI

systems and anticipate that one would have a positive impact in

their departments. Only three percent believe it would have a

negative impact in their departments, another three percent have

no opinion about the effect.

Managers Anticipate Objective Systematic Data

One of the advantages managers anticipate includes objective and

systematic data:

• EI systems allow the agency to use an objective scale by which

to assess the need for intervention and action to prevent future

problems.

• EI systems store data in a single location, thus offering the

potential to identify issues and problems in a timely fashion.

• EI systems offer a systematic approach to identifying officers

with performance problems. 

• EI systems allow supervisors to rely less on reputation and be

more objective in performance evaluations.

Lessons From the Field: Police Managers’ Perceptions And Experiences With Early Intervention Systems

Chapter 4

94%

3% 3%

Positive Negative No Opinion

0%

20%

40%

60%

80%

100%

Anticipated Impact of EI System
• EI systems instill more confidence that performance is being

monitored and dealt with in a consistent fashion.

• EI systems are consistent with what we [as the agency] already

do, but would provide a more formalized system within which

to do it.

Managers Anticipate Positive Impact on

Supervisors

Managers anticipate that an EI system will help supervisors do

their jobs. And as the previous section indicated, these

expectations are generally confirmed in practice.

• EI systems provide high-potential tools to help command staff

members identify trends that they may miss.

• EI systems help supervisors, middle managers, and

administrators identify internal problems dealing with officer

attitudes, performance, and skills.

Managers Anticipate a Reduction in Lawsuits

Against the Department

Managers anticipate that an EI system will help to reduce lawsuits

against the department. Significantly, however, this issue was not

mentioned by managers with EI systems in place. They did,

however, mention a reduction in complaints, including preventing

future liability, enhancing officer retention, fewer personnel

issues, and fewer law suits.

Managers Anticipate Multiple Benefits

A number of managers reported that they anticipate multiple

benefits to their department from having an EI system.

• EI systems help managers deliver employee assistance, risk

management, improved morale, and improved employee and

agency performance.

• EI systems provide liability prevention, good community

relationships, and good labor/management relationships.

86

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 4
87

Anticipate a Positive Impact on Community

Relations

Managers overwhelmingly (82 percent) anticipate improvements

in community relations. This factor was rarely mentioned by

managers in departments with EI systems, however.

Overestimate the Opposition From Within the

Department

Thirty percent of managers anticipate negative responses in their

department if they implemented an EI System and 40 percent

anticipate a mixed response. Thirty-seven percent anticipate

opposition from rank and file officers, for example, and 29

percent would anticipate opposition from the police union.

Another 30 percent anticipate a positive response. This represents

a far more pessimistic view than has proven to be the case in

departments with EI systems as reported in the previous section.

Managers anticipate:

• Some initial reluctance by supervisors and some uncertainty by

officers on the goals of the EI system.

• Skepticism at first, but acceptance that the system will be

deployed.

• Mild grumbling by the rank and file, minor hand-wringing by

mid-management, brief tooth gnashing by the chief, followed by

the system being implemented, rarely triggered, and quickly

assimilated as just another obscure general order in the manual.

• Skepticism until the benefits are proven.

Understand Opposition Will Come From the

Officers With Performance Problems

Many mangers that anticipate opposition from the rank and file,

however, recognize that most of it will come from the few

problem officers. Managers anticipate:

• Negative feedback from the bargaining unit, positive feedback

from the commanders, but negative feedback from the small

percentage of police officers that cause the most problems.

Ninety percent of the officers would understand and accept the

system.

Lessons From the Field: Police Managers’ Perceptions And Experiences With Early Intervention Systems

Chapter 4
• Employees would accept it unless they thought they might be

personally impacted. One respondent said, "Those are the ones

that we need it for."

Overestimate Union Opposition

Many managers overestimate the extent of union opposition.

Managers with experience, however, have not found significant

opposition (See Section One). Often, opposition manifests in

unexpected places. In practice, administrative problems tend to

overshadow most other issues. 

Underestimate Administrative Problems

In light of the problems reported by managers with EI system

experience, managers without experience underestimate potential

administrative problems. Only 29 percent mentioned

administrative problems as the major obstacle they expected to

face if implementing an EI system. Prospective EI implementation

planners anticipate:

• That the main obstacle will be collecting and displaying the

data.

• Inadequate staffing to run the system.

• Concerns about the duplication of work, specifically how it will

co-exist with the current supervisory and complaint tracking

system.

Selling the Idea: Benefit to Subject Officers

When asked how they would sell the idea of an EI system to their

department, almost half (47 percent) mentioned that it would

actually benefit officers who are subject to the system.

Prospective EI system planners anticipated:

• "I would be able to point to specific examples that would

demonstrate that if the agency had such a system in place,

careers may have been saved."

• "It permits early intervention to assist officers with problems

before job security becomes an issue."

88

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 4
89

• "We are trying to save careers and quite possibly families. I

don't think it will take a 'sales job.'"

Selling the Idea: Makes the Supervisor's Job

Easier

Many managers anticipate that they can sell the idea by

emphasizing its proactive problem-solving capacity. Some

managers' responses to the survey included:

• "It would make their job easier. A good system could help in the

early identification of problematic officers and procedures."

• "Allows us to spot problems before they become careerthreatening, eases workload by allowing the EWS to note where

effort should be applied rather than waiting on a supervisor to

recognize the trend."

• "To keep your subordinates out of harm's way makes the

supervisor's job easier."

Recognize Limitations of Current 

Accountability Mechanisms

In describing the potential benefits of an EI system, many

managers acknowledged the limitations of the current

accountability mechanisms in their department. Only 33 percent

rated their departments' current accountability measures as "very

effective." About half (47 percent) rated their departments'

accountability measures as "adequately effective." These

limitations include a lack of consistency, the absence of data or

objective standards, and excessive discretion in handling alleged

misconduct. Some of the limitations managers reported included:

• "[The EI system is] not consistent, no standard system in place.

Only a few supervisors take it seriously."

• "The Professional Standards Unit is responsible for

communicating Internal Affairs' (IA) role to Administration.

This works for identifying recurring problems. The limitation is

that it relies upon one supervisor and some complaints may not

get as high as Internal Affairs."

Lessons From the Field: Police Managers’ Perceptions And Experiences With Early Intervention Systems

Chapter 4
• "Minor complaints are most often handled by first line

supervisors and are not documented in any retrievable form over

time."

• "Lack of guidelines leaves a lot of discretion with individual

supervisors and therefore inconsistency across the department."

• "It is informal and relies on the sergeant's ability to perceive

problems and communicate effectively."

Able to Track Officers Who Change Assignments

Managers recognize that an EI system can help to track the

performance of officers who change assignments and to provide

new supervisors with useful data on their performance histories.

• Officers sometimes switch assignments frequently and get a

fresh start with new supervisors.

• EI systems help track officers across different shifts.

• Traditionally, officers' reputations tend to be directed by rumor

and perception only.

Reactive Versus Proactive

Traditionally, supervisors respond to officers' performance

reactively rather than proactively. 

• Supervisory action traditionally occurs after the fact and after

discipline has been imposed several times.

• Supervisory action traditionally is ad hoc and sometimes the

conduct rises to the level of formal action before we initiate

corrective action.

Prefer Information From Other Law Enforcement

Agencies

Managers expressed a preference for information about EI 

systems from other law enforcement agencies.

• 47 percent of respondents stated that if they needed additional

information about EI Systems, the best source of that

information would be expert personnel from another department

with an EI system.

90

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 4
91

• 30 percent of respondents stated that the best source of that

information would be printed material. 

• 20 percent stated that if they needed additional information

about EI systems, the best source of that information would be

an International Association of Police Chiefs (IACP) training

conference

Conclusion

Police managers who have experience with EI systems report very

positive experiences with them. These managers have found EI

systems to be an extremely useful tool for achieving

accountability, with many benefits for supervisors and rank and

file officers. This testimony is particularly important, as this

survey represents the first systematic investigation of practical

experiences with EI systems. They do not report strong opposition

from rank and file officers or police unions, despite the fact that

these problems are anticipated by managers in departments

without EI systems. Managers do, however, report a number of

problems related to the planning and implementation of their EI

systems, particular with regard to communicating the nature and

purpose of the system to all members of the department.

Significantly, managers in departments without EI systems

underestimate the management problems they will face. 

Chapter Four presents a guide for planning and implementing an

EI system. The guide is based in part on the findings reported in

this chapter.

Lessons From the Field: Police Managers’ Perceptions And Experiences With Early Intervention Systems

Chapter 4

A Planning Guide for Implementing an Early

Intervention System About This Chapter ........................................................................................95

Planning Process and Key Issues .........................................................................96

STEP ONE: Recognize the Need ..........................................................................96

STEP TWO: Involve the Command Staff Early .........................................................96

STEP THREE: Create a Planning Process ................................................................97

STEP Four: Pinpoint Responsibility ......................................................................98

STEP FIVE: Involve Officers From All Ranks............................................................98

STEP SIX: Utilize Outside Experts .......................................................................99

STEP SEVEN: Assess the Department’s Computer Infrastructure ...................................99

STEP EIGHT: Keep the Staff Informed ..................................................................100

STEP NINE: Keep External Stakeholders Informed....................................................100

STEP TEN: Identify and Address Specific Issues .......................................................101

STEP ELEVEN: Take the Time to Get It Right ..........................................................105

STEP TWELVE: Provide Staff Training ...................................................................105

STEP THIRTEEN: Choose Software .......................................................................106

STEP FOURTEEN: Finalize the Plan ......................................................................106

STEP FIFTEEN: Implement the EI System...............................................................106

Chapter Five

This chapter closely follows the planning process described in

Samuel Walker, Carol Archbold and Leigh Herbst, Mediating

Citizen Complaints Against Police Officers: A Guide for Police

and Community Leaders (2002), published by the U.S.

Department of Justice, Office of Community Oriented Policing

Services (COPS). That report is available on the web at

www.cops.usdoj.gov.

About This Chapter

Police managers experienced with EI systems strongly believe that the

development of an EI system should involve careful planning. Many of the

early EI systems made mistakes that had to be corrected later. Often, these

were mistakes that could have been avoided. Also, any agency developing an

EI system today has the benefit of these early experiences and can plan

accordingly.

This chapter is designed as a guide to help law enforcement agencies plan and

implement Early Intervention (EI) systems. Specifically, it:

** Outlines the steps in a planning process

** Identifies the key issues in planning

A Planning Guide for Implementing an

Early Intervention System

5

95
Planning Process and Key Issues

_____________________________________________________

STEP ONE: Recognize the Need

The first step in developing an Early Intervention system is to

recognize the need. EI systems are now a recognized best practice

in law enforcement, so there should be little room for debate on

this question. But law enforcement agencies tend to have specific

problems. It might be large civil litigation claims in one

department, but repeated complaints about traffic enforcement in

another. Some agencies took the first steps toward an EI system

with committees to review abuse of sick leave or excessive motor

vehicle accidents. 

Agencies planning to develop an EI system should begin by

identifying any specific problems they have that should be

addressed by the system as well as existing programs or data

systems that could be incorporated into an EI system.

STEP TWO: Involve the Command Staff Early

The senior command staff should be involved in the planning of

the EI system at the earliest moment. These officials will be

responsible for developing, implementing, and administering the

system, and therefore it is essential that they fully understand and

become committed to it.

It is essential that the chief executive make it clear that the EI

system is a top priority for the agency.

The initial meetings with the senior command staff represent an

opportunity for presentations by outside experts who can answer

questions about such issues as the goals, operating procedures,

potential advantages, potential problems, and administrative

requirements of an EI system. 

96

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 5

Key Issue: Begin with a self-assessment, identify existing

problems, current accountability efforts, and the potential

value of an EI system.
97

STEP THREE: Create a Planning Process

A formal planning process should be created. This involves the

appointment of a planning committee with a clear mandate

regarding the general outlines of the desired EI system. A

timetable for development and implementation of the EI system

should be established at the outset. The planning committee

should include representatives from all ranks of the department.

(This important issue is discussed in more detail later.)

The planning process should involve a series of regular meetings

(weekly, bi-weekly, and monthly) and these meetings should

continue with interruptions only for special circumstances. 

The development and implementation of EI systems in some

agencies took as long as two years. While in some cases the delay

was the result of unavoidable issues related to computer hardware

or software, in others it was because department leaders

deliberately took the time to involve all stakeholders and resolve

possible conflicts. Officials involved in those processes believe

that taking the time was worth the effort. As one commander put

it, "some important things would have been missed" if they had

rushed the development too quickly. The most important things

that would have been missed were (1) attention to certain issues

that could have caused problems later on, and (2) the buy-in from

officers that resulted from their having a chance to become

familiar with the nature and purpose of the system.

The importance of taking the time to involve stakeholders and

resolve all issues in advance is documented in the COPS Office

report, Mediating Citizen Complaints: A Guide for Police and

Community Leaders (2002).

A Planning Guide for Implementing an Early Intervention System

Chapter 5

Key Issue:The chief executive needs to make it clear that

the program is a top priority and that staff members will

be held accountable for failing to follow through on

developing the program.

Key Issue: Create a structured planning process.
STEP FOUR: Pinpoint Responsibility

A commander with the rank of Deputy Chief or Captain should be

appointed chair of the planning committee with responsibility for

managing the process.

STEP FIVE: Involve Officers From All Ranks

Officers representing all ranks of the department should be

appointed to the planning committee. Representation from all

ranks serves two purposes: 

Firstly, the planning process benefits from the insights of staff

members who have different experiences and perspectives on how

an EI system should operate. Sergeants, who play a key role in a

functioning EI system, will have insights into potential problems

regarding how the system will be integrated into their normal

duties. Direct involvement in the planning and the opportunity to

raise important issues ensures buy-in from all ranks in the

department.

Secondly, the members of the committee become an important

channel of communication to other staff members, allowing them

to explain the concept and reduce the potential harm of rumors

and misinformation.

Representatives of the collective bargaining unit for rank and file

officers should also be included in the planning process. It is

important to avoid creating a system that might conflict with

certain clauses in the collective bargaining contract.

98

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 5

Key Issue: Assign and hold one official responsible for the

planning process and the final development of the

program.

Key Issue: Involve officers from all ranks directly in the

planning process. The planning process should represent

the full range of rank, experience, race, ethnicity, and

gender in the agency.
99

If the EI system covers non-sworn employees in the agency, they

should also be included in the planning process. And if they are

represented by a collective bargaining unit, officials from that

organization should be involved.

STEP SIX: Utilize Outside Experts

There is now a substantial body of experience with EI systems in

the law enforcement community. No agency needs to reinvent the

wheel at this point. This report is designed to serve as a guide to

the issues related to developing an EI system. 

Law enforcement officials in departments with operating EI

systems currently in place are particularly useful as resources.

They bring their practical experience with EI systems and as peers

have substantial credibility with officers in the department.

Issues such as the selection of the thresholds for identifying

officers and choice of software are areas where an outside

consultant can be particularly useful.

STEP SEVEN: Assess the Department's Computer

Infrastructure

Based on the information learned from other departments, what is

the current state of the department's computer infrastructure? Is it

capable of supporting the more sophisticated systems that exist in

other departments? For example, some systems allow each firstline supervisor to log on to the system each day before roll call.

Would that be possible in your department?

If the department's computer system would not support the most

sophisticated system, what alternatives are possible? Some of the

A Planning Guide for Implementing an Early Intervention System

Chapter 5

Key Issue: Don't try to reinvent the wheel. Utilize existing

expertise from other law enforcement agencies or other

experts.

Key Issue: Involve representatives from the collective

bargaining unit(s).
best systems currently exist in departments with 2,000 and 8,000

sworn officers. Does a medium-sized department need as complex

an EI system? What kind of system would be appropriate–and

feasible–for a department of 300 to 700 sworn officers?

STEP EIGHT: Keep the Staff Informed

Members of the agency should be kept informed of the

development of the EI system as it develops. 

Involving officers from all ranks on the planning committee helps

address this issue, but other steps can be taken. In one agency a

series of separate meetings were held with the captains,

lieutenants, and sergeants. Keeping all members informed through

regular presentations helps to overcome misunderstandings that

might impede the efficient implementation of the system. In the

PERF survey (See Chapter Three), failures to communicate with

members of the agency were the major problem reported by

managers experienced with EI systems.

STEP NINE: Keep External Stakeholders Informed 

An EI system is an important instrument for strengthening

accountability within a department. Equally important is the task

of building the perception of accountability among important

leaders and groups outside the department. These groups include

the responsible elected officials (mayors, city mangers, city

council or county board members, community groups, etc.).

100

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 5

Key Issue: Begin addressing computer infrastructure

needs at the very outset. Plan a system that is both

appropriate in terms of accountability and feasible for the

department.

Key Issue:To avoid problems arising from

misunderstanding, provide staff at all ranks with detailed

information about the program.
101

Advising the public about the commitment to creating an EI

system and keeping them informed about its progress can help

develop positive relations with the community.

STEP TEN: Identify and Address Specific Issues

A number of specific issues need to be addressed in the planning

process: 

(1) Performance Indicators

The planning process should include extensive discussions of the

performance indicators to be used by the EI system. This process

will facilitate understanding of the nature of an EI system as a

comprehensive personnel assessment system and not just a

process for identifying officers with performance problems. Also,

the process will highlight deficiencies in the agency's existing

data systems that need to be corrected if the EI system is to

function properly.

As Chapter One explained, there is no consensus of opinion about

the exact number of indicators that should be used (except that

relying on only one or two is inappropriate). Existing systems use

as few as eight and as many as 24. 

Planning discussions about the number of indicators should take

into account the specific needs of the department (See Step One,

above) and the capacity of the department to support a large

versus a small system.

A Planning Guide for Implementing an Early Intervention System

Chapter 5

To assist in planning efforts, examples of performance

indicators from several departments are presented in the

Appendix to this report.

Key Issue: Build community and political support by

keeping external stakeholders informed about the

development of the EI system.
(2) Thresholds for Identifying Officers 

The planning process should involve extensive discussions about

what thresholds to set for identifying officers needing

intervention. Although this is an extremely important issue, there

are currently no nationally recognized best practices regarding

thresholds. A variety of different systems are being used (See

Chapter One). This is one area where the use of outside experts

would be particularly valuable so that the planning committee can

learn from the practical experience of other agencies.

The discussion of thresholds is an opportunity for members of the

planning committee to think through current performance

standards within the agency and to reach a consensus about

appropriate standards.

As part of its development process, the Phoenix Police

Department tested proposed thresholds to get a sense of how they

would work in practice. This involved loading past performance

data into the system and then applying the thresholds

retrospectively. Tests of this sort can help to ensure that thresholds

do not identify an unmanageable number of officers or fail to

identify potential problems in certain areas of police activity.

(3) Define the Format of Interventions

The format of the interventions needs to be discussed. There is

presently no national consensus regarding the best format. Some

agencies rely on immediate supervisors to counsel subject

officers. Others have meetings or counseling sessions with two or

102

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 5

To assist in planning efforts, examples of thresholds used

in other departments are presented in the Appendix to this

chapter.

Key Issue:Think through routine operations and identify

key performance indicators. One size does not necessarily

fit all. Indicators used by other agencies may not be

relevant or appropriate. Some other indicators may be

appropriate for your agency.
103

more supervisors present. Still others conduct classes for groups

of officers selected for intervention.

Discussions of alternative approaches will help officials better

understand the purpose of the interventions and feel comfortable

with the format ultimately selected.

(4) Coordinate the EI System With the Disciplinary

Process

The EI system is designed to be an alternative to the normal

disciplinary process. Once an EI system is operational the two

systems will function along separate but often parallel tracks. It is

important to clearly define the boundary between the two systems.

For this reason, it is advisable for representatives of the internal

affairs or professional standards unit to be included on the

planning committee. 

(5) Develop a List of Corrective Actions

Officers conducting intervention sessions should have a number

of specific recommendations for programs or steps that subject

officers can or should take. These programs or steps might

include retraining on specific police actions or a recommendation

that the officer seek family counseling. 

The planning committee should develop this list of programs or

steps. 

A Planning Guide for Implementing an Early Intervention System

Chapter 5

To assist in planning, examples of alternative corrective

actions used by some departments are presented in the

Appendix to this report

Key Issue: Be sure to define clearly the relationship

between the EI system and the existing disciplinary

process.
(6) Decide What Kinds of Documentation Will Be

Required

For purposes of internal accountability, it is important to

document the various steps in the EI system process.

What kinds of reports about officers selected for intervention will

be required? Will they be quarterly, semi-annual, annual? What

will be included in these reports? How much detail will be

included about each officer's performance? 

How will interventions be documented? Who will be responsible

for reports about intervention sessions? How much detail will be

included in these reports?

What kind of post-intervention documentation will be required?

For how long will a subject officer be monitored? Who will be

responsible for that monitoring? What kind of evidence will be

sufficient to remove an officer from the EI system?

(7) Define the Day-to-Day Responsibilities of First-Line

Supervisors

What will be the day-to-day responsibilities of first-line

supervisors? The Pittsburgh PARS requires sergeants to log on to

the system each day prior to roll call. They are expected to review

officer's performance records and identify any potentially

troublesome patterns and take some appropriate action even

before any officer exceeds a formal threshold. 

The purpose of this approach is to integrate the EI system into the

normal supervisory process and to create a culture of

accountability throughout the department.

104

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 5

Key Issue: Provide staff members who will be conducting

the counseling sessions with practical and relevant

alternatives that they can recommend to employees.
105

(8) Decide on Notification Procedures

Some EI systems have the technical capacity to notify anyone

from the first-line supervisor to the chief executive about any

incident involving a particular officer. For example, if Officer X

has been identified as having serious performance problems, the

EI system can be programmed to send selected officials an email

notification about selected performance indicators involving that

officer (e.g., any use-of-force report, citizen complaint, etc.).

Is this kind of process appropriate for the department? Does the

department have an internal email system capable of supporting

such a process?

STEP ELEVEN: Take the Time to Get It Right

Make sure all the issues are resolved before finalizing an EI

system. Proceeding slowly but deliberately allows stakeholders an

opportunity to gain an understanding of the EI system and to

become comfortable with it.

Some agencies developed their EI systems quickly then imposed

on the department without warning. In these cases, the response

has usually been suspicion, hostility, and resistance from officers

at many ranks.

STEP TWELVE: Provide Staff Training

All officers need to be trained in the operation of the system

before it becomes operational. One of the problems identified in

the PERF survey (See Chapter Three) was the failure of some

departments to adequately explain the responsibilities of various

officials in the EI system and to provide the proper training.

Rank and file officers should be fully informed about the nature

of the system and how it will operate in practice. 

A Planning Guide for Implementing an Early Intervention System

Chapter 5

Key Issue: EI Systems are very complex. It is better to take

the time to get it right than to rush and create a system

with unresolved issues.
Sergeants and any others who will be involved in interventions

need to be trained in how to conduct intervention sessions.

Lieutenants, captains, and other higher-ranking officers need to be

trained regarding their responsibilities in the EI system.

STEP THIRTEEN: Choose Software 

The planning committee should begin exploring software at the

earliest moment. There are several commercial software packages

on the market. A number of departments have developed their

own software. The decision regarding software is an issue where

the use of outside experts can be particularly valuable.

STEP FOURTEEN: Finalize the Plan

The last stage of the planning process should involve a final

meeting of the planning committee to ensure that all issues have

been addressed to the satisfaction of all members and to approve

the final plan. The committee should review the final draft of the

General Order and direct that it be sent to the chief executive for

sign-off and official implementation of the EI system.

STEP FIFTEEN: Implement the EI System

Implement the Early Identification system.

106

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 5

Key Issue:Train all officers in advance regarding their

specific responsibilities in the EI system.

Key Issue: Carefully consider the alternatives with regard

to software programs. 
Policy and Research Needs for Early

Intervention Systems

I. Current Issues Regarding EI Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .109

II. The Future: An Action Agenda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111

Chapter Six

I. Current Issues Regarding EI Systems

(1) What are the appropriate performance indicators?

While there is a strong consensus of opinion that an EI system

should not rely on a single performance indicator (such as citizen

complaints) there is no consensus regarding the exact number that

should be used. There is, however, a strong consensus of opinion

that certain indicators, such as use-of-force reports, citizen

complaints, etc., should always be among those indicators chosen. 

The question of the appropriate number of indicators raises a more

fundamental issue of the size and scope of an EI system. Large

and complex EI systems have the capacity for comprehensive

personnel assessment. They can identify top-performing officers

as well as poor performers. Such systems, however, are not only

expensive but pose a number of management challenges in terms

of computer infrastructure, data entry, and ongoing management

E

arly Intervention (EI) systems are presently still evolving. After a number

of years of development, law enforcement agencies now have

considerable experience with developing and operating EI systems. Even

the best systems, however, continue to evolve as the managers responsible for

them attempt to improve them. While there is now a rough consensus of

opinion within the field regarding the basic elements of an effective EI

system, many important questions remain unanswered. 

The purpose of this chapter is to discuss the most important questions

regarding EI systems and to suggest the appropriate policy and research

needs.

Policy and Research Needs for Early

Intervention Systems

6

109
oversight. A system that is not managed properly will not be

effective, no matter how sophisticated it is in theory. Smaller EI

systems that focus on officers with performance problems and

utilize fewer performance indicators cannot do as much in terms

of personnel assessment, but are far easier to create and manage. 

Further research and evaluation is needed with regard to the mot

effective size and scope of an EI system, and the related question

regarding the proper number of performance indicators. 

(2) What are the appropriate thresholds for identifying

officers for intervention?

Several different systems of thresholds for identifying officer for

possible intervention are currently in use. They include

department-level thresholds, peer-officer thresholds, and ratios

among different indicators (See Chapter Two). It is not clear at

this point which formula is the best.

Further research and evaluation is needed regarding the

effectiveness of different systems of thresholds for identifying

officers for possible intervention.

(3) What is the best procedure for ensuring that

supervisors responsible for conducting intervention

sessions perform this task in a manner consistent with

departmental goals?

The intervention component of an EI system is the crucial part of

the attempt to improve an officer's performance. An EI system

involves a shift in the role of supervisors and the development of

skills that many supervisors may not have or have been trained

for.

Additional research and evaluation is needed of different

approaches for training supervisors for their new roles in an EI

system.

110

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 6
111

(4) What are the most effective kinds of intervention? 

It is likely that some forms of intervention are more effective than

others for certain kinds of performance problems. Little attention

has been given to this issue.

Further research and evaluation should be given to developing

more effective ways of tailoring specific interventions to

particular officer performance problems. 

(5) What is the best size and scope of EI systems for

small or medium-sized law enforcement agencies?

Almost all of the information about EI systems available at

present relates to large law enforcement agencies. 

Additional program development is needed with regard to EI

systems appropriate for small and medium-sized law enforcement

agencies. 

II. The Future: An Action Agenda

Policy Development

EI systems are developing all over America. Police managers

associated with even the most sophisticated systems in the

country continue to encounter new issues that need to be resolved.

There is at present a substantial body of experience with EI

systems and a cadre of police managers who have thought about

the issues they face. The law enforcement profession should draw

upon this experience and sponsor a series of activities designed to

further the development of EI systems.

Specifically, we recommend that:

(1) Law enforcement professional associations sponsor a

regular series of workshops and conferences specifically

dedicated to the development and management of Early

Intervention systems.

Policy and Research Needs for Early Intervention Systems

Chapter 6
Experience to date indicates that EI systems are extremely

complex administrative tools and, consequently, it is not

anticipated that the major issues will be resolved quickly.

Consequently, we recommend that the law enforcement

professions establish a regular series of workshops and

conferences specifically dedicated to issues surrounding EI

systems.143

(2) Law Enforcement professional associations undertake the

development of detailed recommended Standards for Early

Intervention Systems.

One purpose of the workshops and conferences should be to begin

the development of recommended Standards for Early

Intervention Systems. These would be very detailed and provide

guidance to the profession on such issues as performance

indicators, thresholds, and so on. It is expected that these

standards would evolve over time, as a better understanding of EI

systems develops. 

(3) In developing the Standards, law enforcement professional

associations need to specifically address the unresolved issues

discussed earlier in this chapter.

Research

Further research is need on the nature and impact of EI systems.

Consequently, we recommend that the federal government and

private funding agencies:

(4) Fund research on the effectiveness of EI systems in

reducing problematic performance among officers subject to

intervention.

(5) Fund research on the relative effectiveness of different EI

systems with regard to such issues as their performance

indicators and intervention strategies.

(6) Fund research on the relationship of EI systems to other

law enforcement accountability mechanisms.

112

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Chapter 6
113

The Intersection of Research and Policy

All too often, academic research is not effectively translated into

operational policy. To overcome this problem, we recommend that

the federal government and private funding agencies:

(7) Fund workshops bringing together police officials

responsible for policy development and academic researchers. 

Policy and Research Needs for Early Intervention Systems

Chapter 6

Appendix A ..................................................................................................117

Appendix B ..................................................................................................120 

Appendix C ..................................................................................................124 

Appendix D ..................................................................................................126

Appendix E...................................................................................................130

Appendix F...................................................................................................137

Reading List .................................................................................................148

Endnotes .....................................................................................................150

Appendices

41. TEAMS II shall contain information on the following matters: 

a. all non-lethal uses of force that are required to be reported in

LAPD "use of force" reports or otherwise are the subject of an

administrative investigation by the Department; 

b. all instances in which a police canine bites a member of the

public; 

c. all officer-involved shootings and firearms discharges, both

on-duty and off-duty (excluding training or target range

shootings, authorized ballistic testing, legal sport shooting

events, or those incidents that occur off-duty in connection

with the recreational use of firearms, in each case, where no

person is hit by the discharge); 

d. all other lethal uses of force;

e. all other injuries and deaths that are reviewed by the LAPD

Use of Force Review Board (or otherwise are the subject of an

administrative investigation); 

f. all vehicle pursuits and traffic collisions; 

Appendix A

Consent Decree, United States v. Los Angeles (2001).

The entire consent decree can be found on the 

U.S. Department of Justice website: www.usdoj.gov/crt/split.

Appendices

117
g. all Complaint Form 1.28 investigations; 

h. with respect to the foregoing clauses (a) through (g), the

results of adjudication of all investigations (whether criminal

or administrative) and discipline imposed or non-disciplinary

action taken; 

i. all written compliments received by the LAPD about officer

performance; 

j. all commendations and awards; 

k. all criminal arrests and investigations known to LAPD of, and

all charges against, LAPD employees; 

l. all civil or administrative claims filed with and all lawsuits

served upon the City or its officers, or agents, in each case

resulting from LAPD operations, and all lawsuits served on an

officer of the LAPD resulting from LAPD operations and

known by the City, the Department, or the City Attorney's

Office; 

m. all civil lawsuits against LAPD officers which are required to

be reported to the LAPD pursuant to paragraph 77; 

n. all arrest reports, crime reports, and citations made by officers,

and all motor vehicle stops and pedestrian stops that are

required to be documented in the manner specified in

paragraphs 104 and 105; 

o. assignment and rank history, and information from

performance evaluations for each officer; 

p. training history and any failure of an officer to meet weapons

qualification requirements; and

q. all management and supervisory actions taken pursuant to a

review of TEAMS II information, include non-disciplinary

actions. 

118

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
119

TEAMS II further shall include, for the incidents included in the

database, appropriate additional information about involved

officers (e.g., name and serial number), and appropriate

information about the involved members of the public (including

demographic information such as race, ethnicity, or national

origin). Additional information on officers involved in incidents

(e.g., work assignment, officer partner, field supervisor, and shift

at the time of the incident) shall be determinable from TEAMS II.

Appendix A

Appendices
Appendix B

Los Angeles Sheriffs Department, Personnel Performance

Index (PPI),

SHERIFF'S BULLETIN #443 June 3,1997 

Performance Review

The purpose of this bulletin is to explain the performance review

process, 

Background

Over the past years, I have consistently expressed my belief that

the Department's most important resource is our employees.

Performance review and follow-up is one of the most significant

ways that the Department demonstrates its commitment to

supporting its employees. 

The tasks of reviewing, assessing, and guiding subordinate

performance are not new ones. They have always been, and will

continue to be, supervisory responsibilities. However, the

structure and procedures that comprise the career performance

review process that is the subject of this bulletin are new. having

been implemented in the early part of 1996. At that time each

division contribute a one supervisor to create a performance

review staff. In addition, the eight Commanders who were serving

on the Executive Risk Management Committee were designated

to serve as members of the Performance Review Committee. 

Identification of Reviewees

There is more than one method of determining which personnel

are to be selected for review. One method starts with an

examination of summary performance information in the Personal

Performance Index (PPI). 

The PPI facilitates the identification of employees who have been

involved in disproportionate numbers of (and/or very serious) risk

120

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
121

incidents. Another method of identifying candidates to be

reviewed involves supervisors' and managers' recommendations

based on long-term observations and knowledge about their

subordinates' work. 

Performance Review Report 

Upon designation of a candidate, the performance review staff

collects virtually all the performance documentation available

from the beginning of the employee's career. The material

assembled includes photocopies of performance evaluations,

commendations, notifications of discipline, and time records, as

well as the detailed documentation on the risk incidents of interest

–shootings, uses of force, public complaints, administrative

investigations, preventable traffic collisions, civil claims, and

lawsuits.

The staff member then compiles a profile report recounting the

individuals career performance history as reflected in the

documentation. The report and the accompanying documentation

form the basis for the review process as conducted by the

Performance Review Committee Commanders. 

Performance Review Committee 

The Commanders who comprise the Performance Review

Committee meet in rotating groups of three members

approximately twice per month as the profile repor1s are

completed. The group is chaired by the Commander assigned to

the Office of Administrative Services, who is the only

Commander who does not rotate. The meetings are also attended

by the Depar1ment Training Administrator, representatives from

Employee Relations! Advocacy Services, Risk Management

Bureau suppor1 staff, and the employee's Unit Commander. 

The committee members are furnished copies of the profile report

prior to the scheduled date of the review. At the meeting, the

committee evaluates the contents of the profile report and the

compiled documentation and decides whether it is necessary to

develop a plan for guiding the employee's future performance. 

Not every employee who is selected for a performance review is

Appendix B

Appendices
deemed to require a performance guidance plan. The committee

carefully assesses each employee's situation and mayor may not

conclude that a reviewed individual presents a performance risk to

himself/herself or to the Department. 

Formation of Performance Plan 

In cases where it is determined that an employee performance

guidance plan is appropriate, the committee Commanders, along

with the Unit Commander, confer to develop a plan. Elements of

plans may range from discussions with the employee. more

training, or temporary work reassignments up to, in extreme

cases, fitness for duty evaluations. 

One feature that is an element of each plan is the Unit

Commander's in-person discussion with the employee. explaining

the concerns about his/her performance as it impacts both the

employee and the Department. Employees are entitled to review

the profile report and the documentation upon which the

committee's decision was based. 

Review of Management Personnel And Systems 

Reviewing an individual employee's performance is only one

objective of this process. Another goal is to assess the

effectiveness of the management/supervisory systems and

personnel at the current and previous units of assignment. In some

cases, it has been determined that a supervisor's or manager's

responsibility to assist an employee in improving performance has

not been fulfilled. In these cases, the review committee

Commanders contact the concerned supervisors and/or managers

to counsel them regarding these supervisorial issues and reinforce

the message that genuine concern for subordinate welfare is

demonstrated by taking the time and energy to be honest,

supportive, and helpful. 

122

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
123

Results of Performance Review 

As of April 1997, 144 employees have been the recipients of

performance reviews–fewer than two percent of the Department's

sworn personnel. Only two people have been recommended for

evaluations of fitness for duty. 

Conclusion

The performance review process is not disciplinary in nature. It

was developed and implemented to provide a fair and responsible

system for improving or enhancing the employee's performance.

Emphasis is placed on the concern for correcting the employee's

work performance, as opposed to administering discipline. The

process benefits a variety of individuals, including the reviewee as

well as his or her current and previous supervisors and managers.

It integrates concern for individual employee interest with good

stewardship of the Department's collective welfare and the

employee's future well being. 

Appendix B

Appendices
Appendix C

Miami-Dade Police Department, Employee Identification

System (EIS) 

(1) systematic review of complaints and use of Force incidents by

sworn/nonsworn personnel 

2) Identify stress or performance related problems 

c. Report. composition 

1) Quarterly/Annual Report criteria 

a) Two or more personnel complaints 

or: 

b) Involvement in three or more Use of Force incidents 

c) Annual Report – Employee identified in two or more

quarters; four or more complaints, seven or more

Supervisor's Use of Force Report to Control, or two or

more shooting incidents in twelve months 

2) Report format 

a) Brief profile of complaints and Use of Force Incidents: 

(1) Complaints profile will include employee's name,

badge number, complainant's name, nature of

complaint, and disposition.

(2) Use of Force profiles will include employee's

name, badge number, subject's name, date of

incident, nature of incident, extent of injury. 

d. Report Disposition 

1) Report disseminated from Director's Office to employee's

chain of command 

124

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
125

2) Summary of supervisory review will be submitted to

Internal Review Bureau within 30 days 

3) Action alternatives 

a) Referral to Psychological services Program for counseling

or referral assistance 

b) Participation in stress Abatement Program for training

assistance (40 hours) 

c) Corrective action by supervisory personnel; counseling,

discipline, termination 

d) Assessment that no problem exists, terminating further

action 

Appendix C

Appendices
Appendix D 

Risk Management System, Cincinnati Police Department 

Memorandum of Understanding, U.S. Department of Justice

and the City of Cincinnati (2002)

VII. Management and Supervision

A. Risk Management System

57. The CPD will enhance and expand its risk management

system to include a new computerized relational database for

maintaining, integrating, and retrieving data necessary for

supervision and management of the CPD. Priority will be given to

the CPD obtaining any established program and system. The CPD

will regularly use this data to promote civil rights and best police

practices; to manage risk and liability; and to evaluate the

performance of CPD officers across all ranks, units and shifts. 

58. The new risk management system will collect and record the

following information: 

(a.) all uses of force;

(b.)canine bite ratios; 

(c.) the number of canisters of chemical spray used by officers; 

(d.)all injuries to prisoners; 

(e.) all instances in which force is used and a subject is charged

with "resisting arrest," "assault on a police officer,"

"disorderly conduct," or "obstruction of official business;"

(f.) all critical firearm discharges, both on-duty and off-duty; 

(g.)all complaints (and their dispositions);

(h.)all criminal proceedings initiated, as well as all civil or

administrative claims filed with, and all civil lawsuits served

126

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
127

upon, the City, or its officers, or agents, resulting from CPD

operations or the actions of CPD personnel; 

(i.) all vehicle pursuits; 

(j.) all incidents involving the pointing of a firearm (if any such

reporting requirement is imposed by paragraph 25); and 

(k.)all disciplinary action taken against officers. 

59. The new risk management system will include, for the

incidents included in the database, appropriate identifying

information for each involved officer (e.g., name, badge number,

shift and supervisor) and civilian (e.g., race, ethnicity or national

origin). 

61. The CPD will, within 90 days, prepare for the review and

approval of DOJ, and thereafter implement, a protocol for using

the risk management system. The City will submit for the review

and approval of DOJ all proposed modifications to the protocol

prior to implementing such modifications. 

62. The protocol for using the risk management system will

include the following provisions and elements: 

(a.) The protocol is comprised of the following components: data

storage, data retrieval, reporting, data analysis, pattern

identification, supervisory assessment, supervisory

intervention, documentation and audit. 

(b.)The protocol will require the automated system to analyze the

data according to the following criteria: i) number of incidents

for each data category by individual officer and by all officers

in a unit; ii) average level of activity for each data category by

individual officer and by all officers in a unit; and iii)

identification of patterns of activity for each data category by

individual officer and by all officers in a unit. 

(c.) The protocol will require the system to generate reports on a

Appendix D

Appendices
monthly basis describing the data and data analysis and

identifying individual and unit patterns. 

(d.)The protocol will require that CPD commanders, managers,

and supervisors will review, on a regular basis but not less

than quarterly, system reports, and will evaluate individual

officer, supervisor, and unit activity. 

(e.) The protocol will require that CPD commanders, managers,

and supervisors initiate intervention for individual officers,

supervisors and for units based on appropriate activity and

pattern assessment of the information contained in the risk

management system. 

(f.) The protocol will require that intervention options include

discussion by commanders, managers, supervisors, and

officers; counseling; training; and supervised, monitored, and

documented action plans and strategies designed to modify

activity. All interventions will be documented in writing and

entered into the automated system (appropriate intervention

options will be employed based on the evaluation described in

subsection (e) above). 

(g.)The protocol will specify that actions taken as a result of

information from the risk management system be based on all

relevant and appropriate information, including the nature of

the officer's assignment, crime trends and crime problems, and

not solely on the number or percentages of incidents in any

category of information recorded in the risk management

system. 

(h.)The protocol will require that the system's data be accessible

to CPD commanders, managers, and supervisors.

Commanders, managers, and supervisors will promptly review

records of all officers recently transferred to their sections and

units. 

(i.) The protocol will require that CPD commanders, managers,

and supervisors be evaluated on their ability to use the risk

management system to enhance effectiveness and reduce risk. 

128

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
129

(j.) The protocol will require that the system be managed and

administered by the Inspections Unit of the CPD. The

Inspections Unit of the CPD will conduct quarterly audits of

the system to ensure action is taken according to the process

described above. 

(k.)The protocol will require regular reviews, at no less than

quarterly intervals, by appropriate managers of all relevant

risk management system information to evaluate officer

performance city wide, and to evaluate and make appropriate

comparisons regarding the performance of all CPD units in

order to identify any significant patterns or series of incidents. 

Appendix D

Appendices
Appendix E

Consent Decree, United States v. State of New Jersey (1999).

Supervisory Measures to Promote Civil Rights Integrity 

Management Awareness Program 

40. The State shall develop and implement computerized systems

for maintaining and retrieving information necessary for the

supervision and management of the State Police to promote

professionalism and civil rights integrity, to identify and modify

potentially problematic behavior, and to promote best practices

(hereinafter, the "Management Awareness Program" or "MAP"). 

41. The MAP shall consist of the following information: 

a. all items of information in connection with all motor vehicle

stops that are required to be recorded in a written report, form,

or log, or reported to the communications center, pursuant to

29 and the protocols listed in 29of this Decree, except that

duplicate information need not be entered, and information as

to whether the incident was recorded with MVR equipment

need not be entered if all patrol cars are equipped with MVR

unless a patrol car was equipped with MVR equipment that

was not functioning; 

b. information on civilian compliments and other indicia of

positive performance; information on misconduct

investigations reports on use of force associated with motor

vehicle stops; on-duty and off-duty criminal arrests and

criminal charges; civil suits involving alleged misconduct by

state troopers while on duty; civil suits in which a trooper is

named as a party involving off-duty conduct that alleges racial

bias, physical violence or threats of violence; and 

c. implementation of interventions; and training information

including the name of the course, date started, date completed

and training location for each member receiving training. 

130

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
131

42. All information in MAP on substantiated misconduct

investigations, civilian compliments, and other indicia of positive

performance which can be attributed to a specific trooper shall be

made available to that trooper on an annual basis upon written

request. Nothing in this paragraph shall be construed as granting

that trooper access to confidential documents other than those

identified in this paragraph, or to any information which cannot

be attributed to the trooper requesting the information. 

43. Regarding the motor vehicle stop information identified in 29

(a) (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, and

19) and recorded in accordance with the protocols identified in

29(a), the MAP shall have the capability to search and retrieve

numerical counts and percentages for any combination of the

above-referenced information and to run reports for different time

periods (e.g., monthly, quarterly, annually) and for individual

troopers, squads, and stations. Regarding the motor vehicle stop

information identified in 29(a)(5A, 8A, 12A, 13A, 14A, 15A, and

17A) and recorded in accordance with the protocols identified in

29(a), it will be sufficient that the MAP shall have the capability

to access (through cross-referenced paper documents or other

method) this descriptive information entered on specific incidents

and matters. Regarding the information identified in 41(b and c),

to the extent technologically feasible, the MAP shall be developed

to have the capability to search and retrieve numerical counts and

percentages for any combination of the information and to run

reports for different time periods and for individual troopers,

squads or stations. To the extent that the MAP shall require

textual or narrative descriptions of misconduct allegations or other

information identified in 41(b and c), it will be sufficient that the

MAP only have the capability to retrieve this descriptive

information. 

44. Where information about a single incident is included within

the MAP from more than one document the State shall use a

common control number or other means to link the information

from different sources so that the user can cross-reference the

information and perform analyses. 

Appendix E

Appendices
45. The State shall ensure that information is included within the

MAP in an accurate and timely fashion and is maintained in a

secure manner. 

46. Within one hundred and eighty (180) days following entry of

this Decree, the State shall develop a plan for designing and

implementing the MAP including the use of the MAP, a timetable

for implementation, and a specification of the information

contained in State records pre-dating the implementation of the

MAP that can reasonably be incorporated in the MAP. Prior to

effectuating the implementation plan, the plan shall be approved

by the United States and the Independent Monitor. Within 180

days following the entry of this Decree, the State shall begin

conducting the supervisory and management reviews required by

48-53. 

Supervisory and Management Reviews 

47. Consistent with the requirements of 48-53 infra, the State

shall develop a protocol specifying the manner in which

supervisory and management reviews of individual state troopers,

and State Police units and sub-units (e.g., troops, stations, and

squads), shall be conducted, and the frequency of such reviews.

Prior to implementation, the protocol shall be approved by the

United States and the Independent Monitor. 

48. At least quarterly, State Police supervisors shall conduct

reviews and analyses of data obtained from the MAP and other

appropriate sources to ensure that individual troopers and State

Police units and subunits are performing their duties in accord

with the provisions of this Decree and associated protocols. 

49. To the extent reflected in 43, reports of MAP data shall

regularly be prepared regarding individual troopers, stations and

squads, for use in reviews as appropriate. The reports shall

include the following information: 

a. the number of motor vehicle stops, by race/ethnicity, reason

for the stop (i.e., moving violation, non moving violation,

other), road, squad, and trooper station; and the number of

132

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
133

enforcement actions and procedures taken in connection with

or during the course of a motor vehicle stop, by race/ethnicity,

reason for the stop (i.e., moving violation, non- moving

violation, other), road, squad and trooper station; 

b. data (including racial/ethnic data) on complaints, misconduct

investigations (for each type of investigation, as delineated in

73), discipline, intervention, and uses of force associated with

motor vehicle stops. 

50. To the extent reflected in 43,analyses of MAP data concerning

motor vehicle stops shall include a comparison of racial/ethnic

percentages of motor vehicle stops (by reason for the stop (i.e.,

moving violation, non moving violation, other)) and racial/ethnic

percentages of enforcement actions and procedures taken in

connection with or during the course of such stops, with a

benchmark racial/ethnic percentage if available (see 54-55); a

comparison of racial/ethnic percentages for such stops with the

racial/ethnic percentages for enforcement actions taken in

connection with or the during the course of such stops; a

comparison of racial/ethnic percentages for consent searches of

vehicles, and requests for consent to search vehicles, with "find"

rates by race/ethnicity for motor vehicle consent searches; a

comparison of racial/ethnic percentages for non-consensual

searches of motor vehicles with "find" rates by race/ethnicity for

motor vehicle non-consensual searches; evaluations of trends and

differences over time; and evaluations of trends and differences

between troopers, units, and subunits. 

51. To the extent reflected in 43,analyses of other data generated

by the MAP shall include evaluations of trends and differences

over time and evaluations of trends and differences between

troopers, units, and subunits. 

52. Each supervisor shall, consistent with his or her authority,

implement any appropriate changes or remedial measures

regarding traffic enforcement criteria, training, and enforcement

practices for particular units or subunits or implement any

appropriate intervention for particular troopers; conduct any

Appendix E

Appendices
necessary additional assessment or investigation regarding

particular units or subunits or particular troopers; and/or make any

appropriate recommendations. 

53. A supervisory review shall be conducted regarding any state

trooper who within a period of two years, is the subject of three

misconduct investigations of any kind initiated pursuant to 73.

Where appropriate, the review may result in intervention being

taken. In the event the supervisory review results in intervention,

the supervisor shall document the nature, frequency, and duration

of the intervention. 

54. To assist in evaluating data reported from the MAP

concerning State Police law enforcement on the New Jersey

Turnpike, the State shall develop (for purposes of implementing

this Decree) a protocol for conducting a survey of a sample of

persons and vehicles traveling on the New Jersey Turnpike to

determine the racial/ethnic percentage of drivers on the Turnpike.

As appropriate, the survey may identify different benchmark

figures for different portions of the Turnpike. Prior to

implementation, the protocol shall be approved by the

Independent Monitor and the United States. The protocol shall be

developed and implemented using a consultant jointly selected by

the parties. The survey shall be completed within one hundred

fifty (150) days of the entry of this Decree. Both the United States

and the State agree that the utility and fairness of the MAP

described in this Consent Decree will depend to some degree on

the development of accurate and reliable benchmarks that account

for all appropriate variables and factors. 

55. For purposes of implementing this Decree, the State may in its

discretion conduct a survey of a sample of persons and vehicles

traveling on any other limited access highway in New Jersey to

determine the racial/ethnic percentage of drivers on that road. In

deciding whether to conduct such a survey, the State shall give

due consideration to any recommendations made by the United

States and the Independent Monitor. The survey shall be

conducted pursuant to a protocol developed by the State and

approved by the Independent Monitor and the United States. The

protocol shall be developed and implemented using a consultant

jointly selected by the parties. 

134

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
135

56. The parties agree that for purposes of developing appropriate

benchmark racial/ethnic percentages, it will be sufficient for the

surveys described by 54 and 55 to determine the racial/ethnic

percentages of persons and vehicles traveling on the subject

roadway. The provisions of 54 and 55 shall not apply to any

traffic survey conducted by the State for purposes of advocating a

position in any criminal case or any civil litigation other than in

the instant case. 

Appendix E

Appendices

Office of Community Oriented Policing Services

U.S. Department of Justice

Appendix F

Appendices

tear along dotted line

Early Intervention Systems for Law

Enforcement Agencies: A Planning

and Management Guide

Prepared by

Samuel Walker, PhD

Department of Criminal Justice

University of Nebraska at Omaha

COPS COMMUNITY ORIENTED POLICING SERVICES

U.S. DEPARTMENT OF JUSTICE

Appendix F

• Data-based management information system.

• Capacity for identifying and correcting performance problems.

• Recommended police accountability "best practice"

• Separate from the formal disciplinary system

• Consistent with the goals of Ccommunity Ppolicing

• Consistent with the process of Pproblem-Ooriented Ppolicing

• Rely on systematic and timely data

• Consistent with the process of COMPSTAT

• Careful planning is needed for development and implementation

EI Systems: An Overview

Office of Community Oriented Policing Services

U.S. Department of Justice

Appendix F

Appendices

tear along dotted line

Early Intervention Systems for Law

Enforcement Agencies: A Planning

and Management Guide

Prepared by

Samuel Walker, PhD

Department of Criminal Justice

University of Nebraska at Omaha

COPS COMMUNITY ORIENTED POLICING SERVICES

U.S. DEPARTMENT OF JUSTICE

The Individual Officer

• Improved Performance

• Higher Standards of Accountability

Supervisors

• Improved Supervisory Practices

• Higher Standards of Accountability as Supervisors

The Department

• Higher standards of accountability

• Reduction in litigation

• Improved Community Relations

Goals and Impacts of EI Systems

Office of Community Oriented Policing Services

U.S. Department of Justice

Appendix F

Appendices

tear along dotted line

Early Intervention Systems for Law

Enforcement Agencies: A Planning

and Management Guide

Prepared by

Samuel Walker, PhD

Department of Criminal Justice

University of Nebraska at Omaha

COPS COMMUNITY ORIENTED POLICING SERVICES

U.S. DEPARTMENT OF JUSTICE

• Performance Indicators

• Identification and Selection

• Intervention

• Post-Intervention Monitoring

Components of an EI System

Office of Community Oriented Policing Services

U.S. Department of Justice

Appendix F

Appendices

tear along dotted line

Early Intervention Systems for Law

Enforcement Agencies: A Planning

and Management Guide

Prepared by

Samuel Walker, PhD

Department of Criminal Justice

University of Nebraska at Omaha

COPS COMMUNITY ORIENTED POLICING SERVICES

U.S. DEPARTMENT OF JUSTICE

Comprehensive Personnel Assessment Model

1. All non-lethal uses of force

Citizen injuries requiring medical attention 

Disciplinary outcomes

2. All officer-involved shooting incidents

On-duty/off-duty discharges, citizen injuries, citizen fatalities

Disciplinary outcomes

3. All officer-involved vehicular pursuits

Accidents

Injuries

Fatalities

Disciplinary outcomes

4. All citizen-initiated complaints

Use of force

Racial bias

Investigative outcomes

5. All citizen-initiated commendations or compliments

6. All departmental commendations and awards

7. Criminal arrests and investigations of subject officer

8. Civil suits or an administrative claim in which subject

officer is named

Nature of claims

Case outcomes

Performance Indicators

Office of Community Oriented Policing Services

U.S. Department of Justice

Appendix F

Appendices

tear along dotted line

Early Intervention Systems for Law

Enforcement Agencies: A Planning

and Management Guide

Prepared by

Samuel Walker, PhD

Department of Criminal Justice

University of Nebraska at Omaha

COPS COMMUNITY ORIENTED POLICING SERVICES

U.S. DEPARTMENT OF JUSTICE

9. All arrest reports, crime reports, and citations made by

officers

10. All motor vehicle stops and pedestrian stops made by

subject officer

Race, ethnicity, gender, age of drivers stopped

11. Performance evaluations for each officer

12. Training history 

Any failure of an officer to meet weapons qualification

requirements

13. All management and supervisory actions taken

pursuant to a review of TEAMS II information, including

non-disciplinary actions

14. Sick leave or family leave record

15. Canine unit deployment involving subject officer

Deployments

Bites

Citizen injuries

Complaints or civil suits arising from deployments

16. Failure to appear in court

Performance Indicators (continued)

Office of Community Oriented Policing Services

U.S. Department of Justice

Appendix F

Appendices

tear along dotted line

Early Intervention Systems for Law

Enforcement Agencies: A Planning

and Management Guide

Prepared by

Samuel Walker, PhD

Department of Criminal Justice

University of Nebraska at Omaha

COPS COMMUNITY ORIENTED POLICING SERVICES

U.S. DEPARTMENT OF JUSTICE

64%

32%

4%

Positive Mixed Negative

0%

10%

20%

30%

40%

50%

60%

70%

49%

28%

0%

23%

Positive Mixed Negative No Impact

0%

10%

20%

30%

40%

50%

General Assessment of EI System

Impact of EI System on Quality

of Police Service
Reading List

Christopher Commission, Report of the Independent Commission

on the Los Angeles Police Department. Los Angeles: Christopher

Commission, 1991.

Commission on Accreditation for Law Enforcement Agencies.

Standards for Law Enforcement Agencies. 4th ed. Fairfax, VA:

CALEA, 1999. www.calea.org

Robert C. Davis, et. al. Turning Necessity into Virtue: Pittsburgh's

Experience with a Federal Consent Decree. New York: Vera

Institute of Justice, 2002. Available at: www.vera.org.

International Association of Chiefs of Police, Building Integrity

and Reducing Drug Corruption in Police Departments.

Gaithersburg: IACP, 1989.

Kolts, James G., and Staff. The Los Angeles County Sheriff's

Department. Los Angeles: Los Angels County, 1992.

Los Angeles Police Commission. Report of the Rampart

Independent Review Panel. Los Angeles: Los Angeles Police

Commission, 2000.

Special Counsel Merrick J. Bobb, 16th Semiannual Report. Los

Angeles: Special Counsel, 2003. Available at: www.parc.info.

Walker, Samuel. "The New Paradigm of Police Accountability,"

St. Louis University Public Law Review, XII (March 2003):

3–52.

Walker, Samuel, Geoffrey P. Alpert and Dennis Kenney. Early

Warning Systems: Responding to the Problem Police Officer.

Research in Brief. Washington, DC: National Institute of Justice,

2001. Available on the web at www.ncjrs.gov. NCJ 188565.

Weisburd, David, Stephen Mastrofski, Ann Marie McNally, and

Rosann Greenspann, Compstat and Organizational Change:

Findings from a National Survey. Washington, DC: The Police

Foundation, 2001.

148

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
149

U.S. Commission on Civil Rights, Who is Guarding the

Guardians? A Report on Police Practices Washington, DC: U.S.

Civil Rights Commission, 1981.

U.S. Department of Justice, Principles for Promoting Police

Integrity. Washington: Department of Justice, 2001. Available on

the web at www.ncjrs.gov, NCJ # 186189.

Reading List

Appendices
Endnotes

1 Samuel Walker, Geoffrey P. Alpert, and Dennis Kenney.

2001. Early Warning Systems: Responding to the Problem

Police Officer. Research in Brief. (Washington, DC:

National Institute of Justice, 2001). Available on the web

at www.ncjrs.gov. NCJ 188565.

2 International Association of Chiefs of Police, Building

Integrity and Reducing Drug Corruption in Police

Departments (Washington, DC: Government Printing

Office, 1989), p. 80.

3 Frank Landy, Performance Appraisal in Police

Departments. (Washington, DC: The Police Foundation,

1977).

4 Bob Stewart, Comments, Focus Group Session, Nebraska

State Patrol, June 9, 2003.

5 Gary Stix, "Bad Apple Picker: Can A Neural Network

Help Find Problem Cops?" Scientific American

(December 1994): 44–45. Bernard Cohen and Jan M.

Chaiken, Police Background Characteristics and

Performance (Lexington, Mass., Lexington Books, 1973). 

6 Herman Goldstein, Police Corruption (Washington, DC:

The Police Foundation, 1975). 

7 It is significant, for example, that a recent National

Institute of Justice (NIJ) publication on developing

programs to deal with law enforcement officer stress

includes a section on "Selecting Target Groups" but

contains no reference to specific performance indicators

such as are commonly used in EI systems. Peter Finn and

Julie Esselman Tomz, Developing a Law Enforcement

Stress Program for Officers and Their Families

(Washington: Government Printing Office, 1997), pp.

23–26. Available on the web at www.ncjrs.gov.

NCJ163175.

150

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
151

8 David Weisburd, Stephen Mastrofski, Ann Marie McNally,

and Rosann Greenspann, Compstat and Organizational

Change: Findings from a National Survey (Washington,

DC: The Police Foundation, 2001). Eli B. Silverman,

NYPD Battles Crime: Innovative Strategies in Policing

(Boston: Northeastern University Press, 1999).

9 Robert L. Mathis and John H. Jackson, Human Resource

Management. Michael Poole and Malcolm Warner, The

IEBM Handbook of Human Resource Management

(London: International Thomson Business Press, 1998).

10 One large city on the West Coast the union demanded that

the EI system be subject to collective bargaining or meet

and confer. The issue remained unresolved at the time this

report was written. Nonetheless, this remains the only

known case of union opposition to date.

11 Walker, Alpert, and Kenney, Early Warning Systems:

Responding to the Problem Police Officer.

12 U.S. Department of Justice, Principles for Promoting

Police Integrity (Washington: Department of Justice,

2001). Available on the web at www.ncjrs.gov, NCJ #

186189.

13 Commission on Accreditation for Law Enforcement

Agencies, Standard 35.1.15. "Personnel Early Warning

System." (2001). Commission on Accreditation for Law

Enforcement Agencies, Standards for Law Enforcement

Agencies, 4th ed. (Fairfax, VA: CALEA, 1999).

www.calea.org

14 International Association of Chiefs of Police, Building

Integrity and Reducing Drug Corruption in Police

Departments. 

15 U. S. Commission on Civil Rights, Who is Guarding the

Guardians? (Washington DC: The United States

Commission on Civil Rights, 1981).

Endnotes

Appendices
16 The various consent decrees and memoranda of

understanding, along with other related documents, are

available at www.usdoj.gov/crt/split.

17 United States v. State of New Jersey, Consent Decree

(1999), Parag. 40. Available at www.usdoj.gov/crt/split.

18 United States v. Los Angeles, Consent Decree (2000),

Section II. Available at www.usdoj.gov/crt/split.

19 Department of Justice, Principles for Promoting Police

Integrity.

20 Robert C. Davis, Christopher Ortiz, Nicole J. Henderson,

Joel Miller and Michelle K. Massie, Turning Necessity

into Virtue: Pittsburgh's Experience with a Federal

Consent Decree (New York: Vera Institute of Justice,

2002), p. 37. Available at www.vera.org.

21 Samuel Walker, "The New Paradigm of Police

Accountability: The U.S. Justice Department 'Pattern or

Practice' Suits in Context," St. Louis University Public

Law Review, XXII (No. 1, 2003): 3–52.

22 Walker, Alpert, and Kenney, Early Warning Systems:

Responding to the Problem Police Officer.

23 Early Intervention System State of the Art Conference,

Report (Omaha, 2003). Available at

www.policeaccountability.org.

24 Davis, et al., Turning Necessity into Virtue, pp. 44 (quote),

43.

25 United States v. City of Los Angeles, Consent Decree, Sec.

II, Para. 47.

26 The Los Angeles and other consent decrees are available

on the U.S. Department of Justice website:

www.usdoj.gov/crt/split.

27 United States v. Cincinnati, 2002: Sec. VII. 62. i.

152

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
153

28 San Jose Independent Police Auditor, 2001 Year End

Report (San Jose: Independent Police Auditor, 2002), pp.

48–49.

29 This program is described in Special Counsel Merrick J.

Bobb, 16th Semiannual Report (Los Angeles: Special

Counsel, 2003), pp. 79–80.

30 United States v. State of New Jersey.

31 Unpublished Remarks. Early Intervention State of the Art

Conference (2003).

32 American Correctional Associatioin, Performance-Based

Standards for Adult Community Residential Services, 4th

ed. (Lanham, MD: American Correctional Association,

2000), Appendix B, p. 82.

33 Timothy N. Oettmeier and Mary Ann Wycoff, Personnel

Performance Evaluations in the Community Policing

Context (Washington, DC: The Police Foundation, 1997).

34 Geoffrey Alpert and Mark H. Moore, "Measuring Police

Performance in the New Paradigm of Policing," Bureau of

Justice Statistics, Performance Measures for the Criminal

Justice System (Washington: Government Printing Office,

1993), pp. 109–140.

35 Ibid.

36 Herman Goldstein, "Improving Policing: A ProblemOriented Approach," Crime and Delinquency, 25 (1979):

236–258. Tara O'Connor Shelley and Anne C. Grant, eds.,

Problem-Oriented Policing (Washington: Police Executive

Research Forum, 1998).

37 Davis, et al., Turning Necessity into Virtue, pp. 45–47.

38 Carol Archbold, Innovations in Police Accountability: An

Exploratory Study of Risk Management and Police Legal

Advising. (March 2002). Available at

www.policeaccountability.org.

Endnotes

Appendices
39 National Advisory Commission on Civil Disorders, Report

(New York: Bantam Books, 1968). NAACP, Beyond the

Rodney King Story (Boston: Northeastern University

Press, 1995). Human Rights Watch, Shielded From

Justice: Police Brutality and Accountability in the United

States (New York: Human Rights Watch, 1998).

40 William A. Westley, Violence and the Police (Cambridge:

MIT Press, 1970).

41 The argument is implicit in the long-standing

recommendations that police departments diversify their

work forces and actively recruit more racial and ethnic

minority officers. See, for example, National Advisory

Commission on Civil Disorders, Report, p. 315,

42 Catherine H. Milton, Women in Policing (Washington:

The Police Foundation, 1970).

43 This view is central to the movement for police

professionalism and the reform proposals that dominated

the professionalization from about 1900 to the present.

Samuel Walker, A Critical History of Police Reform

(Lexington: Lexington Books, 1977). 

44 On using peer officers to evaluate officers' performance,

see Special Counsel, 16th Semiannual Report, p. 74.

45 Samuel Walker, Internal Benchmarking for Traffic Stop

Data: An Early Intervention Approach. Discussion Paper.

Omaha, NE, 2003. Available at

www.policeaccountability.org.

46 Early Intervention State of the Art Conference, Report.

47 Dennis J. Kenney, Early Warning Systems: Responding to

the Problem Police Officer (Washington: Government

Printing Office, 2001), described a three-component

approach. The four-component approach used here reflects

the subsequent research. Available on the web at

www.ncjrs.gov. NCJ 188565.

154

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
155

48 Early Intervention System State of the Art Conference,

Report (Omaha, 2003). Available at

www.policeaccountability.org.

49 A good description of these two systems is available in

Special Counsel Merrick J. Bobb, 16th Semiannual Report

(Los Angeles: Special Counsel, 2003), pp. 68–72. The

report is available at www.parc.info.

50 Robert Davis, Christopher Ortiz, Nicole J. Henderson, Joel

Miller and Michelle K. Massie, Turning Necessity into

Virtue: Pittsburgh's Experience with a Federal Consent

Decree (New York: Vera Institute of Justice, 2002), pp.

44–45. Available at www.vera.org.

51 Samuel Walker, Police Accountability: The Role of Citizen

Oversight (Belmont: Wadsworth, 2001), pp. 121–135.

52 United States v. Los Angeles, Consent Decree (2001),

Section II. Available at www.usdoj.gov/crt/split.

53 An important object lesson on this point is in Special

Counsel Merrick J. Bobb, 16th Semiannual Report, which

found that because of staff cutbacks data were not being

entered into the systems in a timely fashion. The report is

available at www.parc.info.

54 U.S. Department of Justice and the City of Cincinnati,

Memorandum of Understanding, 2001. Available at

www.usdoj.gov/crt/split.

55 Special Counsel Merrick J. Bobb, 16th Semiannual Report,

pp. 71–72.

56 Early Intervention State of the Art Conference, Report.

57 Similar examples are offered in Samuel Walker, Internal

Benchmarking for Traffic Stop Data: An Early

Intervention System Approach. Discussion Paper (Omaha,

2003). Available at www.policeaccountability.org.

Endnotes

Appendices
58 This approach is described in Bobb, 16th Semiannual

Report, pp. 74–75.

59 This approach is described in ibid, pp. 68–69.

60 Walker, Alpert and Kenney, Early Warning Systems:

Responding to the Problem Police Officer.

61 Special Counsel Merrick J. Bobb, 15th Semiannual Report

(Los Angeles: Special Counsel, 2002), p. 39. Available at

www.parc.info.

62 Ibid, p. 42.

63 Davis, et. al., Turning Necessity into Virtue, p. 46.

64 Commission on Accreditation for Law Enforcement

Agencies, Standard 35.1.15. Personnel Early Warning

System.

65 Special Counsel Merrick J. Bobb, 15th Semiannual Report,

pp. 37–75.

66 New Orleans Police Department, Professional

Performance Enhancement Program (PPEP) (New

Orleans: NOPD, 1998).

67 Omaha Police Union, "Bad Boy/Girl Class Notes Shared."

The Shield. (April 1992). 

68 Early Intervention State of the Art Conference, Report.

69 San Jose Independent Police Auditor, 2001 Year End

Report (San Jose: Independent Police Auditor, 2002), pp.

48–49. Available at www.ci.san-jose.us. The San Jose

program is also described in Special Counsel Merrick J.

Bobb, 16th Semiannual Report, p. 79.

70 Davis, et. al., Turning Necessity into Virtue.

71 New Orleans Police Department, Professional

Performance Enhancement Program (PPEP).

156

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
157

72 Early Intervention State of the Art Conference, Report.

73 Comments, police chiefs, Justice Department Conference,

Strengthening Police Community Relationships, June 9-

10, 1999.

74 Herman Goldstein, Policing a Free Society (Cambridge:

Ballinger, 1977), p. 171.

75 Hans Toch, J. Douglas Grant, and Raymond T. Galvin,

Agents of Change: A Study in Police Reform (New York:

John Wiley, 1975).

76 U.S. Commission on Civil Rights, Who is Guarding the

Guardians? A Report on Police Practices (Washington,

DC: U.S. Civil Rights Commission, 1981), pp. 159, 166.

77 Christopher Commission, Report of the Independent

Commission on the Los Angeles Police Department (Los

Angeles, 1991). Available at www.parc.info.

78 Ibid., pp. 40–48.

79 James G. Kolts and Staff, The Los Angeles County

Sheriff's Department (Los Angeles: Los Angels County,

1992), pp. 157–167, 353. Available at www.parc.info.

80 Special Counsel Merrick J. Bobb, 11th Semiannual Report

(Los Angeles: Special Counsel, 1999), pp. 46–47.

Available at www.parc.info.

81 "Kansas City Police Go After Their 'Bad Boys'," The New

York Times (September 10, 1991).

82 "Wave of Abuse Laid to a Few Officers," Boston Globe

(October 4, 1992).

83 Sari Horowitz, "Officers Go Too Far," The Washington

Post (November 19, 1998), p. 1.

Endnotes

Appendices
84 Samuel Walker, Geoffrey P. Alpert, and Dennis J. Kenney,

Early Warning Systems: Responding to the Problem Police

Officer (Washington, DC: National Institute of Justice,

2001). NCJ 188565. Available at www.ncjrs.gov.

85 Catherine H. Milton, et. al., Police Use of Deadly Force

(Washington, DC: The Police Foundation, 1977). Walker,

Alpert, and Kenney, Early Warning Systems: Responding

to the Problem Police Officer.

86 U.S. Commission on Civil Rights, Who is Guarding the

Guardians?, pp. 81–86.

87 New York City, Commission to Investigate Allegations of

Police Corruption and the Anti-Corruption Procedures of

the Police Department [Mollen Commission], Commission

Report (New York, 1994).

88 Tony Pate, Jack W. McCullough, Robert A. Bowers, Amy

Ferrara, Kansas City Peer Review Panel: An Evaluation

(Washington, DC: The Police Foundation, 1976).

89 George L. Kelling, et al., The Kansas City Preventive

Patrol Experiment (Washington: The Police Foundation,

1974).

90 Pate, et al., Kansas City Peer Review Panel, pp. 9–10.

91 Ibid., p. 55.

92 "Kansas City Police Go After Their 'Bad Boys'," New

York Times (September 10, 1991).

93 Toch, Grant and Galvin, Agents of Change.

94 U. S. Commission on Civil Rights, Who is Guarding the

Guardians?.

95 John S. Ross, Memorandum to Chief Kenneth I. Harms,

"Citizen Complaints Against Police Officers," (May 29,

1979), p. 1.

158

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
159

97 Ibid., pp. 2–3.

98 Ibid., p. 7.

99 Ibid., p. 10.

100 Ibid., p. 12.

101 Miami Police Department, Departmental Order, #2,

Chapter 8.

102 Ibid.

103 "Miami Police Face Series of Inquiries After Shootings,"

The New York Times (July 18, 2001).

104 Bruce Porter, The Miami Riot of 1980 (Lexington:

Lexington Books, 1984). U.S. Commission on Civil

Rights, Confronting Racial Isolation in Miami

(Washington: Government Printing Office, 1982).

105 Metro [Miami]-Dade Police Department, Professional

Compliance Bureau, Early Identification System (internal

memorandum).

106 Walker, Alpert, and Kenney, Early Warning Systems:

Responding to the Problem Police Officer.

107 Ibid.,

108 Samuel Walker, Police Accountability: The Role of Citizen

Oversight (Belmont, CA: Wadsworth, 2001). 

109 U.S. Department of Justice. Conference on Police

Integrity. Report, 1997.

110 U.S. Department of Justice, COPS Office, Law

Enforcement Working Session on Police Practices,

November 29-30, 2000.

Endnotes

Appendices
111 U.S. Justice Department, Principles for Promoting Police

Integrity (Washington, DC: U.S. Justice Department,

2001). NCJ 186189. Available at www.ncjrs.gov.

112 United States v. Los Angeles, Consent Decree (2001).

Available at www.usdoj.gov/crt/split/.

113 United States v. Cincinnati, Memorandum of

Understanding (2002). Available at

www.usdoj.gov/crt/split.

114 Commission on Accreditation for Law Enforcement

Agencies (CALEA), Standard 35.1.15 (2001). Available at

www.calea.org.

115 The first published literature are the reports and articles

co-authored by the author of this report: Walker, Alpert,

and Kenney, Early Warning Systems. 

116 Ibid.

117 Special Counsel Merrick J. Bobb, 11th Semiannual Report,

p. 55.

118 A full account is available in Special Counsel Merrick J.

Bobb, 11th Semiannual Report, pp. 45–56. James G. Kolts,

The Los Angeles County Sheriff's Department (Los

Angeles: Los Angeles County, 1992).

119 Special Counsel Merrick J. Bobb, 11th Semiannual Report.

120 Special Counsel Merrick J. Bobb, 16th Semiannual Report,

pp. 43–59, 67–81.

121 The chronology of events is very well documented in Los

Angeles Police Commission, Report of the Rampart

Independent Review Panel (Los Angeles: Los Angeles

Police Commission, 2000), pp. 136–162 and Appendix B,

pp. 217–219.

122 Christopher Commission, Report of the Independent

Commission on the Los Angeles Police Department (Los

Angeles: City of Los Angeles, 1991).

160

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices
161

123 Merrick Bobb, Five Years Later: A Report to the Los

Angeles Police Commission (Los Angeles: Los Angeles

Police Commission, 1996).

124 Los Angeles Police Department, Board of Inquiry into the

Rampart Area Corruption Incident (Los Angeles: LAPD,

2000).

125 Los Angeles Police Commission, Report of the Rampart

Independent Review Panel.

126 Erwin Chemerinsky, An Independent Analysis of the Los

Angeles Police Department's Board of Inquiry Report on

the Rampart Scandal (Los Angeles, 2000). 

127 United States v. Los Angeles, Consent Decree (2000). The

consent decree is available on the DOJ website:

www.usdoj.gov/crt/split.

128 United States v. City of Pittsburgh, Consent Decree

(1997). Available at: www.usdoj.gov/crt/split/.

129 Discussed in Samuel Walker, "The New Paradigm of

Police Accountability: The U.S. Justice Department

'Pattern or Practice' Suits in Context," St. Louis University

Public Law Review, XXII (No. 1, 2003): 3–52.

130 Robert C. Davis, Christopher Ortiz, Nicole J. Henderson,

Joel Miller and Michelle K. Massie, Turning Necessity

into Virtue: Pittsburgh's Experience with a Federal

Consent Decree (New York: Vera Institute of Justice,

2002), p. 44. Available at www.vera.org.

131 City of Pittsburgh, Monitor's Eighteenth Quarterly Report.

Quarter Ending February 16, 2002.

132 Davis, et al., Turning Necessity into Virtue: Pittsburgh's

Experience with a Federal Consent Decree.

133 Davis, et al., Turning Necessity into Virtue, p. 15.

134 Walker, Alpert, and Kenney, Early Warning Systems.

Endnotes

Appendices
135 The evidence is summarized in Samuel Walker and

Charles M. Katz, The Police in America: An Introduction,

4th ed. (New York: McGraw-Hill, 2003), Chs. 13, 14.

136 National Center for Women and Policing, Men, Women

and Police Excessive Force: A Tale of Two Genders (Los

Angeles: NCWP, 2002). Available at www.feminist.org.

137 Davis, et. al., Turning Necessity into Virtue, p. 49-50.

138 Special Counsel Merrick J. Bobb, 15th Semiannual Report

(Los Angeles: Special Counsel, 2002), p. 68. Available at

www.parc.info.

139 Ibid., p. 64.

140 Special Counsel Merrick J. Bobb, 16th Semiannual Report.

141 Early Intervention State of the Art Conference (February

2003), available at www.policeaccountability.org.

142 Walker, Alpert, and Kenney, 2001

143 The first effort in this regard was the Early Intervention

System State of the Art Conference, held in Phoenix,

Arizona, in February, 2003. See the conference Report,

available at www.policeaccountability.org.

162

Early Intervention Systems for Law Enforcement Agencies: A Planning and Management Guide

Appendices

For More Information:

U.S. Department of Justice

Office of Community Oriented Policing Services

1100 Vermont AVenue, N.W.

Washington, D.C. 20530

To obtain details on COPS programs, call the 

U.S. Department of Justice Response Center at 800.421.6770

Visit COPS Online at the address listed below. 

Created: August 25, 2003 www.cops.usdoj.gov

CM#: e07032003

ISBN: 1-932582-26-6