U.S. Department of Justice 
Office of Community Oriented Policing Services 

Image of United States Department of Justice seal

COPS 

Early Intervention Systems for 
Law Enforcement Agencies: 
A Planning and Management Guide 


Early Intervention Systems for
Law Enforcement Agencies:

A Planning and Management Guide

Prepared for
Office of Community Oriented Policing Services
1100 Vermont Ave., N.W.
Washington, D.C. 20005

www.cops.usdo j.gov 

Prepared by
Samuel Walker, PhD
Department of Criminal Justice
University of Nebraska at Omaha

2003

This report was funded by Grant #2000-CK-WX-K124 from the Office of Community
Oriented Policing Services of the U.S. Department of Justice. The views expressed in
this report do not necessarily reflect to views or the official policies of the 
U.S. Department of Justice.

ISBN# 1-932582-26-6


TableofContents 

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .i
Acknowledgments . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .iii
Chapter 1. Introduction to Early Intervention Systems . . . . . . . . . . . . . .3
Chapter 2. The Components of an Early Intervention System . . . . . . . . . . . .27
Chapter 3. Background
: The Origins and Development of Early 
Intervention Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47
Chapter 4. Lessons From the Field
: Police Managers' Perceptions and
Experiences with Early Intervention Systems . . . . . . . . . . . . . . . . . . . . .77 
Chapter 5. Planning and Implementing an EI System . . . . . . . . . . . . . . . . . .99
Chapter 6. Policy and Research Needs for Early Intervention Systems . . . . .105
Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123

ExecutiveSummary 

 
Early Intervention (EI) systems are an effective mechanism for enhancing accountability 
within law enforcement agencies. 
 
Police managers experienced with EI systems report that they have a positive impact on 
officer performance and that they strengthen supervision. 
 
EI systems are extremely complex administrative tools that require close attention by 
police managers. 
 
In creating an EI system, careful decisions need to be made about the number of 
performance indicators to be used and the thresholds to be used to identify potential 
problem officers. 
 
For an EI system to be effective, sergeants and other supervisors need extensive training 
about their new responsibilities. 
 
EI systems are consistent with the goals of Community Policing and can help to improve 
police-community relations problems. 
 
Careful planning is essential in the development of an effective EI system. The key to the 
planning process is the involvement of officers from all ranks of the department. 
 
EI systems are a "work in progress." Existing systems need to engage in a process of 
continuous review and fine-tuning. 
 
This report focuses on EI in larger agencies: the exact size and scope of EI systems for 
small and medium-sized law enforcement agencies is not known at this time. 
 
The law enforcement profession needs to sponsor continuing policy development related 
to EI systems. 
 
Additional research is required on the impact and effectiveness of EI systems. 


Acknowledgments 

This report was supported by Grant #2000-CK-WX-K124 from the U.S. Department of Justice 

Office of Community Oriented Policing Services (COPS). The views expressed in this report do 
not necessarily reflect the views or official policies of the U.S. Department of Justice. 

I would like to express my special thanks to Dr. Ellen Scrivner of the COPS Office for her 
support of this effort and for her on-going commitment to the issue of police accountability. 
Pam Cammarata picked up the project after Dr. Scrivner left the COPS Office and has been 
very supportive of it over the course of the last year. 

Professor Geoffrey P. Alpert of the University of South Carolina served as a major consultant 
to this project, and I would like to express my thanks for all of his help. 

Dawn Irlbeck, my Research Assistant at the University of Nebraska at Omaha, made a 
tremendous contribution to this report. She had major responsibilities for developing and 
administering the PERF survey and for analyzing the results that are presented here in 
Chapter Four. She also proofread the final report and saved me many embarrassments. 
I would like to thank her for all of her work. 

Chapter Two and the Appendix report the findings of a survey administered in collaboration 
with DiscoverWhy and the Police Executive Research Forum (PERF). I would like to thank 
Dr. Lorie Fridell, Research Director at PERF, and both Brad Ashford and Linda Ordoukhanian at 
DiscoverWhy for agreeing to collaborate on this survey. 

This project also involved a series of workshops hosted by four Regional Community Policing 
Institutes (RCPIs) across the country. This included the RCPIs in Tampa, Florida; Baltimore, 
Maryland; Chicago, Illinois; and Sacramento, California. I would like to thank the directors 
and staff of those agencies for their assistance. Sergeant Ed Hudak of the Coral Gables (FL) 
Police Department assisted Professor Alpert and myself in those efforts. 

The author of this report has had the benefit of tremendous cooperation from officials at 
both the Los Angeles County Sheriff's Department (LASD) and the Phoenix Police Department. 
At the LASD I would like to thank Commander Neal Tyler and Commander William J. 
McSweeney. At the Phoenix Police Department I would like to thank Lieutenant Dan Olsen, 
Detective Toye Nash, and Sergeant Ron Snodgrass. 

The Phoenix Police Department hosted the EI System State of the Art Conference in February 
2003. I would like to thank Chief Harold Hurtt, Lieutenant Olsen, Detective Nash, and 
Sergeant Snodgrass for all of their help with that valuable event. 

The author of this report provided technical assistance to the Douglas County (NE) 
Department of Corrections in developing its Performance Enhancement Program (PEP) 
beginning in the year 2000. That experience provided valuable insights into one agency's 
planning process. 


Chapter One

Introduction to Early Intervention Systems

The Concept of Early Intervention . . . . . . . . . . . . . . . . . . . . . . . .3
A Word About Terminology. . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
ARecommended Best Practice in Police Accountability . . . . . . . . . . . . . . .8 
EI Systems and Other Best Practices. . . . . . . . . . . . . . . . . . . . . . .10
Goals and Impacts of EI Systems. . . . . . . . . . . . . . . . . . . . . . . . .10
Individual Officers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Supervisors  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
The Department . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Early Intervention and Community Policing. . . . . . . . . . . . . . . . . . . .16
Early Intervention and Problem-Oriented Policing . . . . . . . . . . . . . . . .17
Early Intervention and COMPSTAT. . . . . . . . . . . . . . . . . . . . . . . . .17 
Early Intervention and Risk Management . . . . . . . . . . . . . . . . . . . . .18
Early Intervention and Police-Community Relations  . . . . . . . . . . . . . . .18
Early Intervention and Racial Profiling. . . . . . . . . . . . . . . . . . . . .19
One Size Does Not Fit All Departments. . . . . . . . . . . . . . . . . . . . . .20
AWork in Progress. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22

Introduction to Early Intervention Systems

The Concept of Early Intervention 

An Early Intervention (EI) system is a data-based management tool designed 
to identify officers whose performance exhibits problems, and then to provide 
interventions, usually counseling or training, to correct those performance 
problems. EI systems have emerged as an important mechanism for ensuring 
police accountability.1 

EI systems do more than just focus on a few problem officers. As a 1989 
report by the International Association of Chiefs of Police (IACP) explains, an 
EI system is "a proactive management tool useful for identifying a wide range 
of problems [and] not just a system to focus on problem officers."2 

In an EI system, performance data are entered into a computerized 
database. These data include departmental use-of-force reports, 
citizen complaints, officer involvement in civil litigation, resisting 
arrest charges, and other performance indicators. Some current EI 
systems use a dozen or more performance indicators while others 
use a smaller number. 

An EI system is early in the sense that it helps to identify officer 
performance problems that do not warrant formal disciplinary 
action but suggest that an officer is having problems dealing with 
citizens. The major contribution of an EI system is its capacity to 
spot patterns of performance and to intervene before problems 
lead to a serious incident such as a lawsuit, a citizen complaint 
over excessive force, or some other public crisis involving the 
department. An EI system warns an officer to the extent that it 
sends an informal but nonetheless clear message that his or her 
performance needs improvement. 


An EI system is officially separate from the formal disciplinary 
system. It is designed to help officers improve their performance 
through counseling, training, or coaching. No record of 
participation in an EI program is placed in an officer's personnel 
file, although a separate record of participation is usually 
maintained by the internal affairs or professional standards unit. 
One or more of the incidents identified by an EI system may 
warrant formal disciplinary action that is officially recorded, but 
identification by the EI system remains separate from the 
disciplinary process. 

EI systems are data-driven mechanisms of accountability, 
providing systematic data as a basis for performance evaluations 
of officers. They differ from traditional performance review 
systems that rely heavily on subjective assessments such as 
"works well with people" or "demonstrates initiative."3 The 
database can identify specific areas of performance that need 
correcting (e.g., a pattern of citizen complaints alleging rudeness). 

Bob Stewart, former Executive Director of the National 
Organization of Black Law Enforcement Executives (NOBLE), 
says that "If I could choose only one accountability mechanism, it 
would be an early intervention system." Stewart has seen police 
performance problems from several perspectives: he was a police 
officer in Washington, DC, the chief of police in Ormond, Florida, 
and is now a consultant to communities and law enforcement 
agencies around the country. Based on this long experience, he 
now regards EI systems as the single most valuable tool for 
achieving accountability, primarily because of their capacity to 
monitor a wide range of officer activities and to spot performance 
problems at an early stage.4 

EI systems are retrospective performance reviews. They do not 
attempt to predict officer performance based on background 
characteristics or other factors, rather they indicate that current 
performance levels, while not warranting disciplinary action, still 
warrant improvement. Past efforts to develop a methodology for 
predicting which applicants for police employment will perform 
well and which are not likely to perform well have not proven 
successful.5 An EI system provides a basis for counseling or 


warning an officer that his or her performance needs to improve 
in the future, and in successful cases, for documenting that the 
improvement has occurred. 

EI systems are designed to help officers improve their 
performance. The names of several EI systems reflect this 
orientation. The New Jersey State Police EI system is known as 
the Management Awareness Program (MAP), indicating its 
purpose of assisting management. In this respect, EI systems 
represent a significant departure from traditional police 
disciplinary practices. Law enforcement agencies have been 
punishment-oriented bureaucracies, with innumerable rules and 
regulations that can be used to punish an officer, but with few 
procedures for either rewarding good conduct6 or helping officers 
with problems. Apart from employee assistance programs (EAP) 
designed to address substance abuse or family problems, police 
departments have done relatively little in a formal way to correct 
problem behavior.7 An EI system helps to identify specific 
performance problems that need to be addressed (e.g., a tendency 
toward verbal abusiveness, frequent charges of resisting arrest, 
etc.). 

EI systems represent a problem-solving approach to officer 
performance. The problem in this instance involves questionable 
officer performance. The problem-solving involves identifying 
officers in need of assistance and providing that assistance 
through counseling or training. (The EI problem-solving process 
is discussed in detail later in this chapter.) As one commander 
with EI system experience explained, his agency's EI system 
"provides a way for the department to provide non-disciplinary 
direction and training before the officer becomes a liability to 
citizens, the department, and him/herself" (See Chapter Four). 

EI systems are very similar to COMPSTAT programs. 
COMPSTAT is one of the most important innovations in police 
management. It is a data-based system designed to help law 
enforcement agencies respond effectively to crime and disorder 
and to hold their managers accountable for their performance.8 
Both EI systems and COMPSTAT programs rely on the analysis 
of systematic and timely datain the case of COMPSTAT the data 


involve crime and disorder, with EI systems the data involve 
individual officer performance. 

EI systems are consistent with the basic principles of personnel 
management and human resource development.9 Employers 
recruit, select, and train employees to effectively serve the goals 
and objectives of their organizations. Effective personnel 
management assumes that employee performance is assessed and 
evaluated on a regular basis, and that the organization takes steps 
to correct unsatisfactory performance. 

Contrary to the expectations of many people, EI systems have not 
encountered significant opposition from police unions 
representing rank and file officers.10 To be sure, there has usually 
been grumbling and fear of the unknown, but in practice unions 
have not succeeded in blocking the operation of an EI system 
once it is in place. The police managers' survey reported in 
Chapter Four of this report found that only 16 percent of 
managers had encountered serious opposition from police unions. 
Potential opposition from unions is best overcome by involving 
union representatives in the planning of an EI system (see Chapter 
Five). 

EI systems are consistent with the goals of community oriented 
policing (COP). Community policing creates demands for 
responsiveness to community residents and in the process new 
measures of police performance. The traditional measures of the 
crime rate and the clearance rate are no longer adequate. EI 
systems have the capacity to quickly document the performance 
of officers who are not effectively serving the community and 
provide a basis for remedial action by the department. 

The available evidence indicates that EI systems are successful in 
achieving their goals of reducing officer misconduct. An NIJ 
study of EI systems in three police departments found significant 
reductions in use-of-force and citizen complaints among officers 
following EI intervention.11 Commanders with EI system 
experience, meanwhile, are able to report specific examples of 
individual officers whose performance improved as a result of 
intervention. One commander, for example, has described an 


officer who had on-the-street problems because of an excessive 
fear of being struck in the face. Intervention counseling identified 
the problem and retraining in tactics helped the officer overcome 
the problem. Commanders report many other such success stories. 
In the survey of managers which is summarized in Chapter Four 
of this report, about half (49 percent) reported that the system has 
had a positive impact on the on-the-street performance of their 
officers, while almost a third (28 percent) reported a mixed 
impact. No commanders reported a negative impact. 

Chapter Three of this report discusses in detail the research 
findings regarding the effectiveness of EI systems. Chapter Four, 
meanwhile, reports the findings of a survey of police managers 
with EI system experience. Generally, managers have had very 
positive experiences with the impact of their systems of 
supervision and accountability. 

BOX 1.1 EI Systems: An Overview
 Data-based management information system. 
 Capacity for identifying and correcting performance problems. 
 Recommended police accountability "best practice" 
 Separate from the formal disciplinary system 
 Consistent with the goals of Ccommunity Ppolicing 
 Consistent with the process of Pproblem-Ooriented Ppolicing 
 Rely on systematic and timely data 
 Consistent with the process of COMPSTAT 
 Careful planning is needed for development and implementation 
 
A Word About Terminology 

Early Intervention 

This report uses the term early intervention. It does not use the 
more widely used term early warning. Early warning has a 
negative connotation, suggesting that the system is primarily 
oriented toward discipline. One department with a comprehensive 
EI system found through interviews with officers that they did not 
like the phrase early warning because of its "big brother" 


connotation. EI systems are evolving in the direction of more 
comprehensive personnel assessment systems, for the purpose of 
examining a broad range of performance issues. For this reason, 
the more positive early intervention term is used here. Other 
terms are also appropriate. These include Personnel Performance 
Index, as used by the Los Angeles County Sheriff's Department, 
or the Personnel Assessment System (PAS), as used by the 
Phoenix Police Department. 

Officers With Performance Problems 

This report uses the term officers with performance problems 
rather than the commonly used term problem officers. The latter 
terms unfairly labels officers and suggests that their performance 
cannot change. The term officers with performance problems 
focuses on behavior without labeling an officer and conveys the 
message that performance can improve. 

A Recommended Best Practice in Police 
Accountability 

EI systems have been recommended by a wide range of 
organizations as a best practice in police accountability. 

 
A January 2001 report by the U.S. Department of Justice 
(2001), Principles for Promoting Police Integrity, includes EI 
systems among its recommended best practices.12 
 
The Commission on Accreditation for Law Enforcement 
Agencies (CALEA, 2001) has adopted Standard 35.1.15 
mandating EI systems for large agencies. The CALEA Standard 
states that: 

A comprehensive Personnel Early Warning System is an 
essential component of good discipline in a well-managed law 
enforcement agency. The early identification of potential 
problem employees and a menu of remedial actions can 
increase agency accountability and offer employees a better 
opportunity to meet the agency's values and mission 
statement.13 


The CALEA Standard warns that a department should not be 
faced with a situation where an officer is alleged to have 
committed a serious act of misconduct where "there was an 
escalating pattern of less serious misconduct, which could have 
been abated through intervention." The Standard suggests but 
does not require the use of such performance indicators as use-of-
force incidents, citizen complaints, and disciplinary actions. 

 
A 1989 report by the International Association of Chiefs of 
Police (IACP) (1989:80) recommends EI systems as a means of 
controlling corruption and building integrity in police 
departments.14 
 
The U.S. Civil Rights Commission was the first agency to 
recommend EI systems, in its 1981 report, Who is Guarding the 
Guardians?15 
 
Since 1997, EI systems have been included in consent decrees 
and memoranda of understandings settling law suits brought by 
the Civil Rights Division of the U.S. Justice Department under 
the "pattern or practice" clause of the 1994 Violent Crime 
Control Act. EI systems are mandated in the agreements related 
to the Pittsburgh Police Bureau; the New Jersey State Police; 
the Metropolitan Police Department of Washington, D.C.; the 
Los Angeles Police Department; and the Cincinnati Police 
Department.16 

The consent decree settling the Justice Department suit against the 
New Jersey State Police, for example, requires the development 
of a Management Awareness Program (MAP) for the purpose of 
"maintaining and retrieving information necessary for the 
supervision and management of the State Police to promote 
professionalism and civil rights integrity, to identify and modify 
potentially problematic behavior, and to promote best 
practices..."17 The consent decree with the Los Angeles Police 
Department (LAPD) sets requirements for an EI system, known as 
TEAMS (Training Evaluation and Management System). The 
consent decree specifies seventeen categories of data to be 
collected and entered into the system.18 


BOX 1.2 EI Systems Recommended By:
 U.S. Department of Justice 
 U. S. Civil Rights Commission 
 Commission on Accreditation for Law Enforcement Agencies 
 International Association of Chiefs of Police 
 
EI Systems and Other Best Practices 

EI Systems are the centerpiece of an emerging package of best 
practices designed to enhance police accountability. The other 
elements in the package of best practices, as recommended by the 

U.S. Department of Justice report Principles for Promoting Police 
Accountability (2001), include a comprehensive use-of-force 
reporting system and an open and accessible citizen complaint 
system. Data on officer traffic enforcement can also be entered 
into an EI system and can be used to address the issue of potential 
racial bias in traffic stops.19 The EI system is the central 
repository of data on use-of-force and citizen complaints (and 
other indicators). In this new paradigm of accountability, 
particular issues such as use-of-force performance are no longer 
treated as separate items but are linked to all areas of officer 
performance in a systematic fashion. 

A Vera Institute evaluation of the implementation of a consent 
decree in Pittsburgh commented that "the early warning system is 
the centerpiece of the Police Bureau's reforms in response to the 
consent decree."20 It is the mechanism by which these 
performance data become useful to managers committed to 
enhancing accountability and reducing officer misconduct.21 

Goals and Impacts of EI Systems 

The goals and potential impacts of EI systems are broader than 
generally understood. When the EI concept originally developed 
more than twenty years ago, EI systems had a narrow focus on 
"problem" police officers. The implicit idea was to "catch" the 
"bad" cops. As EI systems have developed over time, however, 


police managers increasingly understand that they have broader 
goals and potential impacts, including individual officers, 
supervisors, and the department as a whole. 

Individual Officers 

EI systems have a major focus on individual officers. The goal of 
the system is to correct the performance of officers who appear to 
be having performance problems. The NIJ study found that the EI 
systems investigated did succeed in reducing citizen complaints 
and officer use of force among officers subject to intervention. 
This project and the findings are discussed in detail in Chapter 
Two.22 

One important change that has occurred is that there is now a 
broader definition of a "problem" employee. Originally, problem 
officers were defined primarily as those who frequently used 
excessive force and received a large number of citizen complaints. 
Reflecting a broader approach, however, the commander 
responsible for the Pittsburgh Police Bureau's PARS reports that 
the system allows them to identify their "top performers" and 
their "under-performers" as well as their "problem officers". 
Another police department with a comprehensive system, for 
example, identified an officer who had made no arrests, no 
pedestrian stops, and no traffic stops in a given time period. The 
data also indicated that this officer was working the maximum 
number of permissible hours of off-duty employment. These 
indicators allowed the department to take the appropriate 
corrective action. 

The capacity to identify top performers as well as officers with 
performance problems requires a comprehensive EI system such 
as the Pittsburgh PARS. Most of the early EI systems are not able 
to identify under-performers, however, because they do not collect 
the necessary performance data, such as arrest activity. The 
difference between comprehensive and more limited EI systems is 
discussed in detail in Chapter Two. 


Supervisors 

An EI system also reorients the role of supervisors in a 
department. An EI database provides first-line supervisors with 
documentary evidence of officers' performance and permits both a 
detailed analysis of individual officers and comparisons among 
officers. In the survey of police managers (See Chapter Four) one 
manager reported that the EI system as "A useful tool to involve 
supervisors and lieutenants in non-traditional models of problem 
solving. It has served to enhance their management skills and help 
round out their people interaction skills." 

At the same time, however, orienting supervisors to their new role 
is one of the major challenges facing EI systems. Engaging an 
officer with performance problems and helping that person to 
recognize his or her performance problems is a role that many 
supervisors are reluctant to undertake. Some prefer to define their 
role as friend and supporter of their officers. Some do not have 
the skills for the task. Most have not been properly trained for this 
new role. Supervisor training tends to emphasize the formal 
disciplinary aspects of the job. The intervention phase of an EI 
system requires supervisors to be a combination of coach and 
trainer, providing a delicate mix of support, criticism, and help.23 

As already mentioned, the Pittsburgh PARS takes a special 
approach to reorienting the role of supervisors. Sergeants are 
required to access the system's database and review the 
performance of officers under their command. In addition, as 
mentioned earlier, the PARS requires sergeants to conduct "roll-bys" 
of officers who have been identified by the system. In this 
way, PARS creates a structured and systematic form of intensive 
supervision for officers who have been identified as having 
performance problems. (In this and other systems, a supervisor 
has a password granting access to all officers under his or her 
command but no other officers. A district commander at the rank 
of captain, for example, would have access to the records of all 
lieutenants, sergeants, and officers in that district.) 


Some EI systems record each supervisor's access of the system. 
Thus, the performance review of a sergeant can include a review 
of the extent to which he or she accessed the system during the 
period under review. 

An evaluation of the consent decree in Pittsburgh concluded that 
the department's PARS "marks a sweeping change in the duties of 
the lowest level supervisors." Supervisors log on to the PARS 
each day prior to roll call. The screen notifies each supervisor of 
any critical incident involving an officer under his or her 
command since the last time that supervisor logged on.24 

The consent decree regarding the Los Angeles Police Department 
requires that "on a regular basis, supervisors review and analyze 
all relevant information in TEAMS II about officers under their 
supervision...."25 Additionally, the consent decree requires that 
"LAPD managers on a regular basis review and analyze relevant 
information in TEAMS II about subordinate managers and 
supervisors in their command...and that managers' and 
supervisors' performance in implementing the provisions of the 
TEAMS II protocol shall be taken into account in their annual 
personnel performance evaluations."26 The consent decree 
involving the Cincinnati Police Department contains a similar 
requirement that supervisors be evaluated on the basis of "they're 
to use the risk management system to enhance department 
effectiveness and reduce risk"27 
 
BOX 1.3 Goals and Impacts of EI Systems
The Individual Officer 
 Improved Performance 
 Higher Standards of Accountability Supervisors 
 Improved Supervisory Practices 
 Higher Standards of Accountability as Supervisors The Department 
 Higher standards of accountability 
 Reduction in litigation 
 Improved Community Relations 
 

EI systems can also address the problem that, with regular shift 
changes, sergeants often find themselves responsible for officers 
who they do not know. As one commander reports, "There is a lot 
of movement of personnel, so supervisors often do not know the 
histories of their officers. The EWS report brings them up to 
speed in a much more timely fashion." Chapter Four of this report 
contains the findings of a survey of police managers with EI. 

The San Jose Police Department has taken EI systems to a new 
level by developing a Supervisor's Intervention Program (SIP) 
addresses the performance of supervisors as well as rank and file 
officers. Whenever the team of officers under a supervisor's 
command receives three or more citizen complaints within a six-
month period, the supervisor is required to meet with his or her 
chain of command and the head of Internal Affairs. (The San Jose 
EI system relies only on citizen complaints.) In the first year of 
the SIP program, four supervisors met the thresholds and were 
counseled by the department.28 

Some departments are exploring ways to use their EI system to 
hold sergeants accountable. The San Jose Police Department, for 
example, identifies sergeants when officers under their command 
receive a certain number of citizen complaints. This approach 
recognizes that many officer performance problems are the result 
of inadequate supervision.29 

The Department 

An EI system also has the potential for considerable impact on the 
department as a whole. The system defines standards of conduct 
and provides a database for measuring officer performance and 
identifying substandard performance. At the same time, a fully 
operational system redefines the role of supervisors, giving them 
specific duties related to supervision of officers with performance 
problems. And as mentioned in the previous section, the more 
sophisticated EI systems have the capacity to monitor supervisors' 
use of the system and to grade their performance accordingly. 


Problems identified by the EI system can also lead to changes in 
departmental policy. The consent decree between the Justice 
Department and the New Jersey State Police, for example, 
requires that "Each supervisor shall, consistent with his or her 
authority, implement any appropriate changes or remedial 
measures regarding traffic enforcement criteria, training, and 
enforcement practices for particular units or subunits..."30 

Over the long run, the cumulative effect of the changes in the role 
of supervisors and changes in departmental policy has the 
potential for changing the organizational culture of a police 
department and establishing new standards of accountability. In 
the police commander survey (See Chapter Four), one commander 
explained that over time rank and file officers came to accept the 
goals of the EI system: "Most no longer believe this system is out 
to 'get' them, but rather to assist them." A top commander in the 
Pittsburgh Police Bureau claims that PARS has begun to change 
the culture within the department. Two impacts are particularly 
notable. First, the system forces a more intensive form of 
supervision on the part of sergeants. Second, by identifying 
officers who are underperforming because they are devoting too 
much time to off-duty employment, and then correcting that 
problem, PARS helps to develop a greater commitment by officers 
to their primary employment.31 

The positive impact on the organizational culture of a department 
is not guaranteed, however. If there is little effective use of the EI 
systemas in failure to conduct meaningful interventions with 
officersthe impact could easily be negative: reinforcing officer 
cynicism about the gap between stated ideals and actual practice 

Learning from the Corrections Profession 

The corrections profession has already begun to incorporate the 
EI concept into its accreditation process. The American 
Correctional Association (ACA) is adopting a "performance-based" 
approach to accreditation. The indicators for agency 
performance are essentially the same as those used in law 
enforcement EI systems. The Performance-Based Standards for 


Adult Community Residential Services, for example, includes the 
following standard:32 

"Number of offenders' grievances filed alleging inappropriate 
use of force in the past 12 months, divided by [the] Average 
daily offender population in the past 12 months. 

The approach used by the ACA focuses on the agency, with 
organizational-level performance measures, while law 
enforcement EI systems focus on individual officers. There are 
good reasons for suggesting that the ACA approach is the better 
of the two. It properly focuses on organizational-level issues of 
management and supervision where corrective action can prevent 
future misconduct. The current law enforcement focus on 
individual officers tends to make scapegoats of a few officers, 
generate hostility from the rank and file, and ignore important 
issues of management and supervision. 

Early Intervention and Community Policing 

EI systems are fully compatible with the goals of community 
oriented policing (COP). In particular, the community policing 
movement has created demands for new measures of police 
performance.33 Because COP reorients the goals of policing, new 
measures of are needed for both individual police officers and 
entire departments. The traditional measures of the crime rate and 
the clearance rate are no longer adequate. 

Traditional police performance measures are inadequate in several 
regards. They emphasize crime, to the neglect of quality-of-life 
issues; they fail to take into account perceived community needs; 
and they fail to reward adequately good police performance.34 By 
systematically identifying and attempting to control inappropriate 
behavior, an EI system can potentially reduce the number of 
incidents that alienate communities from the police. 

One of the basic goals of COP is to establish closer ties to the 
communities receiving police services. COP places particular 
emphasis on being more responsive to community concerns about 
the quality of police services, particularly with respect to racial 


and ethnic minority communities concern about excessive force or 
other forms of inappropriate behavior by officers. 

In this regard, Alpert and Moore recommend "the development of 
statistical evidence on the use of force and the incidence of 
brutality, discourtesy, and corruption..." This is precisely what an 
EI system does. Indeed, Alpert and Moore's approach suggests 
that a law enforcement agency's personnel data system should 
transcend a narrow focus on suspected officers with performance 
problems and include all current sworn officers.35 

Early Intervention and Problem-Oriented Policing 

EI systems are also consistent with the principles of problem-
oriented policing (POP). In this case, the problem is not graffiti or 
public drunkenness, but those officers with performance 
problems. As initially formulated by Herman Goldstein, POP 
holds that law enforcement agencies should desegregate the 
various aspects of their role and, instead of attempting to address 
"crime" and "disorder" as global categories, should identify 
particular problems within each category and develop narrowly 
tailored responses appropriate to each. In addition, they should 
develop the appropriate performance measures for each problem.36 

The POP process of scanning, analysis, response, and assessment 
(SARA) is, for all practical purposes, the way an EI system 
operates: 

 Scanning: review of data in the EI system database. 
 Analysis: identification and selection of potential officers with 
performance problems. 
 Response: intervention with selected officers. 
 Assessment: post-intervention review of officers' performance. 

Early Intervention and COMPSTAT 

EI systems are very similar to COMPSTAT programs in terms of 
both purpose and process. Both rely on the analysis of systematic 
data for the purpose of addressing problems and holding police 
managers accountable for problems under their command. 


An evaluation of the Pittsburgh PARS (its EI system) described 
one key element as modeled after COMPSTAT. The Quarterly 
COMPSTAT meetings in Pittsburgh involve a review of officers 
who have been identified by PARS. Area commanders make 
presentations about any officers under their command who have 
been identified and conclude with a recommendation regarding 
formal intervention. After a discussion, the chief of police makes 
a final decision on what course of action to take.37 

COMPSTAT is one of the most notable police innovations in 
recent years, and has been widely adopted across the country. A 
COMPSTAT program involves a computerized database of timely 
and systematic data on criminal activity. Analysis of the data 
provides a means of developing rapid and narrowly targeted 
responses designed to reduce crime. Regular COMPSTAT 
meetings are designed to hold police managers accountable by 
requiring them to describe crime trends in their areas of 
responsibility and to explain what responses they have developed. 

Early Intervention and Risk Management 

EI systems can be a key component in a risk management system 
(RMS).38 Risk management is a process for reducing an 
organization's exposure to financial loss due to litigation. In law 
enforcement, the major civil litigation costs arise from fatal 
shootings, excessive physical force, and high-speed pursuit 
incidents. An EI system database can readily identify patterns of 
individual officers and situations that represent actual or potential 
risks. It also provides a structured process for correcting those 
problems and reducing the potential financial risk. In fact, the 
expanded EI system in the Cincinnati Police Department, as 
mandated by a memorandum of understanding with the Justice 
Department, is called the Risk Management System. 

Early Intervention and Police-Community 
Relations 

EI systems represent a potentially effective response to the 
historic problems of officer misconduct and tensions between law 
enforcement professionals and the communities they serve. For 


many decades, alleged police officer misconduct, including 
misuse of deadly force, use of excessive physical force, and 
discourtesy, has been a major cause of tensions between the police 
and racial and ethnic minority communities. Civil rights leaders 
have alleged that minority citizens are not only the targets of 
police misconduct at a rate disproportionate to their presence in 
the population, but that police departments have failed to 
investigate citizen complaints about misconduct and discipline 
guilty officers.39 

To an unfortunate extent, police abuse of citizens has been 
explained in global terms, typically labeling all police officers 
with a broad brush. Abusive behavior for example, has been 
attributed to a general police subculture, with the implicit 
assumption that certain attitudes and behaviors are common to all 
officers in all police departments.40 Other observers have 
attributed abusive behavior to race, arguing that abuse reflects 
racist attitudes and behavior on the part of white police officers 
toward citizens of color.41 Some observers have attributed overly 
aggressive or abusive behavior to gender, holding that reflects 
male norms of behavior.42 Still other observers have attributed 
police misconduct to organizational dysfunction, arguing that poor 
leadership and low standards of professionalism have tolerated 
many different forms of police officer misconduct.43 

The great virtue of an EI system is that it can pinpoint specific 
actions that are creating conflict with the community and the 
particular officers responsible for them. For example, if there are 
complaints in one neighborhood about frequent use of force, an EI 
system can help to confirm or refute the allegations, and if 
confirmed, identify the officers involved. At the same time, it can 
help to identify those officers who are engaged in active police 
work (e.g., a high volume of arrests) without resorting to 
inappropriate behavior. 

Early Intervention and Racial Profiling 

A comprehensive EI system has the potential for addressing the 
issue of racial profiling. The Pittsburgh and Phoenix EI systems 
collect data on traffic stops, including data on the race and gender 
of drivers. 


These data make it possible to identify officers who are stopping a 
suspiciously high number of racial or ethnic minority drivers. As a 
discussion paper by Samuel Walker argues, an EI system solves 
many of the problems associated with interpreting traffic stop 
data. Rather than use resident population data, as many reports do, 
an EI system permits comparisons among officers working similar 
assignments.44 

In a predominantly Latino neighborhood, for example, it is to be 
expected that a high percentage of all stops would involve Latino 
drivers. The EI system database can help identify officers who are 
stopping far more Latino drivers than their peers. The data alone 
would not prove that racial profiling exists. Rather, it would be 
the starting point for supervisory review that would determine 
whether or not an officer's activities involve racial or ethnic bias. 

The peer officer comparison approach is discussed in Chapter 
Two of this report. It is the methodology currently used in the 
Pittsburgh PARS.45 

In one department, the EI system identified an officer who had a 
practice of stopping female drivers for questionable purposes. The 
department received a formal citizen complaint from one female 
driver who alleged that the officer made improper sexual 
advances during a traffic stop. A review of the EI database 
revealed a very high rate of traffic stops of female drivers. 
Combined with the original complaint, these data were sufficient 
to cause a formal departmental intervention. 

Using peer officers as the comparison group solves the most 
vexing problem with regard to the analysis of traffic stop data. 
Most current data collection efforts use the racial and ethnic 
composition of the resident population as the baseline or 
denominator. 

One Size Does Not Fit All Departments 

There is no single model for EI systems. The outline of the basic 
components of a system described above represents a single 
framework. Each department can develop an EI system 


appropriate to its needs. These needs will vary significantly 
according to the size of the department.46 

Medium-sized and small agencies do not need as complex and 
sophisticated a system as do large departments. Most of the 
available information about EI systems at present, however, is 
derived from large law enforcement agencies. Consequently, at 
this time it is not possible to specify the exact needs of medium-
sized and small departments. Developing those needs is on the list 
or priorities for future research and program development (See 
Chapter Five). 

A Work in Progress 

Although EI systems are increasingly popular and have been 
recommended by a wide range of professional groups, they are far 
more complex than is generally realized. They require making a 
number of difficult choices regarding their design, and once 
operational require close, ongoing administrative attention. 

EI systems are in a state of continuing development. There is no 
solid consensus regarding a single best way to operate a system. 
Local police departments have found that they need to continually 
revise and fine-tune their EI systems. As one police manager 
explains, "this is a new practice that may, and has, changed over 
time. It continues to evolve." Another explains that an EI system 
"needs to continuously evolve and expand" (See Chapter Four). 

Additionally, our understanding of EI systems is experiencing a 
paradigm shift. First, they are no longer seen only as a means of 
identifying officers with performance problems, but instead are 
recognized as an effective data-based tool for general personnel 
assessment. Second, instead of focusing exclusively on rank and 
file officers they are seen as a tool for promoting the 
accountability of supervisors. 


Conclusion 

Early Intervention systems are an important new tool for police 
accountability. They are data-based programs for identifying 
officers with performance problems and providing the basis for 
departmental intervention to correct those problems. 

EI systems have emerged as a recommended best practice in 
police accountability. They have been recommended by a wide 
range of professional associations. In addition, EI systems are 
consistent with the goals of both community policing and 
problem-oriented policing. 

Although they are growing in popularity, EI systems are still in 
the early stages of development. Departments are still wrestling 
with a number of difficult and important issues. These issues are 
addressed in the remaining chapters of this report. 


Chapter Two

The Components of an EarlyIntervention System 

The Components.................................................25
Size and Scope: Two Approaches to EI Systems ..................25 
Performance Indicators.........................................26
The Number of Indicators.......................................26
Department-Specific Indicators ................................29
Identification and Selection Process ..........................30
Identifying Problem Performance: Three Examples................30
The Proper Thresholds .........................................31
Mandatory vs. Discretionary Referral to Intervention ..........33
Feedback to Officers...........................................35
Intervention...................................................35
Action Alternatives............................................35
Training Supervisors for Intervention..........................37
Holding Supervisors Accountable ...............................38
Post-Intervention Monitoring...................................39
An EI System Appropriate for Each Department ..................40
The Need for Careful Planning..................................41
Conclusion ....................................................41

The Components of an Early Intervention System 

The Components 

An Early Intervention system consists of four basic components: performance 
indicators, identification and selection process, intervention, and post-
intervention monitoring. An earlier report identified three components of EI 
systems.47 This report, reflecting the maturation of EI systems, utilizes four 
components, because the identification and selection process is a separate 
component from the list of performance indicators. 

Size and Scope: Two Approaches to EI Systems 

While all EI systems will have the same basic components, the 
size and scope of different systems will vary according to their 
goals.48 Two basic types of EI systems currently exist, 
representing differently sized systems with different goals. Some 
systems, such as the Pittsburgh PARS and the Phoenix PAS, are 
comprehensive personnel assessment systems. They collect a very 
wide range of data and are able to conduct complex data analyses 
that can include identifying top-performing and underperforming 
officers as well as officers with performance problems. Systems 
of this sort not only require a sophisticated technological 
infrastructure but also an enormous amount of administrative 
commitment.49 

BOX 2.1 Components of an EI System
 Performance Indicators 
 Identification and Selection 
 Intervention 
 Post-Intervention Monitoring 
 

Many other systems, such as the Miami-Dade and the Tampa, 
Florida systems, are more narrowly focused and may be described 
as performance problem systems. They collect a smaller range of 
data and focus on a more limited range of performance issues. As 
a result, they are also less expensive and require a smaller 
commitment of administrative resources. 

Neither system is necessarily better than the other. They simply 
have different goals and different approaches to police 
accountability. The goals of the more comprehensive systems are 
more ambitious, but they are also more demanding of the 
department. An evaluation of Pittsburgh's PARS, for example, 
pointed out that its comprehensive approach comes at a price. By 
imposing greater administrative responsibilities on sergeants for 
monitoring officer behavior, it necessarily takes time away from 
traditional field supervision and other duties.50 

Performance Indicators 

The performance indicators involve those aspects of officer 
performance that are the subject of official departmental reports 
and are entered into an EI system database. Almost all experts 
argue that an EI system should utilize a broad range of 
performance indicators and not rely on just one indicator as many 
of the first EI systems did (and some still do). Relying on a single 
indicator such as citizen complaints is not recommended. There 
are a number of problems related to official data on citizen 
complaints and they are not consistently reliable indicators of 
officer performance.51 Using a broader range of indicators is more 
likely to identify officers whose behavior requires departmental 
intervention. 

The Number of Indicators 

There is no consensus of opinion regarding the exact number of 
performance indicators to use. The Pittsburgh PARS uses 18 
performance indicators (although the department is considering 
adding five more). The Phoenix PAS utilizes 24 indicators. The 
consent decree between the U.S. Department of Justice and the 
Los Angeles Police Department specifies 17 categories of data to 


be entered into the department's TEAMS II (See Appendix A).52 
The Memorandum of Understanding involving the Cincinnati 
Police Department requires the collection of ten performance 
indicators. 

Box 2.2 represents a list of performance indicators in a model 
comprehensive personnel assessment system. Box 2.3, 
meanwhile, lists the performance indicators in a model problem 
officer system. 

BOX 2.2 Performance Indicators
Comprehensive Personnel Assessment Model 
1. All non-lethal uses of force 
Citizen injuries requiring medical attention 
Disciplinary outcomes 
2. All officer-involved shooting incidents 
On-duty/off-duty discharges, citizen injuries, citizen fatalities 
Disciplinary outcomes 
3. All officer-involved vehicular pursuits 
Accidents 
Injuries 
Fatalities 
Disciplinary outcomes 
4. All citizen-initiated complaints 
Use of force 
Racial bias 
Investigative outcomes 
5. All citizen-initiated commendations or compliments 
6. All departmental commendations and awards 
7. Criminal arrests and investigations of subject officer 
 

Chapter 2

BOX 2.2 Performance Indicators (Continued from page 27)
8. Civil suits or an administrative claim in which subject 
officer is named 
Nature of claims 
Case outcomes 
9. All arrest reports, crime reports, and citations made by 
officers 
10. All motor vehicle stops and pedestrian stops made by 
subject officer 
Race, ethnicity, gender, age of drivers stopped 
11. Performance evaluations for each officer 
12. Training history 
Any failure of an officer to meet weapons qualification 
requirements 
13. All management and supervisory actions, including 
non-disciplinary actions, related to each officer 
14. Sick leave or family leave record 
15. Canine unit deployment involving subject officer 
Deployments 
Bites 
Citizen injuries 
Complaints or civil suits arising from deployments 
16. Failure to appear in court 
 

BOX 2.3 Early Intervention System Focused Performance Model
1. All officer-involved use-of-force incidents 
2. All officer-involved shooting incidents 
3. All citizen-initiated complaints 
4. All officer-involved civil suits or administrative claims 
5. All departmental disciplinary actions 
6. All citizen-initiated or department-initiated commendations or 
awards. 
 
The choice of the number of indicators to use involves a trade-off. 
The greater the number of indicators used, the greater the capacity 
of the system to evaluate the full range of officer performance. 
The more comprehensive Pittsburgh PARS, for example, allows 
commanders to identify top-performers and under-performers, as 
well as officers with performance problems. Because the system 
includes arrest data, commanders can identify officers who have 
several use-of-force reports but who are also making a very large 
number of arrests and compare them with officers who have 
slightly fewer complaints but who make very few arrests. 

At the same time, however, utilizing a greater number of 
indicators imposes greater administrative demands in terms of 
data entry and data analysis.53 A smaller problem officer system, 
with fewer performance indicators, will be much less expensive 
and easier to create and maintain. 

Department-Specific Indicators 

Some performance indicators have been adopted because of 
specific controversies surrounding a particular department. The 
memorandum of understanding between the U.S. Department of 
Justice and the City of Cincinnati requires that the Risk 
Management System include drawing and pointing a weapon at a 
citizen. This indicator is included because there had been many 
complaints from citizens about such action by police officers.54 


The Phoenix PAS includes High Stress Radio Calls (calls 
involving actual or intended suicides, or child abuse, molestation, 
or death) as a performance indicator. This indicator is being 
included because of three incidents involving suicide or attempted 
suicide by officers following a series of calls to incidents of this 
sort. In one case, an officer mentioned to his supervisor that he 
had seen enough of these calls, and expressed suicidal thoughts.55 

Individual departments may have other special problems they are 
aware of and want to incorporate into their EI systems. 

Identification and Selection Process 

Given the existence of a performance database, a strategy is 
needed to identify officers for performance review and to select 
officers for referral to intervention. These are separate stages. In 
many EI systems an officer who is identified will automatically 
be referred to intervention. Others, however, utilize a two-stage 
process through which officers who are identified are subject to a 
full-scale performance review with the result that only some are 
referred to formal intervention. 

Several participants at the Early Intervention State of the Art 
Conference56 vigorously emphasized the point that a computerized 
database only gives you numbers. It does not make decisions 
about what to do with identified officers; only supervisors can do 
that. These statements only highlight the point that there is no 
magic in a sophisticated computerized system. To be effective, an 
EI system requires responsible management by the command 
staff. 

Identifying Performance Problems: Three Examples 

The following examples are based on actual cases adapted from 
the EI system of one large police department.57 

Officer A 

The Indicators: Officer A had five use-of-force reports during one 
reporting period. Relative to other officers in the same unit, this 


was a very low figure. But, he made only eight arrests during this 
period. 

The Analysis: Officer A used force in more than half of the arrests 
he made. The ratio of force to arrests is cause for departmental 
concern and probable intervention. 

Officer B 

The Indicators: A female driver filed a citizen complaint against 
Officer B alleging an inappropriate sexual advance during a traffic 
stop. An examination of Officer B's EI file indicated a very high 
number of traffic stops involving females. 

The Analysis: Officer B appears to be abusing his law 
enforcement powers to harass female drivers. 

Officer C 

The Indicators: Officer C had no citizen complaints or use-of-force 
reports for the reporting period. He also had made no 
arrests, no traffic stops, and no field stops. An examination of his 
EI file indicated that he was working the maximum number of 
permitted hours of off-duty employment. 

The Analysis: Officer C. was devoting all his energy to his off-
duty employment rather than fulfilling his responsibilities to the 
department. 

The Proper Thresholds 

The central issues with regard to identifying and selecting officers 
for intervention the question of the proper performance thresholds 
to be used. (Some systems use the term "triggers" but that word 
has inappropriate connotations and is not recommended.) The 
issue can be framed in the following terms: 

** How many use-of-force incidents over what time period 
should qualify an officer for intervention? 


** What combination of use-of-force reports, citizen complaints, 
and other indicators over what period of time should qualify 
an officer for intervention? 

At present, there is no consensus regarding the best set of 
thresholds. Three different formulas are currently being used: 
department-level thresholds, peer officer thresholds, and ratio-
based formulas. 

1. Department-Level Thresholds 

In the Miami-Dade EIS, an officer is identified if in any quarter 
he or she has two or more citizen complaints or three or more 
use-of-force reports in the previous 90 days (or quarter). A list of 
the officers so identified is circulated to commanders who then 
must take one of four actions: refer the officer to the 
Psychological Services Bureau for counseling, refer the officer to 
the department Stress Abatement Program, refer the officer to 
some other form of corrective action, or conclude and document 
that no action is required. 

The Miami-Dade EIS also operates at an earlier and less formal 
manner than the formal process described above. Monthly reports 
are circulated listing officers who have two or more citizen 
complaints or two or more use-of-force reports (or both). 
Commanders are not required to take any formal action, but are 
expected to review an officer's performance and perhaps conduct 
an informal meeting with him or her. 

2. Peer Officer Averages 

The Pittsburgh PARS utilizes an approach that compares officers 
performances with peer officers, meaning officers with the same 
assignment (e.g., night shift, high-crime area, daytime traffic unit, 
etc.). Some experts have described this approach as comparing 
apples with apples. Officers assigned to high-crime areas are more 
likely to generate use-of-force reports and citizen complaints 
because of the nature of the work environment. It is unfair to 
expect them to meet the same numerical standard as officers 
working low crime areas.58 


The Memorandum of Understanding mandating the EI system in 
the Cincinnati Police Department (officially called the Risk 
Management System) also requires a peer officer formula. The 
RMS is required to report data on the average level of activity for 
each data category by individual officer and by all officers in a 
unit (See Appendix D). Finally, the Phoenix PAS is developing a 
peer-officer-based formula. 

3. Performance Indicator Ratios 

A third approach to thresholds analyzes performance data in terms 
of the ratios between different performance indicators: e.g., the 
ratio of use-of-force reports to arrests. In one example, (See Box 
2.4, Officer A), an officer did not use force very often, but used 
force in more than half of all arrest situations. One of the virtues 
of this approach is that it allows commanders to identify top-
performing officers (e.g., high arrest activity, low arrest-to-complaint 
ratio) and not unfairly subject them to intervention. The 
Phoenix PAS has been testing this approach prior to 
implementation.59 

Mandatory vs. Discretionary Referral to 
Intervention 

Some systems use a mandatory approach for referral to 
intervention while others use a two-stage discretionary approach. 

In the initial Minneapolis Police Department Early Warning 
System, an officer was automatically referred to intervention if he 
or she received three complaints in a twelve-month period.60 Not 
only did this system (which has since been replaced by a more 
sophisticated performance review system) rely only on citizen 
complaints but it did not take into account the nature and context 
of the complaints. 

The Miami-Dade EIS generates monthly reports that list officers 
who have received two or more citizen complaints or have made 
two or more use-of-force reports (or both) in the last 60 days. 
This list is for informational purposes only, and supervisors are 
not required to take any action. It is understood, however, that 


supervisors will review any listed officer's performance and 
possibly speak to individual officers. Quarterly reports list officers 
who have two or more personnel (e.g., citizen) complaints and 
involvement in three or more use of force incidents (or both) 
within the previous 90 days. Supervisors must take action with 
regard to these officers. The alternatives include referring the 
officer to the stress abatement program or to the Psychological 
Services Program for counseling, or requiring some other 
corrective action, or concluding (and documenting) that no action 
is required. In short, commanders are provided information about 
officer performance but are left the discretion to decide whether 
any formal intervention is required, and if so what action is 
appropriate. 

The Los Angeles Sheriff's Department's PPI system utilizes a 
discretionary two-stage approach. Only some of those officers 
initially identified by the system are placed on Performance 
Review. Between 1996 and 2002, a total of 1,213 employees were 
identified, but only 235 (or 19 percent) were placed on 
Performance Review.61 The Performance Review Committee 
screens officers twice. First, the committee solicits the views of 
an officer's captain about whether placement on Performance 
Review is appropriate. A second screening, taking into account 
the captains' views, is conducted to select officers for a detailed 
Employee Profile Report (EPR) (known informally as a "Blue 
Book").62 The EPRs are prepared by sergeants assigned to the 
department's Risk Management Bureau. 

The performance review is designed to place an officer's 
performance in the context of his or her assignment and 
performance history. It considers evidence that might exculpate an 
officer along with evidence indicating a problem that is believed 
to be worse than suggested by the performance data. For example, 
an officer with a number of use-of-force reports might be excused 
because of particularly intensive law enforcement activity during 
the period under review and an otherwise exemplary record. At 
the same time, another officer might have in addition to a number 
of complaints and use-of-force reports a reputation for a bad 
temper and inability to work well with colleagues. In the end, 
some of the identified officers are selected for intervention. 


Pittsburgh's PARS also generates information about officer 
performance for the benefit of supervisors but leaves them the 
discretion to decide what action to take. An evaluation of the 
PARS found that formal intervention is a rare outcome for officers 
initially identified by the system. The most common response 
involves monitoring of the officer through supervisory field 
checks (also referred to as "roll-bys").63 Part of the reason for the 
few formal interventions is that, unlike other EI systems, the 
PARS has a formal process for pushing responsibility for 
monitoring officer behavior down to first line supervisors. As 
explained elsewhere in this chapter, the PARS requires sergeants 
to log on to the system every day, a process that forces daily 
consideration of performance data. 

Feedback to Officers 

One issue that has been discussed among experienced EI system 
commanders is the nature of feedback to officers who are 
identified but not selected. There may be some value in informing 
such officers that their performance was identified by the EI 
system but that upon review, it was determined that there was no 
problem requiring formal intervention. This situation is likely to 
arise with high-performing/low-ratio officers. Informing them of 
the decision not to refer them to intervention may be a valuable 
way of communicating departmental approval and support for 
their performance. 

Intervention 

The intervention phase consists of some form of counseling or 
retraining for officers who have been selected. In some EI 
systems the counseling is done by an officer's immediate 
supervisor. In other departments, the counseling involves other 
command officers. 

Action Alternatives 

The Commission on Accreditation for Law Enforcement Agencies 
(CALEA) Standards state that a departmental Personnel Early 
Warning System should have a menu of remedial actions for 


officers selected for intervention.64 In the Miami-Dade Police 
Department, the action alternatives include referral to the 
departmental Psychological Services Program or participation in 
the Stress Abatement Program. Another alternative is a 
determination that no problem exists and that no formal action is 
necessary. 

One of the virtues of the data-driven nature of EI systems is that 
the performance data provide the basis for discussing specific 
aspects of an officer's performance. Chapter Four of this report 
includes comments by police managers explaining how their EI 
systems strengthen the hand of supervisors in dealing with 
officers with performance problems. Where citizen complaints 
about rudeness are frequent, referral to Verbal Judo might be the 
proper response. Where problems in handling arrest situations are 
the problem, specific retraining can be offered. Depending upon a 
department's existing personnel procedures, the counseling or 
retraining can be recommended or mandated. 

In the Los Angeles Sheriff's Departments PPI system, the 
Performance Review Committee develops a formal performance 
plan for each subject officer.65 

Intervention Group Training Classes 

Some EI systems conduct intervention through classes for groups 
of selected officers.66 There are a number of problems with this 
approach, however. First, the subjects covered in a class are 
general in nature and do not necessarily address the particular 
problems of individual officers. Second, they are difficult to 
schedule and considerable time may pass between the time an 
officer's problems are identified and he or she is enrolled in the 
intervention class. Third, there is evidence that bringing a group 
of selected officers together creates a dynamic of group solidarity 
that reinforces inappropriate attitudes. Such classes have been 
referred to as "bad boys" classes.67 


Training Supervisors for Intervention 

The most important issue identified at the Early Intervention State 
of the Art Conference involved preparing supervisors to conduct 
interventions.68 Lack of adequate orientation and training was also 
one of the most serious problems reported in the police managers 
survey (See Chapter Four). One explained that as a result of 
inadequate training, the system is misunderstood by the first line 
supervisors, and therefore retraining is needed at the application 
level. EI systems represent a new role for supervisors, especially 
sergeants, in which they are expected to coach and help officers 
under their command. The traditional role of sergeants has 
involved formal disciplinary actions, and training has emphasized 
the procedural aspects of the process. 

In an EI system it is essential that first-line supervisors actually 
conduct intervention sessions, and do so in a manner consistent 
with the system's goals. It is possible for a particular sergeant to 
simply tell an officer not to worry about it. This would undermine 
the EI system and breed cynicism among officers. 

A related problem is the potential for favoritism, in which one 
supervisor excuses the performance of one officer under his or her 
command while taking a tough stand on another officer with 
similar performance problems. 

A final problem is the possibility of inconsistent interventions 
across a group of supervisors. Since interventions are designed to 
be confidential, it is not possible to require documentation of the 
actual substance of intervention sessions. 

It is essential that departments provide proper training for 
supervisors regarding their role in interventions and that this be 
done well before an EI system become operational. Several steps 
are possible to ensure that interventions are delivered as desired. 
First, in some departments the intervention is conducted in a 
meeting with more than one supervisor. This helps to ensure that 
one supervisor will not inappropriately excuse the performance of 
an officer. At least one system in operation is programmed to 
require supervisors to notify their own supervisors that the 


intervention has occurred. Failure to do so will result in an 
automatic email notice to the higher-ranking supervisor. This 
procedure, of course, does not guarantee anything about the 
content of the session. 

BOX 2.4 Intervention Action Alternatives
 Referral to department employee assistance program 
 Referral to professional family counseling service 
 Referral to credit counseling services 
 Retraining on traffic stop tactics 
 Verbal judo training 
 
Holding Supervisors Accountable 

Some EI systems have begun to address the issue of inadequate 
first-line supervisors. If one or more officers in a unit are 
identified by an EI system, part of the problem may be inadequate 
supervision. 

The San Jose Police Department has developed a Supervisors 
Intervention Program (SIP). A supervisor is identified by the 
system whenever officers under his or her command receive a 
combined total of three or more complaints in any six-month 
period. The supervisor is then required to meet with his or her 
immediate supervisor, the head of the Professional Standards and 
Conduct Unit (PSCO), and an assistant chief. One limitation of 
the system is that it relies on only one performance indicator, 
citizen complaints.69 The San Jose Police Department is perhaps 
the first to institute a formal process for using the EI system to 
evaluate the performance of supervisors. More discussion and 
program development on this issue is warranted in the immediate 
future. 

The Pittsburgh PARS takes a different approach, integrating the 
EI system into routine supervision. Sergeants are required to 
access the PARS database every day and to review the 
performance of all officers under their command. This approach 
socializes them into use of the system and the habit of data-driven 
supervision. Sergeants are able to intervene with an officer even 


before that officer crosses the system thresholds. Through this 
process, supervisors are expected to regard performance analysis 
and intervention as regular parts of their jobs and not some 
special and onerous task reserved only for serious problems.70 

The Pittsburgh PARS also attempts to ensure that interventions 
are conducted through a process of computerized notifications. 
When an officer is identified by the system, his or her sergeant is 
notified by email that an informal counseling session is required. 
The sergeant must then report by reply email when the counseling 
session has been completed. Failure to provide this notification 
automatically generates an email notice to that sergeant's 
lieutenant. Failure of the lieutenant to take the proper action 
generates an email notice to the next higher commander. 

Post-Intervention Monitoring 

Following an official intervention, supervisors need to follow-up 
by monitoring the performance of an officer for a specified period 
of time. At least three different approaches to follow-up exist in 
current EI systems. These approaches vary by their degree of 
formality and record-keeping. 

In many departments, the monitoring is informal and conducted 
by immediate supervisors. Some departments keep officers on a 
list for a specified time period following intervention and 
continue to monitor their performance. In the Pittsburgh PARS, 
sergeants are required to do "roll-bys" of officers who have been 
identified by the system. A roll-by is a direct observation of an 
officer at work. In effect, this is a form of intensive supervision. 
An EI system such as Pittsburgh's has the effect of structuring and 
systematizing such intensive supervision. 

In the Los Angeles Sheriff's Department, an officer remains on 
Performance Review for two years after initial identification. An 
officer may be removed only upon recommendation of his or her 
captain and approval by the Performance Review Committee. In 
other departments, however, the monitoring involves an elaborate 
system of observation, evaluation, and reporting. The New 
Orleans PPEP involves a six-month follow-up in which 


supervisors are required to observe subject officers while they are 
on duty and to file a signed evaluation of officers' performance 
every two weeks. The resulting documentation ensures that 
monitoring occurs.71 

The post-intervention follow-up component of EI systems has 
received the least attention by officials involved in developing 
and managing EI systems. It is an issue that needs additional 
attention in the immediate future. 

In determining the exact nature of the post-intervention follow-up, 
agencies also encounter the problem of balancing the demands of 
accountability with the need for efficiency. As we indicated in our 
discussion of this issue related to the performance database, there 
is no clear answer to the problem at present. It needs to be the 
subject of continued discussion and experimentation within the 
law enforcement profession. 

An EI System Appropriate for Each Department 

The range of requirements EI can address prohibits the 
development of any single definitive model of an EI system. The 
abovementioned basic system components can provide only a 
general framework. The most effective EI systems are developed 
specifically to address the needs of the personnel they serve, and 
those needs are as unique as the agencies that produce them. 

One of the most influential variables in EI system development is 
the size of the agency it will serve.72 

Small- to mid-sized agencies do not require the same levels of 
complexity and sophistication as their larger peers. Large agencies 
not only have much different problem sets, however, they also 
have significantly different levels of resources available for EI 
system development. Due in large part to this discrepancy in 
resources, the vast majority of information currently available 
about EI system development and implementation is derived from 
work with larger law enforcement agencies. 


This makes for an unfortunate scarcity of information regarding 
the requirements for effective EI systems in smaller and mid-sized 
agencies. Developing this information is high on the list for future 
research and program development (See Chapter Six). 

The Need for Careful Planning 

It is the consensus of opinions among police commanders with 
extensive EI system experience that careful planning is necessary 
for the effective development and implementation of an EI 
system. One of the main points of this report is that EI systems 
are extremely complex administrative tools, with several critical 
components and several different goals. Several of the established 
EI systems have gone through significant revision since they were 
first implemented. The Phoenix Police Department took over two 
years to develop and implement its PAS. Other experts believe 
this was a wise decision. One important aspect of an effective 
planning process is to involve officers from all ranks in the 
process. This helps to ensure that officers fully understand and are 
committed to the goals and procedures of the EI system. 

Conclusion 

EI systems are extremely complex administrative tools. They 
consist of four basic components, each of which requires 
decisions about the exact design. Careful planning and 
consideration of these decisions is necessary for an EI system to 
be implemented effectively. A planning process is described in 
Chapter Five. 


Chapter Three

Background: The Origins and Development of 
Early Intervention Systems 

Evidence About Officers With Chronic Performance Problems .............45
The Early History of EI Systems........................................48
The First Permanent Systems: Miami and Miami Dade .....................50
An Evolving Concept ...................................................53
An Emerging Best Practice..............................................54
Implementing EI Systems: A Major Challenge ............................55
The Los Angeles Sheriffs Department PPI System........................55
The Los Angeles Police Department......................................57
Pittsburgh Police Department...........................................58
Research on the Effectiveness of EI Systems ...........................59
(1) The NIJ Study of Early Warning Systems.............................60
(2) Evaluation of Pittsburgh PARS .....................................67
(3) Assessment of the LASD PPI System..................................68
(4) Anecdotal Evidence From Other Departments .........................69
(5) The Police Managers Survey........................................69
Conclusion ............................................................70

Background: The Origins and Development 
of Early Intervention Systems 

The purpose of this chapter is to describe the origins and development 
of EI systems. The first section covers the basic rationale for EI 
systems: the argument that a small number of police officers are 
responsible for a disproportionate number of problematic incidents 
such as citizen complaints, use-of-force incidents, and civil suits against law 
enforcement agencies. The second section provides a brief overview of the 
history of EI systems. The final section examines the development of EI 
systems in a few selected jurisdictions for the purpose of illustrating the 
administrative difficulties some of them have had in implementing their EI 
systems. The final section summarizes the findings of a National Institute of 
Justice-sponsored evaluation of the EW systems in three major police 
departments. 

Evidence About Officers With Chronic 
Performance Problems 

Early Intervention (EI) systems originated out of a growing 
recognition that a few officers are responsible for a 
disproportionate number of any police department's problems 
related to use-of-force and citizen complaints. The existence of 
the so-called "problem officer" was known informally for a long 
time. The evidence was largely anecdotal, however, and 
departments did not take any official action to address the 
problem. Police chiefs increasingly commented that "90 percent 
of our problems are caused by ten percent of our officers."73 
Professor Herman Goldstein, one of the leading experts on 
policing and the founder of problem-oriented policing, was the 


first expert to discuss the possibility of "Identifying Officers with 
a Propensity for Wrongdoing," and then doing something to 
improve their performance.74 He cited an experimental (but short-
lived) program by Hans Toch in the 1970s in which peer officers 
counseled Oakland, California, police officers with records of 
use-of-force incidents.75 

A 1981 report by the U.S. Civil Rights Commission, Who is 
Guarding the Guardians?, provided the first data documenting the 
phenomenon of a few officers receiving a disproportionate 
number of all citizen complaints. It published a table indicating 
that officers who had received five or more complaints over a 
two-year period represented only 12 percent of all officers 
receiving complaints but accounted for 41 percent of all 
complaints (Table 3-1). One officer had 12 complaints while two 
others had 11 complaints during this two-year period. In response, 
the Commission recommended that "A system should be devised 
in each department to assist officials in early identification of 
violence-prone officers."76 

BOX 3.1 Houston Complaint Data
Number of Complaints per Officer 
 2 complaints 298 officers 
 3 complaints 136 officers 
 4 complaints 70 officers 
 5 complaints 33 officers 
 6 complaints 18 officers 
 7 complaints 8 officers 
 8 complaints 6 officers 
 9 complaints 1 officers 
 10 complaints 0 officers 
 11 complaints 2 officers 
 12 complaints 1 officers 
Source: U.S. Commission on Civil Rights, Who is Guarding the 
Guardians? A Report on Police Practices (Washington, DC: Government 
Printing Office, 1981), p. 166.


The aftermath of the 1991 Rodney King beating heightened 
national awareness of the phenomenon of the problem officer. 
The Christopher Commission, appointed to investigate the Los 
Angeles Police Department following the King incident, 
identified 44 problem officers in the Los Angeles Police 
Department (LAPD) with extremely high rates of complaints.77 
In a particularly notable observation, the Christopher Commission 
pointed out that these 44 officers were "readily identifiable" on 
the basis of existing departmental records. Yet, the LAPD 
appeared to have made no effective use of these records to 
identify the officers and take any corrective action. The 
Commission found, for example, that citizen complaint data were 
not used in making routine performance appraisals.78 

A year after the Christopher Commission report, the Kolts 
Commission investigated the Los Angeles County Sheriff's 
Department (LASD), and using a similar methodology identified a 
group of 62 deputies with significant performance problems in the 
department. These 62 deputies were responsible for almost 500 
use-of-force/harassment complaint investigations. Seventeen of 
these deputies were responsible for 22 civil lawsuits that resulted 
in damage awards or settlements of about $3.2 million against the 
County. The Kolts Commission concluded that the LASD had 
"failed to deal with officers who have readily identifiable patterns 
of excessive force incidents on their records." 

Not only were nearly all of the 62 deputies still on patrol duty at 
the time of the Kolts report, buteven more alarminglymany 
served as field training officers (FTOs). The Kolts Commission 
warned that in that capacity they were "imparting their 'street 
wisdom' to patrol deputies."79 

The report recommended the creation of an EI system within the 
LASD to identify these deputies. The Kolts report staff "were 
struck that the LASD conducted no regular or meaningful analysis 
of lawsuits and administrative investigations" that might help to 
avert further misconduct and save the county millions of dollars. 
It concluded that "the concept of 'risk management' was largely 
unknown and unpracticed in the LASD."80 


Meanwhile, the Christopher Commission report spurred 
journalistic investigations of the problem officer phenomenon in 
other cities. The New York Times reported that in Kansas City, 
two percent of the sworn officers were responsible for 50 percent 
of all citizen complaints.81 A Boston Globe investigation found 
that 11 percent of Boston Police Department officers were 
responsible for 61.5 percent of all complaints.82 And in 
Washington, D.C., investigative journalists identified certain 
officers with long records of complaints and other problematic 
behavior.83 

The first formal evaluation of EI systems confirmed the anecdotal 
evidence regarding problem officers. Sponsored by the National 
Institute of Justice (NIJ) (and directed by the author of this 
report), this study evaluated the EI systems in the Miami-Dade, 
Minneapolis, and New Orleans police departments. It found that 
officers who had been selected for EI system intervention had 
substantially more serious performance records than did peer 
officers in their original recruit classes. Officers selected by the 
Miami-Dade Employee Information System (EIS), for example, 
averaged twice as many use-of-force incidents per year as their 
peers and were twice as likely to have been reprimanded and 
three times as likely to have been suspended.84 A full discussion of 
the findings of this study appears at the end of this chapter. 

The Early History of EI Systems 

Initial Experiments 

EI systems began to emerge in the 1970s as a crisis management 
response to public concern over police abuse of force. The 
histories of most of these programs are lost and may not be 
recoverable.85 In its 1981 report, the U.S. Civil Rights 
Commission cited with favor the EI systems in Oakland, New 
York City, and Kansas City.86 Little is known about those systems 
except for the fact that none appear to have survived very long. 
The Mollen Commission report on corruption and violence in the 
New York City Police Department in 1984, for example, offers no 
evidence of a functioning EI system.87 


Because of a Police Foundation evaluation, information about the 
original Kansas City program is available. The initial Kansas City 
early warning system was authorized in early 1972 and began 
operating in November, 1973, as part of an experimental Peer 
Review Panel Program.88 The Panel was part of the same effort to 
stimulate innovation in the Kansas City Police Department that 
included the well-known Kansas City Preventive Patrol 
Experiment.89 The Panel had the general goal of reducing tensions 
between the KCPD and the public and, to that end, sought to 
effect change in police officer attitudes and behavior through peer 
support and pressure. The program was designed to "identify 
those patrol officers with high frequencies of negative encounters 
with citizens." It used "no single fixed formula" for identifying 
officers with performance problems but utilized a combination of 
four separate criteria. These included: (1) three or more "negative 
encounters" with citizens, defined as both official citizen 
complaints and "interfering with an officer" charges; (2) referral 
by an immediate supervisor or other high-ranking department 
official; (3) an officer's "voluntary request for appearance;" (4) 
involvement in an incident of "public or departmental notoriety."90 

The Police Foundation evaluation found that the Peer Review 
Panels had no significant effect on reducing either citizen 
complaints or interfering with officer charges among those 
officers who were subject to the program. The ultimate fate of the 
Kansas City Peer Review Panel is not known. Given the fact that 
it was an experimental program, funded through a grant from the 
Police Foundation ($73,000 in 1973 dollars), and had no positive 
impact, it appears that the program lapsed.91 A 1991 New York 
Times article on the Kansas City EWS portrayed it as a new 
program, with no reference to the one cited by the Civil Rights 
Commission.92 

In Oakland, an experiment in utilizing police officers as change 
agents within their own department included one component 
designed to reduce citizen-officer conflict among officers with 
performance histories of frequent conflict. These officers were 
counseled by peer officers with exemplary performance records. 
The experiment concluded with some modestly optimistic 


conclusions about the effectiveness of peer counseling to deal 
with officers with performance problems.93 It appears, however, 
that once the formal experiment ended, the peer-counseling 
program ended as well. 

The 1981 report of the U. S. Commission on Civil Rights, Who is 
Guarding the Guardians?,94 which was based on extensive 
hearings on police misconduct in three major cities, took note of 
these early EW systems and recommended that all departments 
establish similar systems. Specifically, the Commission concluded 
that: 

The careful maintenance of records based on written 
complaints is essential to indicate officers who are frequently 
the subject of complaints or who demonstrate identifiable 
patterns of inappropriate behavior. Some jurisdictions have 
"early warning" information systems for monitoring officers' 
involvement in violent confrontations.95 

The First Permanent Systems: Miami and Miami-
Dade 

Two early EW systems that have had continuous operation from 
their creation to the present day, and about which detailed 
information exists, were developed in the City of Miami Police 
Department and the Miami-Dade (formerly Metro-Dade) Police 
Department in the late 1970s and early 1980s. The histories of 
these two systems reflect the process of development that has 
occurred in most of the early systems: the intersection of growing 
public pressure to curb misconduct and improve police-
community relations and creative leadership by a few key 
individuals inside the department, operating without the aid of any 
external models. 

City of Miami 

In 1979, the City of Miami Police Department became concerned 
with its officers' behavior that generated citizen complaints in the 
late 1970s. In a May 29, 1979 memorandum to the chief, the 
commander of the Internal Security Unit suggested an early 
warning system based on concepts of organizational development. 


This involved a "cyclical model where the problem is diagnosed, 
external professionals are consulted, strategies are developed, 
programs are implemented and evaluated, and results are fed back 
to begin the cycle again."96 

To illustrate his idea, the commander identified a list of officers, 
by assignment, who had two or more citizen complaints during a 
two-year period (1976-1978). He also compiled a list of officers 
who had received five or more civilian complaints during that 
period. Armed with these data and the Internal Security Monthly 
Activity Reports, the commander computed statistics indicating 
that five percent of all officers on the force accounted for 25 
percent of all citizen complaints. He noted that "if this group were 
suddenly removed from our department, our complaint picture 
could be reduced by as much as one-fourth. Obviously, this group 
should warrant some special attention, if we are to reduce our 
complaint incidence."97 The data also indicated that the small 
groups of officers with the most complaints were also more likely 
to receive complaints regarding excessive force. 

The result of this analysis was a proposal that the Miami Police 
Department should systematically provide supervisors with 
information "that can be used to identify problem officers."98 The 
IA commander also noted that off-duty employment, including 
rock concerts, wrestling matches, and football games generated a 
high number of citizen complaints. He reasoned that fatigue may 
"heighten an officer's opportunity to react in an aggressive 
manner."99 He suggested that the department should respond to 
these officers before they become involved in self-destructive 
activities or develop a trend of violating departmental orders. His 
proposal included more intensive supervision, counseling by 
outside professionals and training in tactics and strategies:100 

The Miami EW system eventually developed selection criteria 
including four categories of behavior: (1) citizen complaints, (2) 
control of persons (i.e., use-of-force) incidents, (3) official 
reprimands, and (4) firearms discharges.101 Once an officer is 
identified by the system, his or her supervisor is notified and is 
expected to meet with the officer and to determine if he or she 
needs any assistance, such as counseling, training, or other 


intervention. Officers identified by the EW system are then 
closely monitored. Supervisors are expected to investigate any 
problem incident and follow-up with a memorandum containing a 
recommendation. Internal Affairs provides the supervisor with a 
report of each incident that must be reviewed as well as the 
officer's assignment when the incident occurred. After evaluating 
the reports, the supervisor makes a recommendation that may 
include 1. Reassignment, 2. Retraining, 3. Transfer, 4. Referral to 
an Employee Assistance Program, 5. Fitness for Duty Evaluation, 
or 6. Dismissal Pursuant to Civil Rules and Regulations. The 
memorandum goes to the Commander of Internal Affairs through 
the chain of command. Each reviewing supervisor must agree or 
disagree with the recommendation.102 

The success of the Miami EW system, however, is uncertain. In 
2001-2002, the department faced a series of problems, including 
the indictment of a number of police officers on federal charges of 
abuse of authority. News media accounts indicated that some of 
the officers indicted had records of serious performance problems 
which presumably should have been identified by the EW 
system. In late 2002, the chief of police resigned in the wake of 
these events.103 

Miami-Dade 

The EW system in the Miami-Dade police department was 
developed at about the same time as the City of Miami's response 
to a number of racial incidents. (It is not clear what role the City 
of Miami's EW system played in influencing the development of 
the Miami-Dade system.) The beating of an African-American 
school teacher and the highly publicized fatal beating of Arthur 
McDuffie, an African-American insurance agent, by Miami-Dade 
officers, heightened racial tensions in the Miami area. On May 17, 
1980, the acquittal of four officers accused of McDuffie's death 
provoked three days of riots that resulted in civilian deaths and 
millions of dollars in property damage.104 As a result of these 
problems, a local ordinance directed the Miami-Dade Police 
Department to develop an Employee Profile System (EPS) that 
provides detailed information on each employee. In addition to 
demographic data, the EPS includes all complaints, use-of-force 


incidents, commendations, discipline, and disposition of all 
internal investigations. As an adjunct to the EPS, the department 
implemented an Early Identification System (EIS) under the 
supervision of the Internal Review Bureau. 

The EIS was operational by 1982. Officers are identified on the 
basis of two or more "personnel complaints" and/or three use-of-force 
incidents in the previous three months, and then listed on 
the next Quarterly Report. The reports are disseminated to 
supervisors to be used as they deem appropriate with regard to 
each officer. The "Action Alternatives" include referral to 
counseling for stress, referral to a 40-hour training assistance 
program, or counseling by supervisory personnel. The EIS has 
remained basically unchanged for the past two decades.105 The NIJ 
evaluation of the Miami-Dade EIS found that intervention 
succeeded in substantially reducing use-of-force among subject 
officers.106 

An Evolving Concept 

Without any guidance from recommended models, EI systems 
developed in an ad-hoc fashion. Over the years, a rough 
consensus began to emerge about the major system components. 
The most important change was an expansion of the number of 
performance indicators used by a system. Some initial systems 
(e.g., Minneapolis)107 used only citizen complaints. Others used 
citizen complaints and use-of-force reports. As time passed, there 
was increased recognition that one or two indicators are not 
adequate for identifying officers with potential performance 
problems. (This issue is discussed in greater detail in Chapter 
Four). Citizen complaints, for example, are filed in only a small 
number of potential cases, making them a very arbitrary indicator 
of officer performance.108 At the same time, it is increasingly 
recognized that other indicatorssuch as resisting arrest charges, 
use of sick leaveare very useful for identifying officers with 
potential performance problems. 


An Emerging Best Practice 

By the year 2001, EI systems had emerged as a recognized best 
practice in police accountability. A series of actions by the U.S. 
Department of Justice contributed to this development. A 1996 
DOJ conference on Police Integrity recommended that police 
departments explore EI systems as an accountability 
mechanism.109 A 2000 conference sponsored by the COPS Office 
included a discussion of EI systems.110 Finally, the Justice 
Department produced the 2001 report entitled Principles for 
Promoting Police Integrity. Other recommended best practices 
include a comprehensive use-of-force reporting system, an open 
and accessible citizen complaint process, and data collection on 
traffic stops.111 

In 1997, the Civil Rights Division of the DOJ entered into the 
first consent decree under Section 14141 of the 1994 Violent 
Crime Control Act. The consent decree with Pittsburgh included a 
requirement that the police department implement an EI system, 
which became known as the Performance Assessment and Review 
System (PARS). Subsequent consent decrees and memoranda of 
understanding have contained similar requirements. The consent 
decree with Los Angeles, for example, requires the LAPD to 
include seventeen performance indicators in its revised TEAMS II 
system.112 The memorandum of understanding with Cincinnati 
requires the police department to include ten performance 
indicators in its Risk Management System.113 

Finally, in 2001, the Commission on Accreditation for Law 
Enforcement Agencies (CALEA) adopted Standard 35.1.15 
requiring a Personnel Early Warning System.114 This 
recommendation by CALEA is particularly significant since the 
Commission represents the leaders of the law enforcement 
profession. 

In short, by 2001 a consensus of opinion had emerged among the 
U.S. Justice Department, federal courts, the law enforcement 
accrediting agency, and civil rights agencies that EI systems were 
a necessary part of a comprehensive approach to police 
accountability. 


Implementing EI Systems: A Major Challenge 

Although the concept of EI systems has become increasingly 
popular, the history of particular systems indicates that they are 
often extremely difficult to implement and maintain. When law 
enforcement agencies began to develop the first EI systems they 
were forced to do so without guidance from any published 
literature on the subject. There were no descriptions of systems, 
recommended models, or evaluations. Some departments simply 
devised their own systems, while others borrowed from other 
early systems.115 

With the benefit of hindsight, it is now clear that implementation 
is a major issue with respect to EI systems. (This issue is 
discussed in detail in Chapters Three and Four of this report.) The 
NIJ evaluation of EI systems in three cities concluded that EI 
systems extremely complex administrative tools that require 
considerable administrative attention.116 An effective EI system 
requires careful planning and close, ongoing administrative 
attention once it is established. The problems associated with the 
development and implementation of an EI system are illustrated 
by the histories of several large departments. 

The Los Angeles Sheriff's Department PPI System 

The Personnel Performance Index (PPI) in the Los Angeles 
Sheriff's Department (LASD) is widely regarded as the most 
sophisticated EI system in the country. One report described it as 
"without question, the most carefully constructed and powerful 
management tool for control of police misconduct currently 
available in the United States."117 Representatives from the LASD, 
for example, have been invited to describe the PPI at national 
level conferences on police accountability. The history of the PPI, 
however, indicates the difficulties in establishing a first-rate EI 
system. 

The idea for the PPI originated with the Kolts Report on the 
LASD in July 1992. The report pointed out that the LASD did not 
have an effective mechanism for identifying officers who posed a 
high risk to the department with regard to civil liability claims. 


One of the main reasons was that the department did not 
systematically collect and retain relevant performance data. Many 
important items, for example, were maintained on paper 3x5 
cards, and some of these records were periodically discarded. In 
order to achieve an effective risk management mechanism, the 
Kolts Report recommended the creation of what eventually 
became the PPI.118 

The LASD responded favorably to the Kolts recommendation and 
planned to have the PPI operational by 1993. Because of a 
number of delayssimilar to those that have affected other 
agencies-the target date for a fully operational PPI was pushed 
back to 1997. During this period, Los Angeles County settled a 
civil claim against the Sheriff's Department by promising to pay 
an additional $500,000 if the PPI was not fully operational by 
March 31, 1997. When that date arrived, the Special Counsel to 
the Sheriff's Department served as reviewer and concluded that 
the PPI system was indeed fully operational.119 

In short, it took almost five years for the LASD to implement its 
EI system. This occurred in the context of strong support from 
both the Sheriff and the County Board of Supervisors, and in an 
agency that is technologically sophisticated. 

Nonetheless, in early 2003 the Special Counsel to the LASD 
delivered a very critical report on the PPI. A number of 
administrative problems had weakened the effectiveness of this 
potentially outstanding system. In particular, staff cutbacks had 
made it difficult for the LASD to enter performance data into the 
PPI database, with the result that commanders were not able to 
conduct timely analyses of officers' performance. The Special 
Counsel also found that many supervisors in the department were 
not familiar with the PPI and its capabilities. Some district 
commanders had spent time and money developing their own 
local systems because they were not aware that the data they 
wanted was readily available through the PPI. The Special 
Counsel's report clearly indicated that for an EI system to be fully 
effective, a department needs to maintain adequate staffing for the 
system and to engage in a continuous process of training 
officersand commanders in particularon the nature and 
capabilities of the system.120 


The Los Angeles Police Department 

The story of the TEAMS in the Los Angeles Police Department 
(LAPD) represents a more disturbing picture of implementation 
problems associated with EI systems.121 It begins with the highly 
publicized beating of Rodney King on March 3, 1991. The 
Christopher Commission, appointed to investigate problems 
within the LAPD, found both that a small group of 44 officers had 
particularly serious performance records that were readily 
identifiable through existing departmental records, and that the 
department took no effective action to either discipline or correct 
the behavior of these officers. To address this problem, the 
Commission recommended the creation of an EI system.122 

The LAPD's EI system became known as TEAMS (Training 
Evaluation and Management System). As a report by the Los 
Angeles Police Commission documents, development of the 
system over the next few years was erratic at best. In 1996, the 
Police Commission hired Merrick Bobb (then also the Special 
Counsel to the LASD) to conduct a five-year update on 
implementation of Christopher Commission recommendations. 
Bobb reported that the LASD did not have an operational EI 
system as intended by the Christopher Commission and that 
TEAMS was at that point "weak and inadequate."123 

The president of the Los Angeles Police Commission then 
initiated a request for a federal grant to facilitate the 
implementation of an expanded version of TEAMS. The U.S. 
Department of Justice awarded a grant of about $175,000 in 
September, 1997, and the new EI system became known as 
TEAMS II. In 1999, the Rampart scandal embroiled the LAPD in 
further controversy, prompting three separate investigations of the 
department (one by the LAPD itself,124 one by the L.A. Police 
Commission,125 and one sponsored by the police union126). The 
Police Commission's report documented the history of the 
TEAMS II system and the fact that it was not currently 
operational. In fact, the city had not even drawn on the federal 
grant awarded to implement TEAMS II. 


Meanwhile, the U.S. Justice Department sued the City of Los 
Angeles and the LAPD under the 1994 Violent Crime Control 
Act, alleging a pattern or practice of abuse of citizens. The suit 
was settled through a consent decree that closely resembled 
similar decrees and memoranda of understanding negotiated by 
the Justice Department. The consent decree mandated the LAPD 
to complete the planned implementation of TEAMS II and 
specified that it should incorporate 17 separate performance 
indicators.127 

In short, eleven years after the 1991 Christopher Commission 
recommendation, the LAPD did not have a fully operational EI 
system. 

Pittsburgh Police Department 

The Performance Assessment and Review System (PARS) in the 
Pittsburgh Police Bureau represents a case of a successful 
implementation. The system originated with a 1997 consent 
decree settling a U.S. Department of Justice suit against the 
department for failure to address excessive force problems. The 
Pittsburgh suit was the first to be settled under the provisions of 
Section 14141 of the 1994 Violent Crime Control Act authorizing 
the DOJ to bring suit where there is a "pattern or practice" of 
violating citizens' rights.128 

The Pittsburgh consent decree has served as a model for 
subsequent agreements in Section 14141 litigation. Later 
agreements are very similar in overall content, but tend to be 
much longer and more detailed than the Pittsburgh agreement.129 

The Pittsburgh PARS includes data on eighteen performance 
indicators. Officers are identified by the system if their number of 
indicators is one or more standard deviations greater than their 
peer officersthus, night shift officers are compared with other 
night shift officers. One of the most significant aspects of PARS is 
that it is incorporated into the routine supervision process in the 
department. Supervisors are directed to log on to the system each 
day prior to roll call. As a result, each supervisor is able to note 
each and every activity that might require supervisory attention. 


The incidents are not limited to those officers who have been 
previously identified by PARS. A Vera Institute evaluation 
characterized the PARS process in terms of a "sweeping change in 
the duties of the lowest level supervisors."130 

Implementation of the Pittsburgh PARS has been subject to two 
separate monitoring and evaluation procedures. The consent 
decree required the appointment of an independent monitor to 
oversee implementation of the decree and to file semi-annual 
reports with the court. The monitor found the Pittsburgh police 
department to be in compliance with the decree, including the 
provisions related to PARS, and in the summer of 2002 the court 
lifted the decree (for the police department but not for the 
independent Office of Municipal Investigations which is 
responsible for handling citizen complaints).131 

Meanwhile, the consent decree and its implementation were 
evaluated by the Vera Institute of Justice. The evaluation, 
completed in 2002, found not only that the consent decree was 
successfully implemented, that it had a positive effect on many 
aspects of the police department, and that a number of alleged 
unintended consequences did not in fact occur. The Vera 
evaluation found that the EW system was functioning effectively 
as intended.132 

The successful implementation of the EW system in Pittsburgh 
was due in large part to what the Vera Institute described as a 
"dogged determination" on the part of the police chief and other 
city officials to comply with the terms of the consent decree.133 
The lesson of the Pittsburgh experience is that a sophisticated EI 
system can be successfully implemented in a large police 
department in a relatively short period of time, given sufficient 
leadership and resources. 

Research on the Effectiveness of EI Systems 

There are five sources of evidence on the effectiveness of EI 
systems. They include: (1) the NIJ study of three major police 
departments, (2) an evaluation of the consent decree mandating 
the creation of PARS in the Pittsburgh Police Bureau, (3) the 


reports of the Special Counsel to the Los Angeles Sheriff's 
Department which include periodic assessments of the LASD PPI, 

(4) anecdotal evidence from other departments, and (5) the 
UNO/PERF survey of police managers. The first three are 
discussed below. The UNO/PERF survey is reported in detail in 
Chapter Four. 

(1) The NIJ Study of Early Warning Systems 

The National Institute of Justice (NIJ) study was conducted by the 
author of this report and involved the EW systems in three large 
police departments: Miami-Dade, Florida; Minneapolis, 
Minnesota; and New Orleans, Louisiana. The study found all 
three EI (the earlier NIJ report used the term Early Warning, but 
this report uses the now-preferred term Early Intervention) 
systems to be effective in reducing citizen complaints and use of 
force among officers subject to intervention.134 

The NIJ study included both qualitative and quantitative 
components and a number of previously unacknowledged issues 
were identified in the course of the evaluation. In many respects, 
the qualitative findings on the administration of EI systems are 
more significant that the quantitative findings on the impact of EI 
system intervention on individual officers. Most important, the 
study found that EI systems are extremely complex administrative 
tools that require close ongoing administration. Additionally, the 
authors concluded that it was difficult to separate the process and 
impact of an EI system from the larger climate of accountability 
within a police department. 

Qualitative Analysis of the Three EW Systems 

Each of the three police departments had very different histories 
with respect to accountability in the years immediately preceding 
the evaluation, and these histories affected the nature of their 
respective EI systems. 

The Miami-Dade police department had been in the process of 
enhancing accountability for almost twenty years. The department 
had a comprehensive personnel database in place that served the 


EIS but was independent of it. The Minneapolis Police 
Department, meanwhile, was in the process of raising its 
standards of accountability, particularly with respect to the use of 
force. The current chief was appointed in large part because of 
criticism of the lax disciplinary policies of the previous chief. The 
New Orleans police department, meanwhile, was also under the 
direction of a new chief executive who had been appointed to 
reform the department following a number of force and corruption 
scandals that generated considerable national publicity. The NIJ 
study concluded that the EI systems in each of the three 
departments were integral parts of ongoing departmentwide 
accountability efforts. In such a context, it is difficult to specify 
the independent effect of the EI system separate from other 
reforms that occur in the department generally. 

As has already been pointed out, the qualitative research also 
found that EI systems are far more complex administrative 
mechanisms than is generally recognized. The study found that EI 
systems require enormous administrative attention if they are to 
be implemented and maintained effectively. The study dispelled 
the notion that an EI system is simply a tool (e.g., an alarm clock) 
that can be purchased off the shelf, plugged into the department, 
and then left to run by itself. (Chapter Three of this report 
provides even stronger documentation on the importance of 
planning and administrative follow-through.) 

Evidence of the administrative complexity of an EI system is 
illustrated by several examples drawn from the three departments 
studied. One of the EI systems had actually ceased operation for a 
period of about one year because of disruptions related to 
administrative changes in the department. One of the EI systems 
had undergone considerable revision and improvement since it 
was first launched. The officials in charge called their efforts "a 
work in progress." One of the EI systems studied had serious 
problems related to inputting data in a timely fashion. Two of the 
departments studied had rudimentary computer systems for their 
EI systems, such that it was possible to retrieve the files for 
individual officers but not to generate data on larger patterns and 
relationships between incidents and officers. 


Quantitative Evaluation 

The quantitative component of the evaluation attempted to assess 
the impact of each EI system on officers who were subject to 
departmental intervention. Because of the nature of the individual 
EI systems and the data available, two of the EI systems were 
subject to one mode of evaluation and the third subject to a 
different mode. 

In Miami-Dade and Minneapolis, the evaluation methodology 
consisted of an analysis of the performance records of all officers 
hired in certain years (1990-1992 in Miami-Dade; 1991-1992 in 
Minneapolis). Each cohort of officers was partitioned into those 
officers who had been subject to the EI system and their 
colleagues who had not been subject to it (non-EI). The two 
groups were then compared in terms of background characteristics 
and performance records. The performance records of the EI 
group were analyzed in terms of pre-intervention and post-
intervention. 

The nature of the New Orleans PPEP, which involved a four-day 
class with a group of officers, offered a unique opportunity for a 
different methodology. First, one PPEP class was directly 
observed for the purpose of collecting data on the content of the 
intervention and the observed responses of officers in the class. 
Second, because all officers subject to PPEP classes are invited to 
complete a course evaluation form, it was possible to analyze the 
responses of all officers who had ever been subject to the PPEP 
intervention. 

Findings 

(1) Officer Backgrounds 

The data in Miami-Dade and Minneapolis found that race or 
ethnicity was not a factor in with regard to those officers subject 
to EI intervention. White, African American, and Hispanic 
officers were selected for EI in proportions closely related to their 
presence in the larger cohort. This finding is particularly 
significant because it does not support the popular view that the 


problem of police use of excessive force is primarily a matter of 
white police officers mistreating racial and ethnic minorities. It is 
consistent with other research indicating that race or ethnicity is 
not a determining factor with regard to law enforcement officer 
behavior.135 

Female officers, meanwhile, were only about half as likely to be 
subject to EI intervention, as their presence in the larger cohort 
would have predicted. This finding is consistent with other 
research indicating that female officers are less likely to be 
involved in use of force incidents and are rarely identified as 
officers with patterns of repeated use of excessive force.136 

(2) Performance Histories 

Officers selected for intervention by the EI systems in Miami-
Dade and Minneapolis did in fact have significantly worse 
performance histories prior to intervention when compared to 
their colleagues. 

In Miami-Dade, for example, officers selected by the department's 
Early Intervention System (EIS) averaged twice as many use-of-force 
reports per year as non-EIS officers. EIS officers were twice 
as likely to have received four or more citizen complaints than 
their peers, were twice as likely to have received a formal 
reprimand, and almost three times as likely to have been 
suspended. In Minneapolis, officers selected by the department's 
EI system averaged twice as many complaints as their non-EI 
peers, were slightly more likely to have been reprimanded, and 
were three times as likely to have been suspended. In one 
troubling finding, EI officers in both Miami-Dade and 
Minneapolis were slightly more likely to be promoted than their 
non-EI peers, although in Miami-Dade the difference was not 
statistically significant. This issue merits further research. The 
data from this study does not permit investigation into the reasons 
for this finding. It is possible that for some officers, the negative 
aspects of assertive behavior that results in EI system 
identification also includes positive aspects that help earn 
promotion. Future research and policy development can, 
hopefully, permit EI systems to identify and curb the negative 


aspects of assertive behavior while retaining the positive aspects 
of officer assertiveness. 

This finding lends support to the anecdotal view, supported by 
investigative journalists' reports that a small group of officers in 
any department do have significantly worse performance records 
than their colleagues. 

(3) Impact of Intervention 

The evaluation found that in Miami-Dade and Minneapolis the EI 
systems had significant and positive impacts on officers' 
performance histories. Among officers subject to EI system 
intervention, citizen complaints and involvement in use-of-force 
incidents declined by about two-thirds. In Miami, 27 of the 28 
EIS officers had at least one use-of-force report; following 
intervention only 14 had a use-of-force report. Twenty of the 28 
officers had four or more complaints prior to intervention, while 
only nine of the 28 had four or more complaints following 
intervention. In Minneapolis, EI officers received an average of 

1.9 complaints per year prior to intervention and 0.65 per year 
after intervention. 

One curious aspect of the findings is that the improvements in 
officer performances were virtually identical in Miami-Dade and 
Minneapolis despite the fact that the two EI systems had very 
different administrative histories. The Miami-Dade system had 
been in place for over 15 years and was well-integrated into the 
department. The Minneapolis system, on the other hand, was far 
less sophisticated (with regard to performance indicators) and had 
experienced some significant disruptions. One possible 
explanation is that an EI system works effectively with many (and 
perhaps most) officers simply by putting them on notice (e.g., 
warning) and that an elaborate intervention is not necessary. 

(4) Officers' Responses to Intervention 

The nature of the New Orleans PPEP made it possible to both 
observe officers in the PPEP class and to analyze responses to the 
course evaluation. This component of the evaluation yielded 
extremely important findings. 


First, the direct observation of the class found that officers 
responded in very different ways to different units and teaching 
methodologies. During the class, most of the officers expressed 
open hostility to the PPEP, referring to it as "politeness school," 
or "bad boys class." Officers were visibly disengaged from 
aspects of the course that involved traditional lectures and in 
particular lectures about general topics such as the role of the 
police, stress management, and the problem of substance abuse in 
American society. Some officers appeared to be close to falling 
asleep in the class during these units. Some expressed their 
disinterest through obvious body language. 

In sharp contrast, however, they were actively engaged in the unit 
of the course that asked them to critique specific police-citizen 
encounters. The case studies were drawn from the department's 
files, and in a few cases the officers claimed to remember the 
incident (none involved officers in the class). In this exercise, the 
officers appeared to be knowledgeable and professional police 
officers, fully aware of proper procedure in difficult situations, 
and proud of their own knowledge about how to control situations 
effectively. It should be noted that this engagement was exhibited 
by the same officers who had earlier voiced extremely hostility to 
the PPEP. 

The lesson of this finding is that interventions that involve 
traditional lectures are not likely to be effective in engaging 
officers about performance issues. The use of active learning 
techniques that allow officers to draw on their experience and 
knowledge, on the other hand, appear likely to engage their 
attention. In the observed case, however, it is not known whether 
the officers in the class internalized the lessons from the case 
studies and altered their own behavior accordingly. 

With respect to the evaluations of the PPEP classes, officers who 
completed the evaluations gave the class an extremely positive 
rating. On a scale of 1-10, they gave it an average rating of 7.0. 
When the four officers who gave it a rating of one (the lowest 
possible rating) are omitted, the average rating rises to 8.1. These 
responses are extremely significant in light of the fact that the 
officers in the observed class openly disparaged the PPEP 
program. 


The positive responses on the numerical evaluation were 
supported by the narrative comments on the evaluation forms. All 
of the officers made at least one positive comment about the class. 
Comments included favorable references to the verbal 
judo/complaint reduction component and the stress reduction 
component. About three-quarters of the officers also made 
negative comments. Significantly, however, all the negative 
comments were about the PPEP in general or the department and 
its leadership. There were no negative comments about the PPEP 
class per se. 

The positive evaluations of the PPEP classes are an extremely 
important finding. It suggests that most officers appreciate 
practical assistance designed to help improve their performance. 
This finding is reinforced by the PERF survey of police managers 
(See Chapter Three) which found no major hostility to EI systems 
from rank and file officers. 

Limitations of the Study 

While extremely encouraging, the evaluation was subject to a 
number of limitations that need to be noted: 

First, this was a retrospective study utilizing available official 
data. Consequently, it was subject to the limitations of that official 
data. While the study sought to control for important variables, 
the investigators are not 100 percent certain that all the data are 
accurate. One particularly important issue involves officers' 
assignment. Reassignment to a low-crime area could account for a 
significant drop in citizen complaints and use-of-force incidents. 
Official records with regard to assignment histories, while good, 
are not necessarily perfect. 

Prospective evaluations in which investigators can monitor 
assignment patterns and other relevant factors can reduce or even 
eliminate these problems. 

Second, the design of this study does not make it possible to 
determine exactly what aspect of the EI intervention accounts for 
the positive results. It is entirely possible that for many officers 


simply being selected by the EI system was a sufficient warning 
that led to improved performance. A number of police officials 
involved with EI systems believe that this is true. The effect here 
is independent of the actual content of the intervention. It is 
equally possible, however, that some officers benefited from 
specific aspects of the intervention: advice on how to de-escalate 
confrontations, or to handle groups of troublesome citizens. It is 
entirely likely that different officers respond differently. 

Future research on EI systems should focus on specifying what 
kinds of intervention are appropriate and effective for different 
kinds of officers. It may also be useful to think in terms of triage: 
that some officers may need only a warning, that others may need 
help with specific performance issues, and that some may not 
respond at all to intervention. 

(2) Evaluation of the Pittsburgh PARS 

The Pittsburgh PARS has been evaluated in two different ways. 
First, the consent decree included a court-appointed monitor to 
oversee implementation of the decree. The reports of the monitor 
found that the Pittsburgh Police Bureau successfully complied 
with the terms of the decree, including implementation of the 
PARS. The decree was lifted in the summer of 2002. 

Second, the Vera Institute and PARC conducted a general 
evaluation of the impact of the 1997 consent decree settling a 

U.S. Justice Department pattern or practice suit against the 
department. The evaluation found that PARS had a generally 
positive impact on the department. Several supervisors testified 
that it did enhance the quality of supervision in the department. At 
the same time, they expressed concern about the burden of paper 
work required by PARS. One supervisor offered an unverified 
estimate that the amount of time sergeants devoted to traditional 
street-level supervision declined from 70-80 percent of total time 
to 25-30 percent.137 

An important aspect of the Vera/PARC evaluation was an analysis 
of some departmentwide activity data (although trends in these 
data are not necessarily attributable to PARS alone). The data 


found, for example, that despite anecdotal reports of a serious 
decline in officer morale because of the consent decree, the 
number of sick days taken by officers actually declined from 1995 
to 1999. Also, there was no upsurge in separations from the 
department between 1996 and 1999, despite anecdotal reports to 
the contrary. The data indicated a decline in arrests beginning in 
1994, but this trend paralleled a decline in reported crime and 
could not be attributed to the consent decree. Finally, there was no 
decline in traffic tickets issued following the 1997 consent decree 
(a sharp decline occurred in 1996 as a result in a change in the 
rules for overtime pay). 

(3) Assessment of the LASD PPI System 

The Special Counsel to the Los Angeles Sheriff's Department 
reports data indicating that the PPI is very effective in achieving 
its goals. The Special Counsel a form of external citizen oversight 
for the LASD and operates through a contract with the Los 
Angeles County Board of Supervisors. The Special Counsel 
reported monthly average data on officers subject to Performance 
Review (PR) for periods three years prior to PR, the period during 
PR, and the period after PR. The rate of officer-involved 
shootings dropped from an average of .50 per month before PR to 
zero following PR. Use-of-force incidents dropped from an 
average of 7.11 per month before PR to .98 per month after PR. 
And personnel complaints dropped from an average of 3.86 to .74 
in the same time periods.138 

One interesting finding was that 11 percent of all officers placed 
on Performance Review left the department within a short period 
of time. This included ten retirements, six discharges, one 
resignation, and one departure for reasons that were not clear. In 
short, two-thirds were voluntary departures.139 To the extent that 
the PPI encourages problem officers to leave the department 
voluntarily, it saves the department both the direct expense of 
conducting a full two-year Performance Review and the risk that 
an officer will engage in other serious misconduct. 

The 16th Semiannual Report of the Special Counsel, however, was 
very critical of the PPI. As noted earlier, it found that the PPI had 
not been fully staffed and as a result performance data were not 


being entered into the database in a timely fashion. Also, many 
commanders were not aware of the nature and capabilities of the 
system.140 As a result of the Special Counsel's report, LASD 
commanders made a public commitment to make the necessary 
changes to ensure that the PPI was fully effective and capable of 
achieving its basic goal of reducing litigation costs to the county. 
The Special Counsel's report highlighted the administrative 
demands of an EI system and the need to continuously oversee 
the operation of a system. 

(4) Anecdotal Evidence From Other Departments 

At the Early Intervention State of the Art Conference in early 
2003, commanders from several EI systems were able to describe 
case studies of individual officers who had been helped by EI 
intervention. One commander, for example, described the case of 
a female officer who was not engaging citizens in a properly 
assertive manner, a practice that encouraged aggressive behavior 
on the part of some citizens (and resulting uses of force by the 
officer). Intervention counseling determined that the officer had a 
great fear of being struck in the face. Training helped the officer 
become skilled in the proper techniques for engaging potentially 
aggressive citizens. As a result, the officer's performance 
improved dramatically. The commander from another department 
described the case of an officer whose performance was being 
affected by serious personal financial difficulties. Intervention 
counseling helped the officer obtain professional financial advice 
and his performance subsequently improved.141 

The two cases cited above are anecdotal and not sufficient to 
validate the overall effectiveness of an EI system. Discussions at 
the EI State of the Art Conference explored the possibilities of 
developing more systematic ways of documenting individual 
success stories (with personal identifiers eliminated to preserve 
privacy). 

(5) The Police Managers' Survey 

The findings of the police managers' survey, which includes both 
managers experienced with EI systems and managers without 
such experience, is reported in Chapter Four of this report. 


Conclusion 

EI systems originated as administrative tools for identifying a few 
officers with performance problems. They have evolved into more 
elaborate tools for reviewing officer performance and achieving 
higher standards of accountability. Research to date indicates that 
EI systems can be effective in achieving their goals. At the same 
time, the history of EI systems in several departments indicates 
that EI systems are difficult to implement effectively. Careful 
planning and continuous administrative oversight is necessary. 


Chapter Four 

Lessons From the Field: Police Managers Perceptions 
and Experiences With Early Intervention Systems The PERF Survey....................73
Part One: Experiences With Early Intervention Systems .............................74
Source of their EI Systems ........................................................76 
Overall Assessment of the System...................................................76
Impact on the Quality of On-the-Street Police Service..............................76
Positive Contributions to Supervision and Accountability...........................76
Strengthens the Role of Supervisors ...............................................77
Early, Non-Disciplinary Intervention ..............................................78
Ability to Track Officers Who Change Assignments...................................78
Problem-Solving Supervision........................................................78
Timely, Objective Data ............................................................79
Lack of Opposition From Rank and File and Unions ..................................79
More Positive Than Negative Impact on Officer Morale ..............................80
Changes in Officers Attitudes ....................................................80
Differential Impact on Officers ...................................................81
No Problems Regarding Confidentiality .............................................81
Problems With EI System............................................................81

Intervention SystemsImplementation Problems........................................82
Lack of Commitment by the Deparment ...............................................83
Infrastructure Needs...............................................................83
Successful Planning ...............................................................83
Need for Continued Revision and Improvement........................................83
What Needs to be Changes or Improved...............................................84
Part Two: Departments Without Early Intervention Systems ..........................84
Managers Knowledgeable About EI Systems............................................84
Managers Overwhelmingly Anticipate Positive Impact of an EI System.................85
Managers Anticipate Objective Systematic Data......................................85
Managers Anticipate Positive Impact on Supervisors.................................86
Managers Anticipate a Reduction in Lawsuits Against the Department.................86
Managers Anticpate Multiple Benefits...............................................86
Anticipate a Positive Impact on Community Relations ...............................87
Overestimate the Opposition From Within the Department.............................87
Understand Opposition Will Come From the Officers With Performance Problems .......87
Overestimate Union Opposition......................................................88
Underestimate Administrative Problems..............................................88
Selling the Idea: Benefit to Subject Officers......................................88
Selling the idea: Makes the Supervisors Job Easier ...............................89
Recognize Limitations of Current Accountability Mechanisms ........................89
Able to Track Officers Who Change Assignments .....................................90
Reactive Versus Proactive..........................................................90
Prefer Information From Other Law Enforcement Agencies ............................90
Conclusion ........................................................................91

Lessons From the Field: 
Police Managers Perceptions and Experiences 
With Early Intervention Systems 

After more than a decade of experimentation and development (and in 
some departments almost two decades), law enforcement agencies have 
a considerable body of practical experience with Early Intervention 
systems. 

To tap into this experience, we surveyed law enforcement managers. As 
described in detail below, this survey used innovative online techniques, 
surveying members of the Police Executive Research Forum (PERF). The 
methodology and principal findings are reported in the following sections. The 
survey used the term Early Warning, and consequently that term is often used 
in this chapter. 

The PERF Survey 

This survey represents collaboration between the University of 
Nebraska at Omaha, the Police Executive Research Forum 
(PERF), and DiscoverWhy. Funding was provided by the Office 
of Community Oriented Policing Services (COPS) of the U.S. 
Department of Justice. 

DiscoverWhy is a private survey research firm developing 
innovative online technologies for surveying public opinion. This 
project represents an experiment in applying these technologies to 
law enforcement and criminal justice research. PERF is a 
professional association of police managers, with a membership of 
nearly 600 commanders and supervisors across the country. PERF 
has been involved in cutting edge research, training and 


demonstration projects since 1975. The COPS Office provided a 
grant to the Criminal Justice Department of the University of 
Nebraska at Omaha to provide technical assistance to law 
enforcement agencies regarding Early Warning (EW) systems. 
This survey represents one component of that project. 

PERF provided a list of email addresses for its members. Each 
member then received an email invitation to participate in the 
survey. Those individuals indicating they wished to participate 
then received a password allowing them access to the survey 
instrument. Participants could then log on and participate at their 
convenience. The DiscoverWhy technology permits an electronic 
survey, with both closed and open-ended questions, that may be 
accessed and completed at the convenience of each participant. 
The open-ended questions are particularly valuable for tapping 
qualitative assessments of the subject (in this case, EI systems) 
and in the process eliciting valuable data embedded in mixed and 
ambiguous responses. PERF submitted a mailing list of 831 
names. Email invitations were sent to 521 members with currently 
active email addresses. A total of 135 members participated in the 
survey, for a response rate of 26 percent. About 40 percent of the 
respondents work in departments with EI systems, while 60 
percent work in departments without EI systems. This distribution 
is consistent with the findings of an earlier survey of EI 
systems.142 

Part One: Experiences With Early 
Intervention Systems 

Law enforcement managers who have had practical experience 
with an EI system report very positive assessments of its impact 
in their departments. They report that it has had a positive impact 
on supervision and accountability and on the quality of on-the-street 
police work. Especially important, they have not 
experienced serious opposition from rank and file officers or their 
police unions. Many, in fact, report that the attitudes of rank and 
file officers changed from suspicion to understanding and support 
for the EI system. 


Experienced managers also report a number of problems with EI 
systems, however. The most important problems fall into two 
general categories: a failure to communicate the nature and 
purpose of these systems during the planning stage and a failure 
of some command officers to follow through on their 
responsibilities once the system begins operating. In short, these 
are management problems and not problems inherent in EI 
systems. 

General Assessment of EI System 
Image of a bar showing showing the general assessment of EI System

Impact of EI System on Quality of Police Service 
Image of a bar graph showing the impact of EI System on Quality of Police Service


Source of their EI System 

Most departments (70 percent) developed their own EI systems, 
while 20 percent borrowed theirs from other departments. 

Overall Assessment of the System 

Experienced police managers report overwhelmingly positive 
experiences with their EI systems: 64 percent reported a positive 
general assessment and 32 percent reported a mixed general 
assessment. The nature of these mixed assessments is explored in 
responses reported below. 

Impact on the Quality of On-the-Street Police 
Service 

Managers overwhelmingly reported that the EI system had at least 
some positive impact on the quality of on-the-street police 
service. Almost half (49 percent) reported a positive impact, while 
28 percent reported a mixed impact. This represents a combined 
total of 77 percent of respondents who report at least some 
positive impact. Perhaps most important, no respondents reported 
a negative impact on the quality of on-the-street police service. In 
particular, no manager reported that the EI system caused officers 
to back off and reduce their activity level. 

Positive Contribution to Supervision and 
Accountability 

Managers report that their EI systems have made positive 
contributions to supervision and accountability in their 
departments. Comments from these managers include: 

 The process assisted us in identifying officers or non-sworn 
personnel who have field responsibilities who began to 
demonstrate behavior not consistent with our policies and 
standards. 


 Raises the awareness of lack of proper supervision. 

 Supervisors pay more attention to what is going on, document 
all activities more thoroughly, and talk with officers when things 
might look like a problem. 

Early, Non-Disciplinary Intervention 

Mangers report that, consistent with the objectives of EI systems, 
the EI system has been an effective instrument for early, non-
disciplinary intervention with officers who exhibit performance 
problems. Two managers, for example, reported that: 

 "We have been able to intervene with help before misconduct 
occurs that requires discipline." 

 "[The] program provides a way for the department to provide 
non-disciplinary direction and training before the officer 
becomes a liability to citizens, the department, and him/herself" 

Ability to Track Officers Who Change 
Assignments 

One problem in police departments is that because of regular 
changes in assignment many supervisors do not know the 
performance histories of new officers under their command. An 
EI system database overcomes this problem. 

 
There is a lot of movement of personnel, so supervisors often do 
not know the histories of their officers. The EWS report brings 
them up to speed in a much more timely fashion. 
 
No one falls through the cracks any longer. 

Problem-Solving Supervision 

Some managers understand that an EI system facilitates a 
problem-solving approach to supervision. 

 A useful tool to involve supervisors and lieutenants in non-traditional 
model of problem-solving. It has served to enhance 
their management skills and help round out their people 
interaction skills. 


Timely, Objective Data 

Managers report that having objective performance data helps to 
eliminate charges of favoritism, levels the playing field since no 
one can be accused of playing favorites, and makes it much easier 
to deal with the police union. 

Lack of Opposition From Rank and File and 
Unions 

The most surprising finding is that managers report that they have 
not experienced opposition to the EI system from their local 
police union. This finding is particularly significant because 
managers who do not presently have an EI system in place 
anticipate such problems (see below). Eighty-four percent of the 
managers with EI systems report having had no problems with 
their unions regarding their systems. 

Problems With Police Union? 
Image of a bar graph showing the percentage of problems with police unions


 As command staff members, we are constantly looking for ways 
to pick up on officers in need of assistance. The Early Warning 
System has picked up on several officers in the department who 
were headed down the wrong path 

Strengthens the Role of Supervisors 

Police managers report that their EI systems strengthen the role of 
supervisors. Some of their comments include: 

 
A useful tool to involve supervisors and lieutenants in a non-traditional 
model of problem solving. It has served to enhance 
their management skills and help round out their people 
interaction skills. 
 
Sergeants have been able to evaluate the strengths and 
weaknesses of their squads even before meeting with them, and 
therefore are able to develop proactive strategies to address 
personnel and leadership issues. 
 
When an officer is placed in the program, for whatever reason, 
his or her supervisor is also involved in the process. The Early 
Warning System Board also looks at the actions of the 
supervisor to determine if the supervisor is providing the 
supervision needed to correct the problem. 

Impact on Management 
and Supervision
Image of a bar graph showing the impact of management and supervision


One of the few managers who encountered problems with the 
union stated: 

"The union believes that the EWS is a witch hunting tactic, 
used by the administration to target disliked officers. In fact, 
no one has been disciplined because of EWS, it just wasn't 
explained very well." 

More Positive Than Negative Impact on Officer 
Morale 

Twice as many managers (12 percent) reported that the EI system 
had a positive impact on morale among rank-and-file officers as 
reported a negative impact (six percent). Slightly more than half 
(52 percent) reported that the EI system had a mixed impact on 
officer morale. Finally, 30 percent reported that the EI system had 
no impact on rank and file morale. 

Impact on Officer Morale 
Image of a bar graph showing the impact on officer morale

Explaining the positive impact on morale, one manager pointed 
out that "nothing hurts morale like no action being taken against 
problem officers. It shows supervision that problem people can be 
dealt with." 


Changes in Officers' Attitudes 

Managers report changes in rank and file officers attitudes, from 
suspicion or hostility to growing support for EI systems. 

 
The system was met with much cynicism and distrust when first 
introduced. Experience and education has reduced those 
attitudes. 
 
Most no longer believe this system is out to "get" them, but 
rather to assist them. 

Differential Impact on Officers 

Some mangers reported that the impact of the EI system varied 
from officer to officer. They reported, for example, that: 

 
Some officers take it to heart, others see it as unfair discipline. 
 
The solid officers of the department are supportive of the system 
both conceptually and practically. 

No Problems Regarding Confidentiality 

Virtually no problems were reported with regard to breaches of 
confidentiality, such as improperly leaking information about 
which officers had been identified by the EI system. In fact, the 
only reported case involved a situation where the officer subject 
to intervention disclosed that information himself. 

Problems With EI System 

The overwhelming majority (85 percent) reported that they had 
experienced some problems with their EI system. Only 15 percent 
said they had had no problems. 


Problems With EI System?
Image of bar graph showing problems with EI System

Implementation Problems 
Mangers did report a number of problems with their EI systems. 
Virtually all of these problems were related to implementation 
issues, in particular a lack of communication about the nature of 
the EI system and a lack of follow-through on the part of 
responsible officials. 

The greatest number of problems involved a failure to 
communicate the nature and purpose of the EI system to officers 
in the department. Commanders identified this problem in 
response to several different questions on the survey. Some 
commanders explained, for example: 

 
[Top management] did not explain the purpose of the program 
well [to mid-level supervisors]. 
 
[Departments needed to deliver better] training and introduction 
to the first line supervisors. 
 
[The agency] could have done a better job of pre-selling and 
training on the benefits of an EWS. 
 
While the system was crafted appropriately, it was not explained 
to the officers or the first line supervisors to the extent 
necessary to make it an understandable and viable system. 
 
The system is misunderstood by the first line supervisors, and 
therefore retraining is needed at the application level. 


Lack of Commitment by the Department 

Another problem reported by experienced managers is that while 
the EI system is good in principle, the department as a whole and 
some mid-level supervisor did not follow through with their 
responsibilities, thereby undermining the effectiveness of the 
system: 

 
The program does not hold a prominent place in the 
organization. As a result, no results are provided to commanders 
as to its effectiveness or benefits. 
 
Our program lacks real involvement from the commanders. 
Typically the officer's actions are justified or rationalized by the 
commanders and rarely is there a consequence for the officer. 
 
[There was] irregular review and lack of consistent follow up. 

Infrastructure Needs 

The lack of an adequate technological infrastructure was also a 
major problem. About 43 percent of the managers reported that 
their department did not have the necessary infrastructure in place 
when their department began planning its EW system. 

Successful Planning 

Many managers reported that their EI planning effort was 
successful. The key to success was the involvement of all relevant 
stakeholders: 

 
Gathering input from all involved, officers through management 
as well as the union. 
 
We involved Risk Management, the Deputies Union, and 
management. 
 
Involving a wide range of personnel and viewpoints from the 
start. 

Need for Continued Revision and Improvement 

Several managers acknowledged that their EI systems were not 
perfect and that further revision and improvement was necessary. 


"The system has worked well. EWS needs to continuously 

evolve and expand." 

What Needs to be Changed or Improved? 

When asked what they thought needed to be changed in their EI 
systems, managers most commonly reported that communication 
problems needed to be addressed: 27 percent indicated that 
improved communication between command staff and rank-and-file 
was their most pressing need, while 14 percent said that it 
would be most important to improve communication among 
command staff. 

The need to clarify certain aspects of the EI system was also 
mentioned: 22 percent stated that it would be most important to 
clarify the relationship of the EI system to the department's formal 
disciplinary system, while 19 percent report that it would be most 
important to clarify the purpose of the EI system. These problems 
are also essentially communication problems. 

Part Two: Departments Without Early 
Intervention Systems 

About 60 percent of the managers surveyed work in departments 
that do not currently have operational EI systems. Their responses 
indicate their knowledge about EI systems and their perceptions 
of both potential contributions and problems. 

Managers Knowledgeable About EI Systems 

Generally, police managers in departments without EI systems are 
very knowledgeable about them. Only 13 percent of respondents 
said that they had no knowledge of EI systems, while another ten 
percent, based on specific comments they made, were 
misinformed about them. Thus, 77 percent of all managers had at 
least some knowledge about EI systems. 


Managers Overwhelmingly Anticipate Positive 
Impact of an EI System 

Managers overwhelmingly (94 percent) have a positive view of EI 
systems and anticipate that one would have a positive impact in 
their departments. Only three percent believe it would have a 
negative impact in their departments, another three percent have 
no opinion about the effect. 

Anticipated Impact of EI System 
Image of a bar graph showing the anticipated impact of the EI System

Managers Anticipate Objective Systematic Data 

One of the advantages managers anticipate includes objective and 
systematic data: 

 
EI systems allow the agency to use an objective scale by which 
to assess the need for intervention and action to prevent future 
problems. 
 
EI systems store data in a single location, thus offering the 
potential to identify issues and problems in a timely fashion. 
 
EI systems offer a systematic approach to identifying officers 
with performance problems. 
 
EI systems allow supervisors to rely less on reputation and be 
more objective in performance evaluations. 


 
EI systems instill more confidence that performance is being 
monitored and dealt with in a consistent fashion. 
 
EI systems are consistent with what we [as the agency] already 
do, but would provide a more formalized system within which 
to do it. 

Managers Anticipate Positive Impact on 
Supervisors 

Managers anticipate that an EI system will help supervisors do 
their jobs. And as the previous section indicated, these 
expectations are generally confirmed in practice. 

 
EI systems provide high-potential tools to help command staff 
members identify trends that they may miss. 
 
EI systems help supervisors, middle managers, and 
administrators identify internal problems dealing with officer 
attitudes, performance, and skills. 

Managers Anticipate a Reduction in Lawsuits 
Against the Department 

Managers anticipate that an EI system will help to reduce lawsuits 
against the department. Significantly, however, this issue was not 
mentioned by managers with EI systems in place. They did, 
however, mention a reduction in complaints, including preventing 
future liability, enhancing officer retention, fewer personnel 
issues, and fewer law suits. 

Managers Anticipate Multiple Benefits 

A number of managers reported that they anticipate multiple 
benefits to their department from having an EI system. 

 
EI systems help managers deliver employee assistance, risk 
management, improved morale, and improved employee and 
agency performance. 
 
EI systems provide liability prevention, good community 
relationships, and good labor/management relationships. 


Anticipate a Positive Impact on Community 
Relations 

Managers overwhelmingly (82 percent) anticipate improvements 
in community relations. This factor was rarely mentioned by 
managers in departments with EI systems, however. 

Overestimate the Opposition From Within the 
Department 

Thirty percent of managers anticipate negative responses in their 
department if they implemented an EI System and 40 percent 
anticipate a mixed response. Thirty-seven percent anticipate 
opposition from rank and file officers, for example, and 29 
percent would anticipate opposition from the police union. 
Another 30 percent anticipate a positive response. This represents 
a far more pessimistic view than has proven to be the case in 
departments with EI systems as reported in the previous section. 
Managers anticipate: 

 
Some initial reluctance by supervisors and some uncertainty by 
officers on the goals of the EI system. 
 
Skepticism at first, but acceptance that the system will be 
deployed. 
 
Mild grumbling by the rank and file, minor hand-wringing by 
mid-management, brief tooth gnashing by the chief, followed by 
the system being implemented, rarely triggered, and quickly 
assimilated as just another obscure general order in the manual. 
 
Skepticism until the benefits are proven. 

Understand Opposition Will Come From the 
Officers With Performance Problems 

Many mangers that anticipate opposition from the rank and file, 
however, recognize that most of it will come from the few 
problem officers. Managers anticipate: 

 Negative feedback from the bargaining unit, positive feedback 
from the commanders, but negative feedback from the small 
percentage of police officers that cause the most problems. 
Ninety percent of the officers would understand and accept the 
system. 


 Employees would accept it unless they thought they might be 
personally impacted. One respondent said, "Those are the ones 
that we need it for." 

Overestimate Union Opposition 

Many managers overestimate the extent of union opposition. 
Managers with experience, however, have not found significant 
opposition (See Section One). Often, opposition manifests in 
unexpected places. In practice, administrative problems tend to 
overshadow most other issues. 

Underestimate Administrative Problems 

In light of the problems reported by managers with EI system 
experience, managers without experience underestimate potential 
administrative problems. Only 29 percent mentioned 
administrative problems as the major obstacle they expected to 
face if implementing an EI system. Prospective EI implementation 
planners anticipate: 

 
That the main obstacle will be collecting and displaying the 
data. 
 
Inadequate staffing to run the system. 
 
Concerns about the duplication of work, specifically how it will 
co-exist with the current supervisory and complaint tracking 
system. 

Selling the Idea: Benefit to Subject Officers 

When asked how they would sell the idea of an EI system to their 
department, almost half (47 percent) mentioned that it would 
actually benefit officers who are subject to the system. 
Prospective EI system planners anticipated: 

 
"I would be able to point to specific examples that would 
demonstrate that if the agency had such a system in place, 
careers may have been saved." 
 
"It permits early intervention to assist officers with problems 
before job security becomes an issue." 


 "We are trying to save careers and quite possibly families. I 
don't think it will take a 'sales job.'" 

Selling the Idea: Makes the Supervisor's Job 
Easier 

Many managers anticipate that they can sell the idea by 
emphasizing its proactive problem-solving capacity. Some 
managers' responses to the survey included: 

 
"It would make their job easier. A good system could help in the 
early identification of problematic officers and procedures." 
 
"Allows us to spot problems before they become career-
threatening, eases workload by allowing the EWS to note where 
effort should be applied rather than waiting on a supervisor to 
recognize the trend." 
 
"To keep your subordinates out of harm's way makes the 
supervisor's job easier." 

Recognize Limitations of Current 
Accountability Mechanisms 

In describing the potential benefits of an EI system, many 
managers acknowledged the limitations of the current 
accountability mechanisms in their department. Only 33 percent 
rated their departments' current accountability measures as "very 
effective." About half (47 percent) rated their departments' 
accountability measures as "adequately effective." These 
limitations include a lack of consistency, the absence of data or 
objective standards, and excessive discretion in handling alleged 
misconduct. Some of the limitations managers reported included: 

 
"[The EI system is] not consistent, no standard system in place. 
Only a few supervisors take it seriously." 
 
"The Professional Standards Unit is responsible for 
communicating Internal Affairs' (IA) role to Administration. 
This works for identifying recurring problems. The limitation is 
that it relies upon one supervisor and some complaints may not 
get as high as Internal Affairs." 


 
"Minor complaints are most often handled by first line 
supervisors and are not documented in any retrievable form over 
time." 
 
"Lack of guidelines leaves a lot of discretion with individual 
supervisors and therefore inconsistency across the department." 
 
"It is informal and relies on the sergeant's ability to perceive 
problems and communicate effectively." 

Able to Track Officers Who Change Assignments 

Managers recognize that an EI system can help to track the 
performance of officers who change assignments and to provide 
new supervisors with useful data on their performance histories. 

 
Officers sometimes switch assignments frequently and get a 
fresh start with new supervisors. 
 
EI systems help track officers across different shifts. 
 
Traditionally, officers' reputations tend to be directed by rumor 
and perception only. 

Reactive Versus Proactive 

Traditionally, supervisors respond to officers' performance 
reactively rather than proactively. 

 
Supervisory action traditionally occurs after the fact and after 
discipline has been imposed several times. 
 
Supervisory action traditionally is ad hoc and sometimes the 
conduct rises to the level of formal action before we initiate 
corrective action. 

Prefer Information From Other Law Enforcement 
Agencies 

Managers expressed a preference for information about EI 
systems from other law enforcement agencies. 

 47 percent of respondents stated that if they needed additional 
information about EI Systems, the best source of that 
information would be expert personnel from another department 
with an EI system. 


 
30 percent of respondents stated that the best source of that 
information would be printed material. 
 
20 percent stated that if they needed additional information 
about EI systems, the best source of that information would be 
an International Association of Police Chiefs (IACP) training 
conference 

Conclusion 

Police managers who have experience with EI systems report very 
positive experiences with them. These managers have found EI 
systems to be an extremely useful tool for achieving 
accountability, with many benefits for supervisors and rank and 
file officers. This testimony is particularly important, as this 
survey represents the first systematic investigation of practical 
experiences with EI systems. They do not report strong opposition 
from rank and file officers or police unions, despite the fact that 
these problems are anticipated by managers in departments 
without EI systems. Managers do, however, report a number of 
problems related to the planning and implementation of their EI 
systems, particular with regard to communicating the nature and 
purpose of the system to all members of the department. 
Significantly, managers in departments without EI systems 
underestimate the management problems they will face. 
Chapter Four presents a guide for planning and implementing an 
EI system. The guide is based in part on the findings reported in 
this chapter. 


Chapter Five 

A Planning Guide for Implementing an Early
Intervention System
About This Chapter ............................................................95
Planning Process and Key Issues ...............................................96
STEP ONE: Recognize the Need ..................................................96
STEP TWO: Involve the Command Staff Early .....................................96
STEP THREE: Create a Planning Process .........................................97
STEP Four: Pinpoint Responsibility ............................................98
STEP FIVE: Involve Officers From All Ranks.....................................98
STEP SIX: Utilize Outside Experts .............................................99
STEP SEVEN: Assess the Departments Computer Infrastructure....................99
STEP EIGHT: Keep the Staff Informed ..........................................100
STEP NINE: Keep External Stakeholders Informed................................100
STEP TEN: Identify and Address Specific Issues ...............................101
STEP ELEVEN: Take the Time to Get It Right ...................................105
STEP TWELVE: Provide Staff Training ..........................................105
STEP THIRTEEN: Choose Software ...............................................106
STEP FOURTEEN: Finalize the Plan .............................................106
STEP FIFTEEN: Implement the EI System.........................................106

A Planning Guide for Implementing an 
Early Intervention System 

About This Chapter 

Police managers experienced with EI systems strongly believe that the 
development of an EI system should involve careful planning. Many of the 
early EI systems made mistakes that had to be corrected later. Often, these 
were mistakes that could have been avoided. Also, any agency developing an 
EI system today has the benefit of these early experiences and can plan 
accordingly. 

This chapter is designed as a guide to help law enforcement agencies plan and 
implement Early Intervention (EI) systems. Specifically, it: 

** Outlines the steps in a planning process 

** Identifies the key issues in planning 

This chapter closely follows the planning process described in 
Samuel Walker, Carol Archbold and Leigh Herbst, Mediating 
Citizen Complaints Against Police Officers: A Guide for Police 
and Community Leaders (2002), published by the U.S. 
Department of Justice, Office of Community Oriented Policing 
Services (COPS). That report is available on the web at 
www.cops.usdoj.gov. 


Planning Process and Key Issues 

STEP ONE: Recognize the Need 

The first step in developing an Early Intervention system is to 
recognize the need. EI systems are now a recognized best practice 
in law enforcement, so there should be little room for debate on 
this question. But law enforcement agencies tend to have specific 
problems. It might be large civil litigation claims in one 
department, but repeated complaints about traffic enforcement in 
another. Some agencies took the first steps toward an EI system 
with committees to review abuse of sick leave or excessive motor 
vehicle accidents. 

Agencies planning to develop an EI system should begin by 
identifying any specific problems they have that should be 
addressed by the system as well as existing programs or data 
systems that could be incorporated into an EI system. 

Key Issue: Begin with a self-assessment, identify existing 
problems, current accountability efforts, and the potential 
value of an EI system. 

STEP TWO: Involve the Command Staff Early 

The senior command staff should be involved in the planning of 
the EI system at the earliest moment. These officials will be 
responsible for developing, implementing, and administering the 
system, and therefore it is essential that they fully understand and 
become committed to it. 

It is essential that the chief executive make it clear that the EI 
system is a top priority for the agency. 

The initial meetings with the senior command staff represent an 
opportunity for presentations by outside experts who can answer 
questions about such issues as the goals, operating procedures, 
potential advantages, potential problems, and administrative 
requirements of an EI system. 


Key Issue:The chief executive needs to make it clear that 
the program is a top priority and that staff members will 
be held accountable for failing to follow through on 
developing the program. 

STEP THREE: Create a Planning Process 

A formal planning process should be created. This involves the 
appointment of a planning committee with a clear mandate 
regarding the general outlines of the desired EI system. A 
timetable for development and implementation of the EI system 
should be established at the outset. The planning committee 
should include representatives from all ranks of the department. 
(This important issue is discussed in more detail later.) 

Key Issue: Create a structured planning process. 

The planning process should involve a series of regular meetings 
(weekly, bi-weekly, and monthly) and these meetings should 
continue with interruptions only for special circumstances. 

The development and implementation of EI systems in some 
agencies took as long as two years. While in some cases the delay 
was the result of unavoidable issues related to computer hardware 
or software, in others it was because department leaders 
deliberately took the time to involve all stakeholders and resolve 
possible conflicts. Officials involved in those processes believe 
that taking the time was worth the effort. As one commander put 
it, "some important things would have been missed" if they had 
rushed the development too quickly. The most important things 
that would have been missed were (1) attention to certain issues 
that could have caused problems later on, and (2) the buy-in from 
officers that resulted from their having a chance to become 
familiar with the nature and purpose of the system. 

The importance of taking the time to involve stakeholders and 
resolve all issues in advance is documented in the COPS Office 
report, Mediating Citizen Complaints: A Guide for Police and 
Community Leaders (2002). 


STEP FOUR: Pinpoint Responsibility 

A commander with the rank of Deputy Chief or Captain should be 
appointed chair of the planning committee with responsibility for 
managing the process. 

Key Issue: Assign and hold one official responsible for the 
planning process and the final development of the 
program. 

STEP FIVE: Involve Officers From All Ranks 

Officers representing all ranks of the department should be 
appointed to the planning committee. Representation from all 
ranks serves two purposes: 

Firstly, the planning process benefits from the insights of staff 
members who have different experiences and perspectives on how 
an EI system should operate. Sergeants, who play a key role in a 
functioning EI system, will have insights into potential problems 
regarding how the system will be integrated into their normal 
duties. Direct involvement in the planning and the opportunity to 
raise important issues ensures buy-in from all ranks in the 
department. 

Secondly, the members of the committee become an important 
channel of communication to other staff members, allowing them 
to explain the concept and reduce the potential harm of rumors 
and misinformation. 

Key Issue: Involve officers from all ranks directly in the 
planning process. The planning process should represent 
the full range of rank, experience, race, ethnicity, and 
gender in the agency. 

Representatives of the collective bargaining unit for rank and file 
officers should also be included in the planning process. It is 
important to avoid creating a system that might conflict with 
certain clauses in the collective bargaining contract. 


If the EI system covers non-sworn employees in the agency, they 
should also be included in the planning process. And if they are 
represented by a collective bargaining unit, officials from that 
organization should be involved. 

Key Issue: Involve representatives from the collective 
bargaining unit(s). 

STEP SIX: Utilize Outside Experts 

There is now a substantial body of experience with EI systems in 
the law enforcement community. No agency needs to reinvent the 
wheel at this point. This report is designed to serve as a guide to 
the issues related to developing an EI system. 

Law enforcement officials in departments with operating EI 
systems currently in place are particularly useful as resources. 
They bring their practical experience with EI systems and as peers 
have substantial credibility with officers in the department. 

Key Issue: Don't try to reinvent the wheel. Utilize existing 
expertise from other law enforcement agencies or other 
experts. 

Issues such as the selection of the thresholds for identifying 
officers and choice of software are areas where an outside 
consultant can be particularly useful. 

STEP SEVEN: Assess the Department's Computer 
Infrastructure 

Based on the information learned from other departments, what is 
the current state of the department's computer infrastructure? Is it 
capable of supporting the more sophisticated systems that exist in 
other departments? For example, some systems allow each first-
line supervisor to log on to the system each day before roll call. 
Would that be possible in your department? 

If the department's computer system would not support the most 
sophisticated system, what alternatives are possible? Some of the 


best systems currently exist in departments with 2,000 and 8,000 
sworn officers. Does a medium-sized department need as complex 
an EI system? What kind of system would be appropriateand 
feasiblefor a department of 300 to 700 sworn officers? 

Key Issue: Begin addressing computer infrastructure 
needs at the very outset. Plan a system that is both 
appropriate in terms of accountability and feasible for the 
department. 

STEP EIGHT: Keep the Staff Informed 

Members of the agency should be kept informed of the 
development of the EI system as it develops. 

Involving officers from all ranks on the planning committee helps 
address this issue, but other steps can be taken. In one agency a 
series of separate meetings were held with the captains, 
lieutenants, and sergeants. Keeping all members informed through 
regular presentations helps to overcome misunderstandings that 
might impede the efficient implementation of the system. In the 
PERF survey (See Chapter Three), failures to communicate with 
members of the agency were the major problem reported by 
managers experienced with EI systems. 

Key Issue:To avoid problems arising from 
misunderstanding, provide staff at all ranks with detailed 
information about the program. 

STEP NINE: Keep External Stakeholders Informed 

An EI system is an important instrument for strengthening 
accountability within a department. Equally important is the task 
of building the perception of accountability among important 
leaders and groups outside the department. These groups include 
the responsible elected officials (mayors, city mangers, city 
council or county board members, community groups, etc.). 


Advising the public about the commitment to creating an EI 
system and keeping them informed about its progress can help 
develop positive relations with the community. 

Key Issue: Build community and political support by 
keeping external stakeholders informed about the 
development of the EI system. 

STEP TEN: Identify and Address Specific Issues 

A number of specific issues need to be addressed in the planning 
process: 

(1) Performance Indicators 

The planning process should include extensive discussions of the 
performance indicators to be used by the EI system. This process 
will facilitate understanding of the nature of an EI system as a 
comprehensive personnel assessment system and not just a 
process for identifying officers with performance problems. Also, 
the process will highlight deficiencies in the agency's existing 
data systems that need to be corrected if the EI system is to 
function properly. 

As Chapter One explained, there is no consensus of opinion about 
the exact number of indicators that should be used (except that 
relying on only one or two is inappropriate). Existing systems use 
as few as eight and as many as 24. 

Planning discussions about the number of indicators should take 
into account the specific needs of the department (See Step One, 
above) and the capacity of the department to support a large 
versus a small system. 

To assist in planning efforts, examples of performance 
indicators from several departments are presented in the 
Appendix to this report. 


(2)Thresholds for Identifying Officers 

The planning process should involve extensive discussions about 
what thresholds to set for identifying officers needing 
intervention. Although this is an extremely important issue, there 
are currently no nationally recognized best practices regarding 
thresholds. A variety of different systems are being used (See 
Chapter One). This is one area where the use of outside experts 
would be particularly valuable so that the planning committee can 
learn from the practical experience of other agencies. 

The discussion of thresholds is an opportunity for members of the 
planning committee to think through current performance 
standards within the agency and to reach a consensus about 
appropriate standards. 

As part of its development process, the Phoenix Police 
Department tested proposed thresholds to get a sense of how they 
would work in practice. This involved loading past performance 
data into the system and then applying the thresholds 
retrospectively. Tests of this sort can help to ensure that thresholds 
do not identify an unmanageable number of officers or fail to 
identify potential problems in certain areas of police activity. 

To assist in planning efforts, examples of thresholds used 
in other departments are presented in the Appendix to this 
chapter. 

Key Issue:Think through routine operations and identify 
key performance indicators. One size does not necessarily 
fit all. Indicators used by other agencies may not be 
relevant or appropriate. Some other indicators may be 
appropriate for your agency.

(3) Define the Format of Interventions 

The format of the interventions needs to be discussed. There is 
presently no national consensus regarding the best format. Some 
agencies rely on immediate supervisors to counsel subject 
officers. Others have meetings or counseling sessions with two or 


more supervisors present. Still others conduct classes for groups 
of officers selected for intervention. 

Discussions of alternative approaches will help officials better 
understand the purpose of the interventions and feel comfortable 
with the format ultimately selected. 

(4) Coordinate the EI System With the Disciplinary 
Process 

The EI system is designed to be an alternative to the normal 
disciplinary process. Once an EI system is operational the two 
systems will function along separate but often parallel tracks. It is 
important to clearly define the boundary between the two systems. 

For this reason, it is advisable for representatives of the internal 
affairs or professional standards unit to be included on the 
planning committee. 

Key Issue: Be sure to define clearly the relationship 
between the EI system and the existing disciplinary 
process. 

(5) Develop a List of Corrective Actions 

Officers conducting intervention sessions should have a number 
of specific recommendations for programs or steps that subject 
officers can or should take. These programs or steps might 
include retraining on specific police actions or a recommendation 
that the officer seek family counseling. 

The planning committee should develop this list of programs or 
steps. 

To assist in planning, examples of alternative corrective 
actions used by some departments are presented in the 
Appendix to this report 


Key Issue: Provide staff members who will be conducting 
the counseling sessions with practical and relevant 
alternatives that they can recommend to employees. 

(6) Decide What Kinds of Documentation Will Be 
Required 

For purposes of internal accountability, it is important to 
document the various steps in the EI system process. 

What kinds of reports about officers selected for intervention will 
be required? Will they be quarterly, semi-annual, annual? What 
will be included in these reports? How much detail will be 
included about each officer's performance? 

How will interventions be documented? Who will be responsible 
for reports about intervention sessions? How much detail will be 
included in these reports? 

What kind of post-intervention documentation will be required? 
For how long will a subject officer be monitored? Who will be 
responsible for that monitoring? What kind of evidence will be 
sufficient to remove an officer from the EI system? 

(7) Define the Day-to-Day Responsibilities of First-Line 
Supervisors 

What will be the day-to-day responsibilities of first-line 
supervisors? The Pittsburgh PARS requires sergeants to log on to 
the system each day prior to roll call. They are expected to review 
officer's performance records and identify any potentially 
troublesome patterns and take some appropriate action even 
before any officer exceeds a formal threshold. 

The purpose of this approach is to integrate the EI system into the 
normal supervisory process and to create a culture of 
accountability throughout the department. 


(8) Decide on Notification Procedures 

Some EI systems have the technical capacity to notify anyone 
from the first-line supervisor to the chief executive about any 
incident involving a particular officer. For example, if Officer X 
has been identified as having serious performance problems, the 
EI system can be programmed to send selected officials an email 
notification about selected performance indicators involving that 
officer (e.g., any use-of-force report, citizen complaint, etc.). 

Is this kind of process appropriate for the department? Does the 
department have an internal email system capable of supporting 
such a process? 

STEP ELEVEN: Take the Time to Get It Right 

Make sure all the issues are resolved before finalizing an EI 
system. Proceeding slowly but deliberately allows stakeholders an 
opportunity to gain an understanding of the EI system and to 
become comfortable with it. 

Some agencies developed their EI systems quickly then imposed 
on the department without warning. In these cases, the response 
has usually been suspicion, hostility, and resistance from officers 
at many ranks. 

Key Issue: EI Systems are very complex. It is better to take 
the time to get it right than to rush and create a system 
with unresolved issues. 

STEP TWELVE: Provide Staff Training 

All officers need to be trained in the operation of the system 
before it becomes operational. One of the problems identified in 
the PERF survey (See Chapter Three) was the failure of some 
departments to adequately explain the responsibilities of various 
officials in the EI system and to provide the proper training. 

Rank and file officers should be fully informed about the nature 
of the system and how it will operate in practice. 


Sergeants and any others who will be involved in interventions 
need to be trained in how to conduct intervention sessions. 

Lieutenants, captains, and other higher-ranking officers need to be 
trained regarding their responsibilities in the EI system. 

STEP THIRTEEN: Choose Software 

The planning committee should begin exploring software at the 
earliest moment. There are several commercial software packages 
on the market. A number of departments have developed their 
own software. The decision regarding software is an issue where 
the use of outside experts can be particularly valuable. 

STEP FOURTEEN: Finalize the Plan 

The last stage of the planning process should involve a final 
meeting of the planning committee to ensure that all issues have 
been addressed to the satisfaction of all members and to approve 
the final plan. The committee should review the final draft of the 
General Order and direct that it be sent to the chief executive for 
sign-off and official implementation of the EI system. 

STEP FIFTEEN: Implement the EI System 

Implement the Early Identification system. 


Chapter Six 

Policy and Research Needs for Early
Intervention 
Systems 
I. Current Issues Regarding EI Systems . . . . . . . . . . . . . . . . . . . . . .109 
II. 
The Future: An Action Agenda . . . . . . . . . . . . . . . . . . . . . . . . . . .111


Policy and Research Needs for Early 
Intervention Systems 

Early Intervention (EI) systems are presently still evolving. After a number 
of years of development, law enforcement agencies now have 
considerable experience with developing and operating EI systems. Even 
the best systems, however, continue to evolve as the managers responsible for 
them attempt to improve them. While there is now a rough consensus of 
opinion within the field regarding the basic elements of an effective EI 
system, many important questions remain unanswered. 

The purpose of this chapter is to discuss the most important questions 
regarding EI systems and to suggest the appropriate policy and research 
needs. 

I. Current Issues Regarding EI Systems 

(1) What are the appropriate performance indicators? 

While there is a strong consensus of opinion that an EI system 
should not rely on a single performance indicator (such as citizen 
complaints) there is no consensus regarding the exact number that 
should be used. There is, however, a strong consensus of opinion 
that certain indicators, such as use-of-force reports, citizen 
complaints, etc., should always be among those indicators chosen. 

The question of the appropriate number of indicators raises a more 
fundamental issue of the size and scope of an EI system. Large 
and complex EI systems have the capacity for comprehensive 
personnel assessment. They can identify top-performing officers 
as well as poor performers. Such systems, however, are not only 
expensive but pose a number of management challenges in terms 
of computer infrastructure, data entry, and ongoing management 


oversight. A system that is not managed properly will not be 
effective, no matter how sophisticated it is in theory. Smaller EI 
systems that focus on officers with performance problems and 
utilize fewer performance indicators cannot do as much in terms 
of personnel assessment, but are far easier to create and manage. 

Further research and evaluation is needed with regard to the mot 
effective size and scope of an EI system, and the related question 
regarding the proper number of performance indicators. 

(2) What are the appropriate thresholds for identifying 
officers for intervention? 

Several different systems of thresholds for identifying officer for 
possible intervention are currently in use. They include 
department-level thresholds, peer-officer thresholds, and ratios 
among different indicators (See Chapter Two). It is not clear at 
this point which formula is the best. 

Further research and evaluation is needed regarding the 
effectiveness of different systems of thresholds for identifying 
officers for possible intervention. 

(3) What is the best procedure for ensuring that 
supervisors responsible for conducting intervention 
sessions perform this task in a manner consistent with 
departmental goals? 

The intervention component of an EI system is the crucial part of 
the attempt to improve an officer's performance. An EI system 
involves a shift in the role of supervisors and the development of 
skills that many supervisors may not have or have been trained 
for. 

Additional research and evaluation is needed of different 
approaches for training supervisors for their new roles in an EI 
system. 


(4) What are the most effective kinds of intervention? 

It is likely that some forms of intervention are more effective than 
others for certain kinds of performance problems. Little attention 
has been given to this issue. 

Further research and evaluation should be given to developing 
more effective ways of tailoring specific interventions to 
particular officer performance problems. 

(5) What is the best size and scope of EI systems for 
small or medium-sized law enforcement agencies? 

Almost all of the information about EI systems available at 
present relates to large law enforcement agencies. 

Additional program development is needed with regard to EI 
systems appropriate for small and medium-sized law enforcement 
agencies. 

II. The Future: An Action Agenda 

Policy Development 

EI systems are developing all over America. Police managers 
associated with even the most sophisticated systems in the 
country continue to encounter new issues that need to be resolved. 
There is at present a substantial body of experience with EI 
systems and a cadre of police managers who have thought about 
the issues they face. The law enforcement profession should draw 
upon this experience and sponsor a series of activities designed to 
further the development of EI systems. 

Specifically, we recommend that: 

(1) Law enforcement professional associations sponsor a 
regular series of workshops and conferences specifically 
dedicated to the development and management of Early 
Intervention systems. 


Experience to date indicates that EI systems are extremely 
complex administrative tools and, consequently, it is not 
anticipated that the major issues will be resolved quickly. 
Consequently, we recommend that the law enforcement 
professions establish a regular series of workshops and 
conferences specifically dedicated to issues surrounding EI 
systems.143 

(2) Law Enforcement professional associations undertake the 
development of detailed recommended Standards for Early 
Intervention Systems. 

One purpose of the workshops and conferences should be to begin 
the development of recommended Standards for Early 
Intervention Systems. These would be very detailed and provide 
guidance to the profession on such issues as performance 
indicators, thresholds, and so on. It is expected that these 
standards would evolve over time, as a better understanding of EI 
systems develops. 

(3) In developing the Standards, law enforcement professional 
associations need to specifically address the unresolved issues 
discussed earlier in this chapter. 

Research 

Further research is need on the nature and impact of EI systems. 
Consequently, we recommend that the federal government and 
private funding agencies: 

(4) 
Fund research on the effectiveness of EI systems in 
reducing problematic performance among officers subject to 
intervention. 
(5) 
Fund research on the relative effectiveness of different EI 
systems with regard to such issues as their performance 
indicators and intervention strategies. 
(6) 
Fund research on the relationship of EI systems to other 
law enforcement accountability mechanisms. 


The Intersection of Research and Policy 

All too often, academic research is not effectively translated into 
operational policy. To overcome this problem, we recommend that 
the federal government and private funding agencies: 

(7) Fund workshops bringing together police officials 
responsible for policy development and academic researchers. 


Appendices 

Appendix A .................................................117
Appendix B .................................................120 
Appendix C .................................................124 
Appendix D .................................................126
Appendix E..................................................130
Appendix F..................................................137
Reading List ...............................................148
Endnotes ...................................................150

Appendices

Appendix A 

Consent Decree, United States v. Los Angeles (2001). 
The entire consent decree can be found on the 

U.S. Department of Justice website: www.usdoj.gov/crt/split. 

41. TEAMS II shall contain information on the following matters: 

a. all non-lethal uses of force that are required to be reported in 
LAPD "use of force" reports or otherwise are the subject of an 
administrative investigation by the Department; 
b. all instances in which a police canine bites a member of the 
public; 
c. all officer-involved shootings and firearms discharges, both 
on-duty and off-duty (excluding training or target range 
shootings, authorized ballistic testing, legal sport shooting 
events, or those incidents that occur off-duty in connection 
with the recreational use of firearms, in each case, where no 
person is hit by the discharge); 
d. all other lethal uses of force; 
e. all other injuries and deaths that are reviewed by the LAPD 
Use of Force Review Board (or otherwise are the subject of an 
administrative investigation); 
f. all vehicle pursuits and traffic collisions; 


g. all Complaint Form 1.28 investigations; 
h. with respect to the foregoing clauses (a) through (g), the 
results of adjudication of all investigations (whether criminal 
or administrative) and discipline imposed or non-disciplinary 
action taken; 
i. all written compliments received by the LAPD about officer 
performance; 
j. all commendations and awards; 
k. all criminal arrests and investigations known to LAPD of, and 
all charges against, LAPD employees; 
l. all civil or administrative claims filed with and all lawsuits 
served upon the City or its officers, or agents, in each case 
resulting from LAPD operations, and all lawsuits served on an 
officer of the LAPD resulting from LAPD operations and 
known by the City, the Department, or the City Attorney's 
Office; 

m. 
all civil lawsuits against LAPD officers which are required to 
be reported to the LAPD pursuant to paragraph 77; 
n. 
all arrest reports, crime reports, and citations made by officers, 
and all motor vehicle stops and pedestrian stops that are 
required to be documented in the manner specified in 
paragraphs 104 and 105; 
o. 
assignment and rank history, and information from 
performance evaluations for each officer; 
p. 
training history and any failure of an officer to meet weapons 
qualification requirements; and 
q. 
all management and supervisory actions taken pursuant to a 
review of TEAMS II information, include non-disciplinary 
actions. 


TEAMS II further shall include, for the incidents included in the 
database, appropriate additional information about involved 
officers (e.g., name and serial number), and appropriate 
information about the involved members of the public (including 
demographic information such as race, ethnicity, or national 
origin). Additional information on officers involved in incidents 
(e.g., work assignment, officer partner, field supervisor, and shift 
at the time of the incident) shall be determinable from TEAMS II. 


Appendix B 

Los Angeles Sheriffs Department, Personnel Performance 
Index (PPI), 

SHERIFF'S BULLETIN #443 June 3,1997 

Performance Review 

The purpose of this bulletin is to explain the performance review 
process, 

Background 

Over the past years, I have consistently expressed my belief that 
the Department's most important resource is our employees. 
Performance review and follow-up is one of the most significant 
ways that the Department demonstrates its commitment to 
supporting its employees. 

The tasks of reviewing, assessing, and guiding subordinate 
performance are not new ones. They have always been, and will 
continue to be, supervisory responsibilities. However, the 
structure and procedures that comprise the career performance 
review process that is the subject of this bulletin are new. having 
been implemented in the early part of 1996. At that time each 
division contribute a one supervisor to create a performance 
review staff. In addition, the eight Commanders who were serving 
on the Executive Risk Management Committee were designated 
to serve as members of the Performance Review Committee. 

Identification of Reviewees 

There is more than one method of determining which personnel 
are to be selected for review. One method starts with an 
examination of summary performance information in the Personal 
Performance Index (PPI). 

The PPI facilitates the identification of employees who have been 
involved in disproportionate numbers of (and/or very serious) risk 


incidents. Another method of identifying candidates to be 
reviewed involves supervisors' and managers' recommendations 
based on long-term observations and knowledge about their 
subordinates' work. 

Performance Review Report 

Upon designation of a candidate, the performance review staff 
collects virtually all the performance documentation available 
from the beginning of the employee's career. The material 
assembled includes photocopies of performance evaluations, 
commendations, notifications of discipline, and time records, as 
well as the detailed documentation on the risk incidents of interest 
shootings, uses of force, public complaints, administrative 
investigations, preventable traffic collisions, civil claims, and 
lawsuits. 

The staff member then compiles a profile report recounting the 
individuals career performance history as reflected in the 
documentation. The report and the accompanying documentation 
form the basis for the review process as conducted by the 
Performance Review Committee Commanders. 

Performance Review Committee 

The Commanders who comprise the Performance Review 
Committee meet in rotating groups of three members 
approximately twice per month as the profile repor1s are 
completed. The group is chaired by the Commander assigned to 
the Office of Administrative Services, who is the only 
Commander who does not rotate. The meetings are also attended 
by the Depar1ment Training Administrator, representatives from 
Employee Relations! Advocacy Services, Risk Management 
Bureau suppor1 staff, and the employee's Unit Commander. 

The committee members are furnished copies of the profile report 
prior to the scheduled date of the review. At the meeting, the 
committee evaluates the contents of the profile report and the 
compiled documentation and decides whether it is necessary to 
develop a plan for guiding the employee's future performance. 
Not every employee who is selected for a performance review is 


deemed to require a performance guidance plan. The committee 
carefully assesses each employee's situation and mayor may not 
conclude that a reviewed individual presents a performance risk to 
himself/herself or to the Department. 

Formation of Performance Plan 

In cases where it is determined that an employee performance 
guidance plan is appropriate, the committee Commanders, along 
with the Unit Commander, confer to develop a plan. Elements of 
plans may range from discussions with the employee. more 
training, or temporary work reassignments up to, in extreme 
cases, fitness for duty evaluations. 

One feature that is an element of each plan is the Unit 
Commander's in-person discussion with the employee. explaining 
the concerns about his/her performance as it impacts both the 
employee and the Department. Employees are entitled to review 
the profile report and the documentation upon which the 
committee's decision was based. 

Review of Management Personnel And Systems 

Reviewing an individual employee's performance is only one 
objective of this process. Another goal is to assess the 
effectiveness of the management/supervisory systems and 
personnel at the current and previous units of assignment. In some 
cases, it has been determined that a supervisor's or manager's 
responsibility to assist an employee in improving performance has 
not been fulfilled. In these cases, the review committee 
Commanders contact the concerned supervisors and/or managers 
to counsel them regarding these supervisorial issues and reinforce 
the message that genuine concern for subordinate welfare is 
demonstrated by taking the time and energy to be honest, 
supportive, and helpful. 


Results of Performance Review 

As of April 1997, 144 employees have been the recipients of 
performance reviewsfewer than two percent of the Department's 
sworn personnel. Only two people have been recommended for 
evaluations of fitness for duty. 

Conclusion 

The performance review process is not disciplinary in nature. It 
was developed and implemented to provide a fair and responsible 
system for improving or enhancing the employee's performance. 
Emphasis is placed on the concern for correcting the employee's 
work performance, as opposed to administering discipline. The 
process benefits a variety of individuals, including the reviewee as 
well as his or her current and previous supervisors and managers. 
It integrates concern for individual employee interest with good 
stewardship of the Department's collective welfare and the 
employee's future well being. 


Appendix C 

Miami-Dade Police Department, Employee Identification 
System (EIS) 

(1) systematic review of complaints and use of Force incidents by 
sworn/nonsworn personnel 

2) Identify stress or performance related problems 

c. Report. composition 

1) Quarterly/Annual Report criteria 
a) Two or more personnel complaints 

or: 

b) Involvement in three or more Use of Force incidents 

c) Annual Report  Employee identified in two or more 
quarters; four or more complaints, seven or more 
Supervisor's Use of Force Report to Control, or two or 
more shooting incidents in twelve months 

2) Report format 

a) Brief profile of complaints and Use of Force Incidents: 

(1) 
Complaints profile will include employee's name, 
badge number, complainant's name, nature of 
complaint, and disposition. 
(2) 
Use of Force profiles will include employee's 
name, badge number, subject's name, date of 
incident, nature of incident, extent of injury. 

d. Report Disposition 

1) Report disseminated from Director's Office to employee's 
chain of command 


2) Summary of supervisory review will be submitted to 
Internal Review Bureau within 30 days 

3) Action alternatives 

a) Referral to Psychological services Program for counseling 
or referral assistance 

b) Participation in stress Abatement Program for training 
assistance (40 hours) 

c) Corrective action by supervisory personnel; counseling, 
discipline, termination 

d) Assessment that no problem exists, terminating further 
action 


Appendix D 

Risk Management System, Cincinnati Police Department 

Memorandum of Understanding, U.S. Department of Justice 
and the City of Cincinnati (2002) 

VII. Management and Supervision 

A. Risk Management System 

57. 
The CPD will enhance and expand its risk management 
system to include a new computerized relational database for 
maintaining, integrating, and retrieving data necessary for 
supervision and management of the CPD. Priority will be given to 
the CPD obtaining any established program and system. The CPD 
will regularly use this data to promote civil rights and best police 
practices; to manage risk and liability; and to evaluate the 
performance of CPD officers across all ranks, units and shifts. 
58. 
The new risk management system will collect and record the 
following information: 

(a.) all uses of force; 

(b.)canine bite ratios; 

(c.) the number of canisters of chemical spray used by officers; 

(d.)all injuries to prisoners; 

(e.) all instances in which force is used and a subject is charged 
with "resisting arrest," "assault on a police officer," 
"disorderly conduct," or "obstruction of official business;" 

(f.) all critical firearm discharges, both on-duty and off-duty; 

(g.)all complaints (and their dispositions); 

(h.)all criminal proceedings initiated, as well as all civil or 
administrative claims filed with, and all civil lawsuits served 


upon, the City, or its officers, or agents, resulting from CPD 

operations or the actions of CPD personnel; 

(i.) all vehicle pursuits; 

(j.) all incidents involving the pointing of a firearm (if any such 
reporting requirement is imposed by paragraph 25); and 

(k.)all disciplinary action taken against officers. 

59. The new risk management system will include, for the 
incidents included in the database, appropriate identifying 
information for each involved officer (e.g., name, badge number, 
shift and supervisor) and civilian (e.g., race, ethnicity or national 
origin). 

61. 
The CPD will, within 90 days, prepare for the review and 
approval of DOJ, and thereafter implement, a protocol for using 
the risk management system. The City will submit for the review 
and approval of DOJ all proposed modifications to the protocol 
prior to implementing such modifications. 
62. 
The protocol for using the risk management system will 
include the following provisions and elements: 

(a.) The protocol is comprised of the following components: data 
storage, data retrieval, reporting, data analysis, pattern 
identification, supervisory assessment, supervisory 
intervention, documentation and audit. 

(b.)The protocol will require the automated system to analyze the 
data according to the following criteria: i) number of incidents 
for each data category by individual officer and by all officers 
in a unit; ii) average level of activity for each data category by 
individual officer and by all officers in a unit; and iii) 
identification of patterns of activity for each data category by 
individual officer and by all officers in a unit. 

(c.) The protocol will require the system to generate reports on a 


monthly basis describing the data and data analysis and 
identifying individual and unit patterns. 

(d.)The protocol will require that CPD commanders, managers, 
and supervisors will review, on a regular basis but not less 
than quarterly, system reports, and will evaluate individual 
officer, supervisor, and unit activity. 

(e.) The protocol will require that CPD commanders, managers, 
and supervisors initiate intervention for individual officers, 
supervisors and for units based on appropriate activity and 
pattern assessment of the information contained in the risk 
management system. 

(f.) The protocol will require that intervention options include 
discussion by commanders, managers, supervisors, and 
officers; counseling; training; and supervised, monitored, and 
documented action plans and strategies designed to modify 
activity. All interventions will be documented in writing and 
entered into the automated system (appropriate intervention 
options will be employed based on the evaluation described in 
subsection (e) above). 

(g.)The protocol will specify that actions taken as a result of 
information from the risk management system be based on all 
relevant and appropriate information, including the nature of 
the officer's assignment, crime trends and crime problems, and 
not solely on the number or percentages of incidents in any 
category of information recorded in the risk management 
system. 

(h.)The protocol will require that the system's data be accessible 
to CPD commanders, managers, and supervisors. 
Commanders, managers, and supervisors will promptly review 
records of all officers recently transferred to their sections and 
units. 

(i.) The protocol will require that CPD commanders, managers, 
and supervisors be evaluated on their ability to use the risk 
management system to enhance effectiveness and reduce risk. 


(j.) The protocol will require that the system be managed and 
administered by the Inspections Unit of the CPD. The 
Inspections Unit of the CPD will conduct quarterly audits of 
the system to ensure action is taken according to the process 
described above. 

(k.)The protocol will require regular reviews, at no less than 
quarterly intervals, by appropriate managers of all relevant 
risk management system information to evaluate officer 
performance city wide, and to evaluate and make appropriate 
comparisons regarding the performance of all CPD units in 
order to identify any significant patterns or series of incidents. 


Appendix E 

Consent Decree, United States v. State of New Jersey (1999). 

Supervisory Measures to Promote Civil Rights Integrity 

Management Awareness Program 

40. 
The State shall develop and implement computerized systems 
for maintaining and retrieving information necessary for the 
supervision and management of the State Police to promote 
professionalism and civil rights integrity, to identify and modify 
potentially problematic behavior, and to promote best practices 
(hereinafter, the "Management Awareness Program" or "MAP"). 
41. 
The MAP shall consist of the following information: 

a. 
all items of information in connection with all motor vehicle 
stops that are required to be recorded in a written report, form, 
or log, or reported to the communications center, pursuant to 
29 and the protocols listed in 29of this Decree, except that 
duplicate information need not be entered, and information as 
to whether the incident was recorded with MVR equipment 
need not be entered if all patrol cars are equipped with MVR 
unless a patrol car was equipped with MVR equipment that 
was not functioning; 
b. 
information on civilian compliments and other indicia of 
positive performance; information on misconduct 
investigations reports on use of force associated with motor 
vehicle stops; on-duty and off-duty criminal arrests and 
criminal charges; civil suits involving alleged misconduct by 
state troopers while on duty; civil suits in which a trooper is 
named as a party involving off-duty conduct that alleges racial 
bias, physical violence or threats of violence; and 
c. 
implementation of interventions; and training information 
including the name of the course, date started, date completed 
and training location for each member receiving training. 


42. 
All information in MAP on substantiated misconduct 
investigations, civilian compliments, and other indicia of positive 
performance which can be attributed to a specific trooper shall be 
made available to that trooper on an annual basis upon written 
request. Nothing in this paragraph shall be construed as granting 
that trooper access to confidential documents other than those 
identified in this paragraph, or to any information which cannot 
be attributed to the trooper requesting the information. 
43. 
Regarding the motor vehicle stop information identified in 29 

(a) (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, and 
19) and recorded in accordance with the protocols identified in 
29(a), the MAP shall have the capability to search and retrieve 
numerical counts and percentages for any combination of the 
above-referenced information and to run reports for different time 
periods (e.g., monthly, quarterly, annually) and for individual 
troopers, squads, and stations. Regarding the motor vehicle stop 
information identified in 29(a)(5A, 8A, 12A, 13A, 14A, 15A, and 
17A) and recorded in accordance with the protocols identified in 
29(a), it will be sufficient that the MAP shall have the capability 
to access (through cross-referenced paper documents or other 
method) this descriptive information entered on specific incidents 
and matters. Regarding the information identified in 41(b and c), 
to the extent technologically feasible, the MAP shall be developed 
to have the capability to search and retrieve numerical counts and 
percentages for any combination of the information and to run 
reports for different time periods and for individual troopers, 
squads or stations. To the extent that the MAP shall require 
textual or narrative descriptions of misconduct allegations or other 
information identified in 41(b and c), it will be sufficient that the 
MAP only have the capability to retrieve this descriptive 
information. 

44. Where information about a single incident is included within 
the MAP from more than one document the State shall use a 
common control number or other means to link the information 
from different sources so that the user can cross-reference the 
information and perform analyses. 


45. 
The State shall ensure that information is included within the 
MAP in an accurate and timely fashion and is maintained in a 
secure manner. 
46. 
Within one hundred and eighty (180) days following entry of 
this Decree, the State shall develop a plan for designing and 
implementing the MAP including the use of the MAP, a timetable 
for implementation, and a specification of the information 
contained in State records pre-dating the implementation of the 
MAP that can reasonably be incorporated in the MAP. Prior to 
effectuating the implementation plan, the plan shall be approved 
by the United States and the Independent Monitor. Within 180 
days following the entry of this Decree, the State shall begin 
conducting the supervisory and management reviews required by 
48-53. 

Supervisory and Management Reviews 

47. 
Consistent with the requirements of 48-53 infra, the State 
shall develop a protocol specifying the manner in which 
supervisory and management reviews of individual state troopers, 
and State Police units and sub-units (e.g., troops, stations, and 
squads), shall be conducted, and the frequency of such reviews. 
Prior to implementation, the protocol shall be approved by the 
United States and the Independent Monitor. 
48. 
At least quarterly, State Police supervisors shall conduct 
reviews and analyses of data obtained from the MAP and other 
appropriate sources to ensure that individual troopers and State 
Police units and subunits are performing their duties in accord 
with the provisions of this Decree and associated protocols. 
49. 
To the extent reflected in 43, reports of MAP data shall 
regularly be prepared regarding individual troopers, stations and 
squads, for use in reviews as appropriate. The reports shall 
include the following information: 

a. the number of motor vehicle stops, by race/ethnicity, reason 
for the stop (i.e., moving violation, non moving violation, 
other), road, squad, and trooper station; and the number of 


enforcement actions and procedures taken in connection with 
or during the course of a motor vehicle stop, by race/ethnicity, 
reason for the stop (i.e., moving violation, non- moving 
violation, other), road, squad and trooper station; 

b. data (including racial/ethnic data) on complaints, misconduct 
investigations (for each type of investigation, as delineated in 
73), discipline, intervention, and uses of force associated with 
motor vehicle stops. 

50. 
To the extent reflected in 43,analyses of MAP data concerning 
motor vehicle stops shall include a comparison of racial/ethnic 
percentages of motor vehicle stops (by reason for the stop (i.e., 
moving violation, non moving violation, other)) and racial/ethnic 
percentages of enforcement actions and procedures taken in 
connection with or during the course of such stops, with a 
benchmark racial/ethnic percentage if available (see 54-55); a 
comparison of racial/ethnic percentages for such stops with the 
racial/ethnic percentages for enforcement actions taken in 
connection with or the during the course of such stops; a 
comparison of racial/ethnic percentages for consent searches of 
vehicles, and requests for consent to search vehicles, with "find" 
rates by race/ethnicity for motor vehicle consent searches; a 
comparison of racial/ethnic percentages for non-consensual 
searches of motor vehicles with "find" rates by race/ethnicity for 
motor vehicle non-consensual searches; evaluations of trends and 
differences over time; and evaluations of trends and differences 
between troopers, units, and subunits. 
51. 
To the extent reflected in 43,analyses of other data generated 
by the MAP shall include evaluations of trends and differences 
over time and evaluations of trends and differences between 
troopers, units, and subunits. 
52. 
Each supervisor shall, consistent with his or her authority, 
implement any appropriate changes or remedial measures 
regarding traffic enforcement criteria, training, and enforcement 
practices for particular units or subunits or implement any 
appropriate intervention for particular troopers; conduct any 


necessary additional assessment or investigation regarding 
particular units or subunits or particular troopers; and/or make any 
appropriate recommendations. 

53. 
A supervisory review shall be conducted regarding any state 
trooper who within a period of two years, is the subject of three 
misconduct investigations of any kind initiated pursuant to 73. 
Where appropriate, the review may result in intervention being 
taken. In the event the supervisory review results in intervention, 
the supervisor shall document the nature, frequency, and duration 
of the intervention. 
54. 
To assist in evaluating data reported from the MAP 
concerning State Police law enforcement on the New Jersey 
Turnpike, the State shall develop (for purposes of implementing 
this Decree) a protocol for conducting a survey of a sample of 
persons and vehicles traveling on the New Jersey Turnpike to 
determine the racial/ethnic percentage of drivers on the Turnpike. 
As appropriate, the survey may identify different benchmark 
figures for different portions of the Turnpike. Prior to 
implementation, the protocol shall be approved by the 
Independent Monitor and the United States. The protocol shall be 
developed and implemented using a consultant jointly selected by 
the parties. The survey shall be completed within one hundred 
fifty (150) days of the entry of this Decree. Both the United States 
and the State agree that the utility and fairness of the MAP 
described in this Consent Decree will depend to some degree on 
the development of accurate and reliable benchmarks that account 
for all appropriate variables and factors. 
55. 
For purposes of implementing this Decree, the State may in its 
discretion conduct a survey of a sample of persons and vehicles 
traveling on any other limited access highway in New Jersey to 
determine the racial/ethnic percentage of drivers on that road. In 
deciding whether to conduct such a survey, the State shall give 
due consideration to any recommendations made by the United 
States and the Independent Monitor. The survey shall be 
conducted pursuant to a protocol developed by the State and 
approved by the Independent Monitor and the United States. The 
protocol shall be developed and implemented using a consultant 
jointly selected by the parties. 


56. The parties agree that for purposes of developing appropriate 
benchmark racial/ethnic percentages, it will be sufficient for the 
surveys described by 54 and 55 to determine the racial/ethnic 
percentages of persons and vehicles traveling on the subject 
roadway. The provisions of 54 and 55 shall not apply to any 
traffic survey conducted by the State for purposes of advocating a 
position in any criminal case or any civil litigation other than in 
the instant case. 


Appendices 

Appendix F 
EI Systems: An Overview
 Data-based management information system. 
 Capacity for identifying and correcting performance problems. 
 Recommended police accountability "best practice" 
 Separate from the formal disciplinary system 
 Consistent with the goals of Ccommunity Ppolicing 
 Consistent with the process of Pproblem-Ooriented Ppolicing 
 Rely on systematic and timely data 
 Consistent with the process of COMPSTAT 
 Careful planning is needed for development and implementation 
 
Office of Community Oriented Policing Services 
U.S. Department of Justice 
Early Intervention Systems for Law 
Enforcement Agencies: A Planning 
and Management Guide 
Prepared by 
Samuel Walker, PhD 
Department of Criminal Justice 
University of Nebraska at Omaha 
COPS 
COMMUNITY OORIENTED PPOLICING SSERVICES 
U.S. DDEPARTMENT OOF JJUSTICE 


Goals and Impacts of EI Systems
The Individual Officer 
 Improved Performance 
 Higher Standards of Accountability 
Supervisors 
 Improved Supervisory Practices 
 Higher Standards of Accountability as Supervisors 
The Department 
 Higher standards of accountability 
 Reduction in litigation 
 Improved Community Relations 
 
Office of Community Oriented Policing Services 
U.S. Department of Justice 
Early Intervention Systems for Law 
Enforcement Agencies: A Planning 
and Management Guide 
Prepared by 
Samuel Walker, PhD 
Department of Criminal Justice 
University of Nebraska at Omaha 
COPS 
COMMUNITY OORIENTED PPOLICING SSERVICES 
U.S. DDEPARTMENT OOF JJUSTICE 


Components of an EI System
 Performance Indicators 
 Identification and Selection 
 Intervention 
 Post-Intervention Monitoring 
 
Office of Community Oriented Policing Services 
U.S. Department of Justice 
Early Intervention Systems for Law 
Enforcement Agencies: A Planning 
and Management Guide 
Prepared by 
Samuel Walker, PhD 
Department of Criminal Justice 
University of Nebraska at Omaha 
COPS 
COMMUNITY OORIENTED PPOLICING SSERVICES 
U.S. DDEPARTMENT OOF JJUSTICE 


Performance Indicators
Comprehensive Personnel Assessment Model 
1. All non-lethal uses of force 
Citizen injuries requiring medical attention 
Disciplinary outcomes 
2. All officer-involved shooting incidents 
On-duty/off-duty discharges, citizen injuries, citizen fatalities 
Disciplinary outcomes 
3. All officer-involved vehicular pursuits 
Accidents 
Injuries 
Fatalities 
Disciplinary outcomes 
4. All citizen-initiated complaints 
Use of force 
Racial bias 
Investigative outcomes 
5. All citizen-initiated commendations or compliments 
6. All departmental commendations and awards 
7. Criminal arrests and investigations of subject officer 
8. Civil suits or an administrative claim in which subject 
officer is named 
Nature of claims 
Case outcomes 
Office of Community Oriented Policing Services 
U.S. Department of Justice 
Early Intervention Systems for Law 
Enforcement Agencies: A Planning 
and Management Guide 
Prepared by 
Samuel Walker, PhD 
Department of Criminal Justice 
University of Nebraska at Omaha 
COPS 
COMMUNITY OORIENTED PPOLICING SSERVICES 
U.S. DDEPARTMENT OOF JJUSTICE 


Performance Indicators (continued)
9. All arrest reports, crime reports, and citations made by 
officers 
10. All motor vehicle stops and pedestrian stops made by 
subject officer 
Race, ethnicity, gender, age of drivers stopped 
11. Performance evaluations for each officer 
12. Training history 
Any failure of an officer to meet weapons qualification 
requirements 
13. All management and supervisory actions taken 
pursuant to a review of TEAMS II information, including 
non-disciplinary actions 
14. Sick leave or family leave record 
15. Canine unit deployment involving subject officer 
Deployments 
Bites 
Citizen injuries 
Complaints or civil suits arising from deployments 
16. Failure to appear in court 
 
Office of Community Oriented Policing Services 
U.S. Department of Justice 
Early Intervention Systems for Law 
Enforcement Agencies: A Planning 
and Management Guide 
Prepared by 
Samuel Walker, PhD 
Department of Criminal Justice 
University of Nebraska at Omaha 
COPS 
COMMUNITY OORIENTED PPOLICING SSERVICES 
U.S. DDEPARTMENT OOF JJUSTICE 


General Assessment of EI System
Image of a bar graph showing the general assessment of EI System

Impact of EI System on Quality 
of Police Service 
Image of a bar graph showing the impact of the EI System on Quality


Reading List 

Christopher Commission, Report of the Independent Commission 
on the Los Angeles Police Department. Los Angeles: Christopher 
Commission, 1991. 

Commission on Accreditation for Law Enforcement Agencies. 
Standards for Law Enforcement Agencies. 4th ed. Fairfax, VA: 
CALEA, 1999. www.calea.org 

Robert C. Davis, et. al. Turning Necessity into Virtue: Pittsburgh's 
Experience with a Federal Consent Decree. New York: Vera 
Institute of Justice, 2002. Available at: www.vera.org. 

International Association of Chiefs of Police, Building Integrity 
and Reducing Drug Corruption in Police Departments. 
Gaithersburg: IACP, 1989. 

Kolts, James G., and Staff. The Los Angeles County Sheriff's 
Department. Los Angeles: Los Angels County, 1992. 

Los Angeles Police Commission. Report of the Rampart 
Independent Review Panel. Los Angeles: Los Angeles Police 
Commission, 2000. 

Special Counsel Merrick J. Bobb, 16th Semiannual Report. Los 
Angeles: Special Counsel, 2003. Available at: www.parc.info. 

Walker, Samuel. "The New Paradigm of Police Accountability," 
St. Louis University Public Law Review, XII (March 2003): 
352. 

Walker, Samuel, Geoffrey P. Alpert and Dennis Kenney. Early 
Warning Systems: Responding to the Problem Police Officer. 
Research in Brief. Washington, DC: National Institute of Justice, 
2001. Available on the web at www.ncjrs.gov. NCJ 188565. 

Weisburd, David, Stephen Mastrofski, Ann Marie McNally, and 
Rosann Greenspann, Compstat and Organizational Change: 
Findings from a National Survey. Washington, DC: The Police 
Foundation, 2001. 


U.S. 
Commission on Civil Rights, Who is Guarding the 
Guardians? A Report on Police Practices Washington, DC: U.S. 
Civil Rights Commission, 1981. 
U.S. 
Department of Justice, Principles for Promoting Police 
Integrity. Washington: Department of Justice, 2001. Available on 
the web at www.ncjrs.gov, NCJ # 186189. 


Endnotes 

1 Samuel Walker, Geoffrey P. Alpert, and Dennis Kenney. 
2001. Early Warning Systems: Responding to the Problem 
Police Officer. Research in Brief. (Washington, DC: 
National Institute of Justice, 2001). Available on the web 
at www.ncjrs.gov. NCJ 188565. 

2 International Association of Chiefs of Police, Building 
Integrity and Reducing Drug Corruption in Police 
Departments (Washington, DC: Government Printing 
Office, 1989), p. 80. 

3 Frank Landy, Performance Appraisal in Police 
Departments. (Washington, DC: The Police Foundation, 
1977). 

4 Bob Stewart, Comments, Focus Group Session, Nebraska 
State Patrol, June 9, 2003. 

5 Gary Stix, "Bad Apple Picker: Can A Neural Network 
Help Find Problem Cops?" Scientific American 
(December 1994): 4445. Bernard Cohen and Jan M. 
Chaiken, Police Background Characteristics and 
Performance (Lexington, Mass., Lexington Books, 1973). 

6 Herman Goldstein, Police Corruption (Washington, DC: 
The Police Foundation, 1975). 

7 It is significant, for example, that a recent National 
Institute of Justice (NIJ) publication on developing 
programs to deal with law enforcement officer stress 
includes a section on "Selecting Target Groups" but 
contains no reference to specific performance indicators 
such as are commonly used in EI systems. Peter Finn and 
Julie Esselman Tomz, Developing a Law Enforcement 
Stress Program for Officers and Their Families 
(Washington: Government Printing Office, 1997), pp. 
2326. Available on the web at www.ncjrs.gov. 
NCJ163175. 


8 David Weisburd, Stephen Mastrofski, Ann Marie McNally, 
and Rosann Greenspann, Compstat and Organizational 
Change: Findings from a National Survey (Washington, 
DC: The Police Foundation, 2001). Eli B. Silverman, 
NYPD Battles Crime: Innovative Strategies in Policing 
(Boston: Northeastern University Press, 1999). 

9 Robert L. Mathis and John H. Jackson, Human Resource 
Management. Michael Poole and Malcolm Warner, The 
IEBM Handbook of Human Resource Management 
(London: International Thomson Business Press, 1998). 

10 One large city on the West Coast the union demanded that 
the EI system be subject to collective bargaining or meet 
and confer. The issue remained unresolved at the time this 
report was written. Nonetheless, this remains the only 
known case of union opposition to date. 

11 Walker, Alpert, and Kenney, Early Warning Systems: 
Responding to the Problem Police Officer. 

12 U.S. Department of Justice, Principles for Promoting 
Police Integrity (Washington: Department of Justice, 
2001). Available on the web at www.ncjrs.gov, NCJ # 
186189. 

13 Commission on Accreditation for Law Enforcement 
Agencies, Standard 35.1.15. "Personnel Early Warning 
System." (2001). Commission on Accreditation for Law 
Enforcement Agencies, Standards for Law Enforcement 
Agencies, 4th ed. (Fairfax, VA: CALEA, 1999). 
www.calea.org 

14 International Association of Chiefs of Police, Building 
Integrity and Reducing Drug Corruption in Police 
Departments. 

15 U. S. Commission on Civil Rights, Who is Guarding the 
Guardians? (Washington DC: The United States 
Commission on Civil Rights, 1981). 


16 The various consent decrees and memoranda of 
understanding, along with other related documents, are 
available at www.usdoj.gov/crt/split. 

17 United States v. State of New Jersey, Consent Decree 
(1999), Parag. 40. Available at www.usdoj.gov/crt/split. 

18 United States v. Los Angeles, Consent Decree (2000), 
Section II. Available at www.usdoj.gov/crt/split. 

19 Department of Justice, Principles for Promoting Police 
Integrity. 

20 Robert C. Davis, Christopher Ortiz, Nicole J. Henderson, 
Joel Miller and Michelle K. Massie, Turning Necessity 
into Virtue: Pittsburgh's Experience with a Federal 
Consent Decree (New York: Vera Institute of Justice, 
2002), p. 37. Available at www.vera.org. 

21 Samuel Walker, "The New Paradigm of Police 
Accountability: The U.S. Justice Department 'Pattern or 
Practice' Suits in Context," St. Louis University Public 
Law Review, XXII (No. 1, 2003): 352. 

22 Walker, Alpert, and Kenney, Early Warning Systems: 
Responding to the Problem Police Officer. 

23 Early Intervention System State of the Art Conference, 
Report (Omaha, 2003). Available at 
www.policeaccountability.org. 

24 Davis, et al., Turning Necessity into Virtue, pp. 44 (quote), 
43. 

25 United States v. City of Los Angeles, Consent Decree, Sec. 
II, Para. 47. 

26 The Los Angeles and other consent decrees are available 
on the U.S. Department of Justice website: 
www.usdoj.gov/crt/split. 

27 United States v. Cincinnati, 2002: Sec. VII. 62. i. 


28 San Jose Independent Police Auditor, 2001 Year End 
Report (San Jose: Independent Police Auditor, 2002), pp. 
4849. 

29 This program is described in Special Counsel Merrick J. 
Bobb, 16th Semiannual Report (Los Angeles: Special 
Counsel, 2003), pp. 7980. 

30 United States v. State of New Jersey. 

31 Unpublished Remarks. Early Intervention State of the Art 
Conference (2003). 

32 American Correctional Associatioin, Performance-Based 
Standards for Adult Community Residential Services, 4th 
ed. (Lanham, MD: American Correctional Association, 
2000), Appendix B, p. 82. 

33 Timothy N. Oettmeier and Mary Ann Wycoff, Personnel 
Performance Evaluations in the Community Policing 
Context (Washington, DC: The Police Foundation, 1997). 

34 Geoffrey Alpert and Mark H. Moore, "Measuring Police 
Performance in the New Paradigm of Policing," Bureau of 
Justice Statistics, Performance Measures for the Criminal 
Justice System (Washington: Government Printing Office, 
1993), pp. 109140. 

35 Ibid. 

36 Herman Goldstein, "Improving Policing: A Problem-
Oriented Approach," Crime and Delinquency, 25 (1979): 
236258. Tara O'Connor Shelley and Anne C. Grant, eds., 
Problem-Oriented Policing (Washington: Police Executive 
Research Forum, 1998). 

37 Davis, et al., Turning Necessity into Virtue, pp. 4547. 

38 Carol Archbold, Innovations in Police Accountability: An 
Exploratory Study of Risk Management and Police Legal 
Advising. (March 2002). Available at 
www.policeaccountability.org. 


39 National Advisory Commission on Civil Disorders, Report 
(New York: Bantam Books, 1968). NAACP, Beyond the 
Rodney King Story (Boston: Northeastern University 
Press, 1995). Human Rights Watch, Shielded From 
Justice: Police Brutality and Accountability in the United 
States (New York: Human Rights Watch, 1998). 

40 William A. Westley, Violence and the Police (Cambridge: 
MIT Press, 1970). 

41 The argument is implicit in the long-standing 
recommendations that police departments diversify their 
work forces and actively recruit more racial and ethnic 
minority officers. See, for example, National Advisory 
Commission on Civil Disorders, Report, p. 315, 

42 Catherine H. Milton, Women in Policing (Washington: 
The Police Foundation, 1970). 

43 This view is central to the movement for police 
professionalism and the reform proposals that dominated 
the professionalization from about 1900 to the present. 
Samuel Walker, A Critical History of Police Reform 
(Lexington: Lexington Books, 1977). 

44 On using peer officers to evaluate officers' performance, 
see Special Counsel, 16th Semiannual Report, p. 74. 

45 Samuel Walker, Internal Benchmarking for Traffic Stop 
Data: An Early Intervention Approach. Discussion Paper. 
Omaha, NE, 2003. Available at 
www.policeaccountability.org. 

46 Early Intervention State of the Art Conference, Report. 

47 Dennis J. Kenney, Early Warning Systems: Responding to 
the Problem Police Officer (Washington: Government 
Printing Office, 2001), described a three-component 
approach. The four-component approach used here reflects 
the subsequent research. Available on the web at 
www.ncjrs.gov. NCJ 188565. 
154 


48 Early Intervention System State of the Art Conference, 
Report (Omaha, 2003). Available at 
www.policeaccountability.org. 

49 A good description of these two systems is available in 
Special Counsel Merrick J. Bobb, 16th Semiannual Report 
(Los Angeles: Special Counsel, 2003), pp. 6872. The 
report is available at www.parc.info. 

50 Robert Davis, Christopher Ortiz, Nicole J. Henderson, Joel 
Miller and Michelle K. Massie, Turning Necessity into 
Virtue: Pittsburgh's Experience with a Federal Consent 
Decree (New York: Vera Institute of Justice, 2002), pp. 
4445. Available at www.vera.org. 

51 Samuel Walker, Police Accountability: The Role of Citizen 
Oversight (Belmont: Wadsworth, 2001), pp. 121135. 

52 United States v. Los Angeles, Consent Decree (2001), 
Section II. Available at www.usdoj.gov/crt/split. 

53 An important object lesson on this point is in Special 
Counsel Merrick J. Bobb, 16th Semiannual Report, which 
found that because of staff cutbacks data were not being 
entered into the systems in a timely fashion. The report is 
available at www.parc.info. 

54 U.S. Department of Justice and the City of Cincinnati, 
Memorandum of Understanding, 2001. Available at 
www.usdoj.gov/crt/split. 

55 Special Counsel Merrick J. Bobb, 16th Semiannual Report, 
pp. 7172. 

56 Early Intervention State of the Art Conference, Report. 

57 Similar examples are offered in Samuel Walker, Internal 
Benchmarking for Traffic Stop Data: An Early 
Intervention System Approach. Discussion Paper (Omaha, 
2003). Available at www.policeaccountability.org. 


58 This approach is described in Bobb, 16th Semiannual 
Report, pp. 7475. 

59 This approach is described in ibid, pp. 6869. 

60 Walker, Alpert and Kenney, Early Warning Systems: 
Responding to the Problem Police Officer. 

61 Special Counsel Merrick J. Bobb, 15th Semiannual Report 
(Los Angeles: Special Counsel, 2002), p. 39. Available at 
www.parc.info. 

62 Ibid, p. 42. 

63 Davis, et. al., Turning Necessity into Virtue, p. 46. 

64 Commission on Accreditation for Law Enforcement 
Agencies, Standard 35.1.15. Personnel Early Warning 
System. 

65 Special Counsel Merrick J. Bobb, 15th Semiannual Report, 
pp. 3775. 

66 New Orleans Police Department, Professional 
Performance Enhancement Program (PPEP) (New 
Orleans: NOPD, 1998). 

67 Omaha Police Union, "Bad Boy/Girl Class Notes Shared." 
The Shield. (April 1992). 

68 Early Intervention State of the Art Conference, Report. 

69 San Jose Independent Police Auditor, 2001 Year End 
Report (San Jose: Independent Police Auditor, 2002), pp. 
4849. Available at www.ci.san-jose.us. The San Jose 
program is also described in Special Counsel Merrick J. 
Bobb, 16th Semiannual Report, p. 79. 

70 Davis, et. al., Turning Necessity into Virtue. 

71 New Orleans Police Department, Professional 
Performance Enhancement Program (PPEP). 


72 Early Intervention State of the Art Conference, Report. 

73 Comments, police chiefs, Justice Department Conference, 
Strengthening Police Community Relationships, June 910, 1999. 

74 Herman Goldstein, Policing a Free Society (Cambridge: 
Ballinger, 1977), p. 171. 

75 Hans Toch, J. Douglas Grant, and Raymond T. Galvin, 
Agents of Change: A Study in Police Reform (New York: 
John Wiley, 1975). 

76 U.S. Commission on Civil Rights, Who is Guarding the 
Guardians? A Report on Police Practices (Washington, 
DC: U.S. Civil Rights Commission, 1981), pp. 159, 166. 

77 Christopher Commission, Report of the Independent 
Commission on the Los Angeles Police Department (Los 
Angeles, 1991). Available at www.parc.info. 

78 Ibid., pp. 4048. 

79 James G. Kolts and Staff, The Los Angeles County 
Sheriff's Department (Los Angeles: Los Angels County, 
1992), pp. 157167, 353. Available at www.parc.info. 

80 Special Counsel Merrick J. Bobb, 11th Semiannual Report 
(Los Angeles: Special Counsel, 1999), pp. 4647. 
Available at www.parc.info. 

81 "Kansas City Police Go After Their 'Bad Boys'," The New 
York Times (September 10, 1991). 

82 "Wave of Abuse Laid to a Few Officers," Boston Globe 
(October 4, 1992). 

83 Sari Horowitz, "Officers Go Too Far," The Washington 
Post (November 19, 1998), p. 1. 


84Samuel Walker, Geoffrey P. Alpert, and Dennis J. Kenney, 
Early Warning Systems: Responding to the Problem Police 
Officer (Washington, DC: National Institute of Justice, 
2001). NCJ 188565. Available at www.ncjrs.gov. 

85Catherine H. Milton, et. al., Police Use of Deadly Force 
(Washington, DC: The Police Foundation, 1977). Walker, 
Alpert, and Kenney, Early Warning Systems: Responding 
to the Problem Police Officer. 

86 U.S. Commission on Civil Rights, Who is Guarding the 
Guardians?, pp. 8186.

87 New York City, Commission to Investigate Allegations of 
Police Corruption and the Anti-Corruption Procedures of 
the Police Department [Mollen Commission], Commission 
Report (New York, 1994).

88 Tony Pate, Jack W. McCullough, Robert A. Bowers, Amy 
Ferrara, Kansas City Peer Review Panel: An Evaluation 
(Washington, DC: The Police Foundation, 1976).

89 George L. Kelling, et al., The Kansas City Preventive 
Patrol Experiment (Washington: The Police Foundation, 
1974).

90 Pate, et al., Kansas City Peer Review Panel, pp. 910.

91 Ibid., p. 55.

92 "Kansas City Police Go After Their 'Bad Boys'," New 
York Times (September 10, 1991).

93 Toch, Grant and Galvin, Agents of Change.

94 U. S. Commission on Civil Rights, Who is Guarding the 
Guardians?.

95 John S. Ross, Memorandum to Chief Kenneth I. Harms, 
"Citizen Complaints Against Police Officers," (May 29, 
1979), p. 1.
 

97 Ibid., pp. 23.

98 Ibid., p. 7.

99 Ibid., p. 10.

100 Ibid., p. 12.

101 Miami Police Department, Departmental Order, #2,

Chapter 8. 

102 Ibid. 

103 "Miami Police Face Series of Inquiries After Shootings," 

The New York Times (July 18, 2001). 

104 Bruce Porter, The Miami Riot of 1980 (Lexington: 
Lexington Books, 1984). U.S. Commission on Civil 
Rights, Confronting Racial Isolation in Miami 
(Washington: Government Printing Office, 1982). 

105 Metro [Miami]-Dade Police Department, Professional 
Compliance Bureau, Early Identification System (internal 
memorandum). 

106 Walker, Alpert, and Kenney, Early Warning Systems: 

Responding to the Problem Police Officer. 

107 Ibid., 

108 Samuel Walker, Police Accountability: The Role of Citizen 

Oversight (Belmont, CA: Wadsworth, 2001). 

109 U.S. Department of Justice. Conference on Police 
Integrity. Report, 1997. 

110 U.S. Department of Justice, COPS Office, Law 
Enforcement Working Session on Police Practices, 
November 29-30, 2000. 


111 U.S. Justice Department, Principles for Promoting Police 
Integrity (Washington, DC: U.S. Justice Department, 
2001). NCJ 186189. Available at www.ncjrs.gov. 

112 United States v. Los Angeles, Consent Decree (2001). 
Available at www.usdoj.gov/crt/split/. 

113 United States v. Cincinnati, Memorandum of 
Understanding (2002). Available at 
www.usdoj.gov/crt/split. 

114 Commission on Accreditation for Law Enforcement 
Agencies (CALEA), Standard 35.1.15 (2001). Available at 
www.calea.org. 

115 The first published literature are the reports and articles 
co-authored by the author of this report: Walker, Alpert, 
and Kenney, Early Warning Systems. 

116 Ibid. 

117 Special Counsel Merrick J. Bobb, 11th Semiannual Report, 
p. 55. 

118 A full account is available in Special Counsel Merrick J. 
Bobb, 11th Semiannual Report, pp. 4556. James G. Kolts, 
The Los Angeles County Sheriff's Department (Los 
Angeles: Los Angeles County, 1992). 

119 Special Counsel Merrick J. Bobb, 11th Semiannual Report. 

120 Special Counsel Merrick J. Bobb, 16th Semiannual Report, 
pp. 4359, 6781. 

121 The chronology of events is very well documented in Los 
Angeles Police Commission, Report of the Rampart 
Independent Review Panel (Los Angeles: Los Angeles 
Police Commission, 2000), pp. 136162 and Appendix B, 
pp. 217219. 

122 Christopher Commission, Report of the Independent 
Commission on the Los Angeles Police Department (Los 
Angeles: City of Los Angeles, 1991). 


123 Merrick Bobb, Five Years Later: A Report to the Los 
Angeles Police Commission (Los Angeles: Los Angeles 
Police Commission, 1996). 

124 Los Angeles Police Department, Board of Inquiry into the 
Rampart Area Corruption Incident (Los Angeles: LAPD, 
2000). 

125 Los Angeles Police Commission, Report of the Rampart 
Independent Review Panel. 

126 Erwin Chemerinsky, An Independent Analysis of the Los 
Angeles Police Department's Board of Inquiry Report on 
the Rampart Scandal (Los Angeles, 2000). 

127 United States v. Los Angeles, Consent Decree (2000). The 
consent decree is available on the DOJ website: 
www.usdoj.gov/crt/split. 

128 United States v. City of Pittsburgh, Consent Decree 
(1997). Available at: www.usdoj.gov/crt/split/. 

129 Discussed in Samuel Walker, "The New Paradigm of 
Police Accountability: The U.S. Justice Department 
'Pattern or Practice' Suits in Context," St. Louis University 
Public Law Review, XXII (No. 1, 2003): 352. 

130 Robert C. Davis, Christopher Ortiz, Nicole J. Henderson, 
Joel Miller and Michelle K. Massie, Turning Necessity 
into Virtue: Pittsburgh's Experience with a Federal 
Consent Decree (New York: Vera Institute of Justice, 
2002), p. 44. Available at www.vera.org. 

131 City of Pittsburgh, Monitor's Eighteenth Quarterly Report. 
Quarter Ending February 16, 2002. 

132 Davis, et al., Turning Necessity into Virtue: Pittsburgh's 
Experience with a Federal Consent Decree. 

133 Davis, et al., Turning Necessity into Virtue, p. 15. 

134 Walker, Alpert, and Kenney, Early Warning Systems. 


135 The evidence is summarized in Samuel Walker and 
Charles M. Katz, The Police in America: An Introduction, 
4th ed. (New York: McGraw-Hill, 2003), Chs. 13, 14. 

136 National Center for Women and Policing, Men, Women 
and Police Excessive Force: A Tale of Two Genders (Los 
Angeles: NCWP, 2002). Available at www.feminist.org. 

137 Davis, et. al., Turning Necessity into Virtue, p. 49-50. 

138 Special Counsel Merrick J. Bobb, 15th Semiannual Report 
(Los Angeles: Special Counsel, 2002), p. 68. Available at 
www.parc.info. 

139 Ibid., p. 64. 

140 Special Counsel Merrick J. Bobb, 16th Semiannual Report. 

141 Early Intervention State of the Art Conference (February 
2003), available at www.policeaccountability.org. 

142 Walker, Alpert, and Kenney, 2001 

143 The first effort in this regard was the Early Intervention 
System State of the Art Conference, held in Phoenix, 
Arizona, in February, 2003. See the conference Report, 
available at www.policeaccountability.org. 


Image of United States Department of Justice seal

For More Information: 

U.S. Department of Justice 
Office of Community Oriented Policing Services 
1100 Vermont AVenue, N.W. 
Washington, D.C. 20530 
To obtain details on COPS programs, call the 
U.S. Department of Justice Response Center at 800.421.6770 

Visit COPS Online at the address listed below. 

www.cops.usdoj.gov 

Created: August 25, 2003 
CM#: e07032003 
ISBN: 1-932582-26-6