DOI: 10.3886/icpsr35467
title: Assessing Police Performance in Citizen Encounters, Schenectady and Syracuse, NY, 2011-2014
title: Archival Version

description: These data are part of NACJD's Fast Track Release and are distributed as they were received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except for the removal of direct identifiers. Users should refer to the accompanying readme file for a brief description of the files available with this collection and consult the investigator(s) if further information is needed.
This study examined how police managers would use information about their officers' performance in procedural justice terms. The project provided for the injection of citizen assessment of service quality into systems of police performance measurement and accountability. Information on the quality of police-citizen encounters was drawn from surveys of citizens who had contact with the police in each of two cities, Schenectady and Syracuse, New York. Following the accumulation of survey data to form a baseline, survey results on citizens' satisfaction and judgments about procedural justice in their police contacts were summarized and reported to command staffs on a monthly basis
through the departments' respective Compstat meetings. In this way the project
provided for measures of police performance with respect to procedural justice with sufficient periodicity that the information was potentially useful in managing performance.
The study addressed four specific questions:
Does performance on these outcomes - procedural justice and citizen satisfaction - improve when information on these outcomes is incorporated into departments' systems of performance measurement and accountability?What do police managers do with this information, and how (if at all) are field supervisors and patrol officers affected by it?Are survey-based measures of citizens' subjective experiences valid measures of police performance, that is, do they reflect the procedural justice with which police act?Can survey based measures be deployed economically (e.g., through targeted sampling), and can other, less expensive measures of the quality of police-citizen encounters be substituted for survey-based measures?
To answer these questions, researches used a mixed methods data collection plan. In both Schenectady and Syracuse, a survey was administered to people who had recent contact with the police. Semi-monthly samples were randomly drawn from police records of calls for service, stops, and arrests from mid-July, 2011, through mid-January, 2013. Across the 18 months of surveying, 3,603 interviews were completed. Also carried out, was a survey of key informants in each city - neighborhood association leaders - in order to extend the assessment of public perceptions of the local police beyond those who have direct contact with police to the larger community. Interviews with patrol officers and supervisors were also conducted in both sites, once at about the mid-point of the 18-month police services survey and again at the conclusion of the surveying. Interviews were conducted with the commanders shortly after the project was introduced to them in October of 2011. Finally, in Schenectady, "armchair" observation of a subset of the 1,800 encounters about which the citizen had already been interviewed was conducted.
The collection contains 7 SPSS data files and 6 Syntax files:archive_Census_beat.sav (n=30; 28 variables)archive_keyinformant_analysis.spsarchive_keyinformant_survey.sav (n=90; 28 variables)archive_obs_byenc.sav (n=476; 79 variables)archive_obs_byobserver.sav (n=1,078; 476 variables)archive_obs_enc_analysis.spsarchive_obs_enc_var_construction.spsarchive_police_data.sav (n=3,603; 9 variables)archive_policeservices_survey_analysis.spsarchive_policeservices_survey_closed.sav (n=3,603; 148 variables)archive_policeservices_survey_open.sav (n=1,218; 23 variables)archive_policeservices_survey_var_construction.spsSyntax to replicate results - list by table.pdf
For confidentiality reasons, the qualitative interviews with citizens, police sergeants, patrol officers, and commanders regarding their experiences are not available as part of this collection.
description: The primary objective of the project was to learn about whether and how police managers would use information about their officers' performance in procedural justice terms.
description: All research was conducted in and with the cooperation of the Schenectady and Syracuse Police Departments. Data collection was carried out using four main methods:
<strong>Citizens (archive_policeservices_survey_closed.sav, archive_policeservices_survey_open.sav, archive_police_data.sav, and archive_Census_beat.sav)</strong> - In both Schenectady and Syracuse, a survey was administered to a sample of respondents who had recent contact with the police. In each of the two study departments, semi-monthly samples were randomly drawn from police records of calls for service, stops, and arrests, from mid-July, 2011, through mid-January, 2013. Respondents were interviewed by phone within one to five weeks of their contact with police. This 36-wave police services survey tapped citizens' subjective experience with police along several dimensions, along with their broader outlooks about the local police, and formed the basis for performance measures that were compiled on a monthly basis and presented to the departments' command staffs at Compstat meetings. Across the 18 months of surveying, 3,603 interviews were conducted, or approximately 100 per month in each city. The interviewed samples bore strong resemblances to the populations from which they were drawn, though in both sites (for different reasons) the subsample of stops without arrest was small. <strong>Neighborhood Association Leaders (archive_keyinformant_survey.sav)</strong> - A survey of key informants in each city was conducted in order to extend the assessment of public perceptions of the local police beyond those who have direct contact with police to the larger community. Key informants were surveyed electronically or by mail at three points in time: at the outset of the project (in September of 2011 and January of 2012 in Syracuse and Schenectady, respectively), mid-way through surveying (in December of 2012), and after the police services survey had concluded.<strong>"Armchair" Observation (archive_obs_byobserver.sav and archive_obs_byenc.sav)</strong> - In Schenectady, an "armchair" observation of a subset of the
1,800 encounters about which citizens had been interviewed previously was conducted, relying on the
video and audio recordings that are routinely made in that department. For this purpose an observation protocol was formulated that built on the platform of more than 40 years
of systematic social observation of police in the field, adapted to the purpose of
measuring procedural justice. Pairs of observers each coded 539 encounters, in
411 of which both observers were successful in identifying the "primary citizen," the
citizen about whose subjective experience survey data was collected. With these data citizens' subjective experience in terms of how officers act could be analyzed.<strong>Patrol Officers, Field Supervisors, and Commanders</strong> - Interviews with patrol officers and supervisors were conducted in both sites, once
at about the mid-point of the 18-month police services survey and again at the
conclusion of the surveying. Interviews were also conducted with commanders shortly
after the project was introduced to them in October of 2011. The patrol interviews
enabled the assessment of officers' and field supervisors' view about the importance of
"customer service," whether and if so what supervisors were doing to raise awareness of the importance of high-quality interactions with citizens, and any sources of
resistance on officers' or supervisors' parts. The interviews with commanders allowed for the following: gathering their views on the Compstat process, assess their understanding
of "what matters" in police performance, and to assess whether commanders were
doing anything to raise awareness about the importance of "customer service." These data are unavailable with the collection at this time.
description: The survey data are contained in seven SPSS data files:
The data file<strong> archive_policeservices_closed.sav (n=3,603; 148 variables)</strong> includes variables related to the respondent's general opinion of the police, the police in the respondent's neighborhood, the respondent's feelings about obeying the law, and about recent contact with the police. In addition, there are variables about motor vehicle stops, field stops, assistance questions (when the respondent contacted police), other police contacts, arrests, judgment and acceptance, and basic demographics.
The data file<strong> archive_policeservices_survey_open.sav (n=1,218; 23 variables)</strong> includes variables that detail respondents dissatisfaction with police treatment, police respond time, view of police protocols and assistance, as well as whether the respondent was released or spent time in jail.
The data file <strong>archive_keyinformant_survey.sav (n=90; 28 variables)</strong> contains variables related to feelings of safety, perceptions of the Schenectady Police Department, organizational policies, and demographics.
The data files <strong>archive_obs_byenc.sav (n=476; 79 variables)</strong> and <strong>archive_obs_byobserver (n=1,078; 476 variables)</strong> contain variables that capture information on requests that citizens made of officers and how police responded to those requests, requests or commands by officers and how citizens responded to those requests or commands, officers' use of police authority (e.g., searching or frisking the citizen, the use of physical force, arrests, citations), and forms of disrespect by citizens and/or officers. In addition observers summary characterizations of selected features of interactions, such as how much patience officers exhibited, how well officers listened to citizens, and how much consideration the officers showed for the citizens' point of view were also included. Data file <strong>archive_obs_byenc.sav</strong> contains one record for each encounter in which at least one observer was able to identify the primary citizen, and for which the two observers' coding has been combined. Data file <strong>archive_obs_byobserver.sav</strong> contains two records for each encounter and one record for each of two observers.
 The data file <strong>archive_police_data.sav (n=3,603; 9 variables)</strong> includes variables that detail police department incident records from Schenectady and Syracuse. They include police beat/zone, police shift, call for service problem category, arrest type and disposition, arrestee condition, and most serious charge level.
The data file <strong>archive_Census_beat.sav (n=30; 28 variables)</strong> includes variables that detail Census data on police beats, drawn from the 5 year American Community Survey, 2008-2012 estimates for block groups, which can be matched to police data (and thereby to survey data) with the key variables <strong>'site'</strong> and <strong>'beat'</strong>. These include basic demographic information, as well as police beat/zone, and research site.
description: Sampling for neither the Police Services Survey nor Schenectady observations was simply random, as stops and arrests were oversampled. Weight variables were formed to adjust for sampling probabilities, and additional weight variables were formed to adjust for both sampling probabilities and sample attrition.
The dataset archive_obs_byenc.sav contains three weight variables: v_wt, v_wt_pop, and v_wt_pop2. Additionally, the dataset archive_policeservices_survey_survey_closed.sav contains seven weight variables: wt_sch, wt_syr, wt_sat, wt_PJ, wt_sch2, wt_syr2, and wt_all2.
description: Response Rates: <strong>archive_policeservices_survey_closed.sav</strong>, <strong>archive_policeservices_suvey_open.sav</strong>, and <strong>archive_police_data.sav</strong>
The following Survey Sampling and Disposition Summary Completed Response Rates are available for Schenectady and Syracuse, respectively:
Schenectady - Arrests (62.7 percent), Field interviews (58 percent), and Calls for service (64.5 percent).
Syracuse - Arrests (61.9 percent), Stops (49.8 percent), and Calls for service (65 percent).
<strong>archive_keyinformant_survey.sav</strong>
Researchers contacted 37 prospective Syracuse respondents in Wave 1, followed by 36, and 35 in the final wave. Syracuse final response rates in Waves 1 through 3 were as
follows: 67 percent (N=25); 44.4 percent (N=16); and 28.5 percent (N=10), respectively.
Researchers contacted 29 prospective respondents in Schenectady in Wave 1, 27 in Wave 2,
and 26 in Wave 3. Schenectady final response rates for Waves 1 through 3 were as
follows: 48.2 percent (N=14); 44.4 percent (N=12); and 50.0 percent (N=13), respectively.
<strong>archive_obs_byobserver.sav</strong> and <strong>archive_obs_byenc.sav</strong>: not applicable
<strong>archive_Census_beat.sav</strong>: not applicable
description: Presence of Common Scales: 
<strong>archive_Census_beat.sav</strong>: none


<strong>archive_keyinformant_survey.sav</strong>: Several Likert-type scales were used.


<strong>archive_obs_byenc.sav</strong>: none


<strong>archive_obs_byobserver.sav</strong>: none


<strong>archive_police_data.sav</strong>: none


<strong>archive_policeservices_survey_closed.sav</strong>: Several Likert-type scales were used.


<strong>archive_policeservices_survey_open.sav</strong>: none
description: Data files <strong>archive_police_data.sav, archive_policeservices_survey_closed.sav</strong>, and <strong>archive_policeservices_survey_open.sav</strong>:


The design provided for sampling contacts from police records of calls for service, stops, and arrests in each department, and conducting interviews by phone with the citizens named in those records. Samples were drawn semi-monthly from records of contacts that occurred between July 15, 2011 and January 15, 2013. Calls for service records were extracted from each department's computer-aided dispatch
(CAD) system. Arrest records were extracted from each department's record management system, and included custodial arrests as well as cases in which suspected offenders were either issued appearance tickets or released on their own recognizance. Records of stops differed across the departments: Syracuse has for many years provided for a citizen contact form on which officers record all stops that do not result in arrest; these records include stops in which a traffic ticket is issued. Schenectady, however, does not have a comparable record of stops, but rather separate records of traffic tickets and "field interview cards." Field interview cards may be completed pursuant to any contact with a citizen, whether it is police-initiated or not, but most field interview cards are based on police-initiated contacts. In Schenectady researchers sampled only from field interview cards, since sampling from traffic tickets was not at that time feasible. Thus the samples of stops in the two cities are different, in that routine traffic stops are included only in Syracuse.


Recorded contacts were assessed for their eligibility for inclusion in the survey sample. From CAD records, officer-initiated incidents were removed, as were records that lacked a recognizable first or last name, such that records with only, e.g., "unknown," "passerby," "neighbor," or "security" in the name fields were eliminated. Arrests that had been sealed by the courts and included no name were removed. Stop or field interview records that listed neither a phone number nor an address (or recorded the address as "homeless") were eliminated. In Syracuse, stop records did not include a phone number, and so efforts were made to "append" a telephone number given a name and address. Finally, if the same person appeared more than once in the same subpopulation in a single sampling period, then only the most recent incident was sampled.


During each sampling period researchers randomly sampled 300 calls for service, and researchers included all arrests and stops/field interviews in the sample of contacts. Researchers overrepresented arrests and stops/field interviews in order to better capture for separate analysis a number of incidents in which procedural justice would presumably be more challenging for officers. All of the eligible arrests and stops/field interviews were sampled, as were more than 30 percent of the calls for service in Schenectady and nearly 10 percent of the calls for service in Syracuse. Most of those who were sampled were called, excepting the stops in Syracuse, for which the process of appending telephone numbers to names and addresses was only partially successful. Substantial fractions - nearly 20 percent in each city - of the people who had called for service were contacted, and of those who were contacted, nearly two-thirds completed the interview. People who were arrested or stopped/field interviewed were much more difficult to contact; 6 to 12 percent of those who were called were contacted by interviewers. Of those who were contacted, however, completion rates were generally around 60 percent, excepting Syracuse stops. About one-quarter of those contacted declined to participate; small proportions were screened out (people who were under 18 years of age and who had called for service could not be removed from the sample as ineligible because no information about their age was in the CAD record) or were unable to complete the interview in English.


Since the samples were stratified, with different probabilities of sample selection across the different subpopulations, and since the response rates varied across subpopulations, researchers weighted the cases for most analyses of the survey data in order to represent the entire contact population in each site. Researchers applied weights that reproduced the original population proportion that each subpopulation represented, though these weights were very nearly the same as those that are based only on the probabilities of sample selection.


Researchers also note that the sampling frames were not mutually exclusive; while it does not occur often, some people who called for assistance were arrested. Furthermore, the same event, involving multiple persons, could be the subject of multiple interviews, each concerning the experience of a different person. In addition, researchers noted that across 18 months of surveying, any one person might well appear in the sample multiple times, and in fact, 101 people were each interviewed more than once about different incidents. Finally, a non-trivial proportion of those who called for service reported that they did not have a face-to-face interaction with an officer, and these respondents are not included in the analyses of procedural justice.


Data file <strong>archive_keyinformant_survey.sav</strong>:


Researchers operationalized key informants as current leaders of a neighborhood association in Schenectady or Syracuse. To identify neighborhood associations and their respective leaders in each city, researchers relied on contact lists provided by representatives of the police departments. Both police departments maintain up-to-date lists of associations and contact information for leaders, which included, in many cases, email addresses.


Data file <strong>archive_Census_beat.sav</strong>:


Census data on police beats were drawn from the 5 year American Community Survey estimates 2008-2012 for block groups.


Data file <strong>archive_obs_byobserver.sav</strong> and <strong>archive_obs_byenc.sav</strong>:


At the conclusion of the police services survey, researchers sampled from among incidents about which they had completed an interview with the citizen. Researchers requested copies of the video/audio files, with which the Schenectady Police Department obliged. Among the 1,800 incidents about which citizens were surveyed, researchers oversampled arrests and field interviews, on the assumption that these are the kinds of incidents in which procedural justice may be less readily practiced, and to ensure as much as possible that the subsamples would support separate analysis.
description: Datasets: 
DS1: Assessing Police Performance in Citizen Encounters, Schenectady and Syracuse, NY, 2011-2014
description: <strong>archive_Census_beat.sav</strong>: All persons and housing units in the United States and Puerto Rico
<strong>archive_keyinformant_survey.sav</strong>: Neighborhood association leaders in the cities of Schenectady and Syracuse, New York
<strong>archive_obs_byenc.sav</strong> and <strong>archive_obs_byobserver.sav</strong>: Police encounters with citizens in the cities of Schenectady and Syracuse, New York
<strong>archive_police_data.sav, archive_policeservices_survey_closed.sav</strong>, and <strong>archive_policeservices_survey_open.sav</strong>: Citizens in the cities of Schenectady and Syracuse, New York. Smallest Geographic Unit: <strong>archive_Census_beat.sav</strong> and <strong>archive_police_data.sav</strong>: police beat/zone ; <strong>archive_keyinformant_survey.sav, archive_obs_byenc.sav, archive_obs_byobserver.sav</strong>, and <strong>archive_policeservices_survey_closed.sav</strong>: city ; <strong>archive_policeservices_survey_open.sav</strong>: none
description: record abstracts; computer-assisted telephone interview (CATI); coded video observation; face-to-face interview; mail questionnaire; web-based surveyThese data are part of NACJD's Fast Track Release and are distributed as they were received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except for the removal of direct identifiers. Users should refer to the accompanying readme file for a brief description of the files available with this collection and consult the investigator(s) if further information is needed.For confidentiality reasons, the qualitative interviews with citizens, police sergeants, patrol officers, and commanders regarding their experiences are not available as part of this collection.