Multilevel Evaluation of 

Project Safe Neighborhoods

November 29, 2021 | 1:00 PM ET

The webinar will begin shortly
Important Notification

The evaluation project presented here is supported by Award 

Number 2019-75-CX-0008 from the National Institute of 

Justice of the U.S. Department of Justice. 

The opinions, findings, and conclusions or recommendations 

expressed here are those of the authors and do not 

necessarily reflect the positions or policies of the Department 

of Justice.
RTI International is a registered trademark and a trade name of Research Triangle Institute. www.rti.org

Project Safe Neighborhoods 

National Evaluation –

Overview & Preliminary Findings 

November 29, 2021

PSN is a nationwide initiative 

that brings together federal, 

state, local, and tribal law 

enforcement officials, 

prosecutors, community-based 

partners, and other 

stakeholders to identify the 

most pressing violent crime 

problems in a community and 

develop comprehensive 

solutions to address them.
Evaluation Team

• James Trudeau

• Lynn Langton

• Christine Lindquist

• Roger Przybylski

• Kristina Lugo-Graulich

• Basia Lopez
RTI International is a registered trademark and a trade name of Research Triangle Institute. www.rti.org

Introduction to the National Evaluation of 

Project Safe Neighborhoods

James Trudeau, RTI 

November 29, 2021

PSN is a nationwide initiative 

that brings together federal, 

state, local, and tribal law 

enforcement officials, 

prosecutors, community-based 

partners, and other 

stakeholders to identify the 

most pressing violent crime 

problems in a community and 

develop comprehensive 

solutions to address them.
Webinar Overview

Agenda

1) Introduction to PSN & 

the National Evaluation

2) Case Studies Approach

& Preliminary Findings 

3) National Assessment 

Approach & Preliminary 

Findings

4) Availability of Crime 

Statistics: Challenges & 

Solutions

5) Looking Ahead

6) Discussion & Q/A
Background: Federal Districts & US Attorneys

PSN grants are allocated to each Federal district

94 federal districts across the U.S. & territories

 31 states, D.C., or territories have 1 district each

 12 states have 2 districts each

 9 states have 3 districts each

 3 states have 4 districts each

93 US Attorneys (Guam & Northern Mariana Islands share 1) 

 Appointed by the President

 The chief federal law enforcement officer in their district

– prosecute criminal cases brought by the federal government 

– prosecute or defend civil cases where the United States is a party
PSN History & Key Elements

 Initially launched in 2001

– Initially a formula grant program

 Prior national evaluation (McGarrell et al. 2010)

– Some districts received competitive PSN grants in 2015, 2016 and 2017

 “Enhanced” model (PSN 2.0) was introduced late 2017

– “Formula grant” awarded annually to all 93 US Attorneys Offices (USAO)

– 5 key elements

 Leadership by the USAO

 Partnerships with law enforcement & community groups

 Targeted & prioritized enforcement using data & technology

 Prevention of additional violence through varied efforts

 Accountability of the USAO to produce results

– The focus of this evaluation, funded by the National Institute of Justice
2021 Updates to Key Elements

 The administration updated the elements in mid-2021 based on 

successful violent crime reduction initiatives: 

– Community engagement

– Prevention and intervention

– Focused and strategic enforcement 

– Accountability

 An important aspect of PSN remains using research and analysis to 

inform decision-making on violence reduction strategies

 More broadly, the administration’s violence reduction strategy 

expands upon PSN:

– Stemming the flow of firearms used in crime

– Supporting successful reentry 

– Community violence interventions

– Perceptions of police legitimacy
PSN Varies Greatly in Local Practice

 Target Enforcement Area(s) 

– The entire USAO district – which can be an entire state!

– 1 or more entire cities

– Specific neighborhoods in 1 or more cities

 Types of crime

– Most target firearm-related violent crime

– Some focus on specific types of crime (domestic violence, gang involved)

 Approaches

– Law enforcement is usually central – local, state & Federal

– Prosecution: coordination among Federal, state, tribal & local prosecutors

– Targeting the most violent offenders is common 

– Variety of community organizations & models

– How PSN fits into the array of other crime & violence reduction efforts
PSN Evaluation Goal & Objectives

PSN National Evaluation

Goal: to develop & disseminate evidence of the effectiveness of PSN in 

reducing violent crime and the factors that influence effectiveness

Objectives:

1. Conduct implementation and outcome evaluations at the national level 

and in 10 selected sites

2. Develop findings, materials, and reports that benefit federal personnel 

who administer PSN and local PSN practitioners

3. Disseminate findings to practitioners, policymakers, and researchers 

through scholarly products and presentations; and

4. Submit all resulting data sets and supporting documentation for future 

replication and reanalysis by other researchers

The evaluation is anticipated to run through 2023.
Overview of the PSN National Evaluation

Evaluation includes two tiers

With these complementary tiers, the evaluation can study PSN both broadly & deeply 

NATIONAL ASSESSMENT

CASE STUDY

• All 94 USAO Districts in the U.S. & territories

• Broad, relatively high-level look at the PSN program nationally

• In 10 districts selected in collaboration with DOJ

• Intensive, detailed look at examples of local PSN programs

• Not considered as necessarily representative of all PSN programs
What do the two tiers have in common?

Implementation: How PSN is implemented locally, particularly regarding 

the 5 key principles of the enhanced model: 

 Leadership by the U.S. Attorney

 Partnerships with law enforcement & community groups

 Targeted & prioritized enforcement using data & technology

 Prevention of additional violence through varied efforts

 Accountability of the U.S. Attorney to produce results

Outcomes: Changes in violent crime & prosecutions

 Pre/post the enhanced model: trends in 2015-17 vs. 2018 & later (and 

2021 & later in subsequent analyses)

 Using available data to account for contextual influences 

Analyzing implementation & outcomes together to identify approaches 

that lead to better outcomes

Focused on the enhanced PSN model introduced in late 2017 (and 

changes introduced in 2021) 
How do the two tiers differ?

National Assessment 

(94 districts)

Case Studies 

(10 districts)

Implementation 

Data Sources 

- Materials provided to DOJ 

by Task Forces

- Interviews 

- Web surveys

- Monthly Calls with PSN 

Coordinators

- Semi-annual site visits & 

stakeholder interviews

Outcome Data 

Sources

- FBI Uniform Crime 

Reporting Program Data

- Prosecutions: EOUSA 

filings & sentencing data

- Incident-based, geo-coded 

crime data from LEA

- Calls-for-service data in 

some districts

- Resident surveys (under 

consideration)

Unit of Analysis - USAO district

- Target enforcement areas, 

if possible

- Target enforcement areas

Compared to 

What?

- Pre/post enhanced PSN

- Areas within districts, 

where possible

- Similar comparison areas 

(if possible)
RTI International is a registered trademark and a trade name of Research Triangle Institute. www.rti.org

Preliminary Findings from the Case 

Study Component

Christine Lindquist, RTI
Recap of case study goal and research questions

Goal

– assess the implementation and impact of the enhanced PSN 

model in an in-depth manner among select PSN sites

Research questions

– Within each case study TEA, how were the core PSN 

principles implemented? What partnerships, strategies, and 

activities were leveraged?

– Within each TEA, how did PSN affect violent crime...

 over time (pre- and post- enhanced PSN)?

 compared to trends in comparable, non-PSN areas?

 compared to trends in crimes not targeted by PSN?
Case study site selection

Site selection criteria

– ability to offer a strong test of the enhanced PSN model

– availability of incident-level crime data for the TEA(s)

– USAO interest in participating

Sites and TEAs selected

– 10 USAO districts (14 TEAs) selected through a rigorous 

process & in collaboration with PSN Evaluation Working 

Group 

– not necessarily representative of all PSN initiatives

– collectively offer some geographic and programmatic 

diversity
Case study sites: 14 TEAs in 10 USAO districts
Case study site visits

Methods

– First round held virtually 

(May-September 2021)

– 1 hour, video interviews 

covering:

 district’s history with PSN

 TEA context

 Partnerships

 implementation of 5 

principles

 challenges and 

innovations

– 145 PSN stakeholders 

interviewed across the 10 

districts

Stakeholders interviewed

 USAO staff

 local LEA partners

 State LEA partners

 federal LEA partners

 District Attorney’s office partners

 corrections partners

 community- and faith-based 

organizations

 research partners 
Case study analysis

– Site-specific reports and 

logic models produced 

 Provided to sites

– Findings synthesized by 

evaluation team to 

identify 

 cross-site similarities

 site-specific themes

Community context

• USAO leadership

• Partnerships 

• Targeted enforcement efforts 

• Prevention of additional violence 

• Accountability 

Implementation of 

enhanced PSN principles

Challenges

Innovations

Focus of today’s presentation:
Case study framework: high level “PSN” logic model

• DOJ guidance re: violence 

reduction strategies (PSN pillars, 

other guidance)

• Funding (federal and other) 

• Training & technical assistance 

(federal and other)

Key PSN Strategies and Activities Outputs and Intended Outcomes

Community Context

• Violent crime trends (rates, offender characteristics, gang-involvement, weapon 

involvement, geographic concentration) 

• Key issues facing the community (e.g., racial segregation, poverty, opioid epidemic)

• Community-law enforcement relationships

Inputs and Resources

• USAO leadership (Principle 1)

• Partnerships (Principle 2)

• Other resources and process for 

developing and overseeing PSN 

strategy in district (e.g., PSN 

Working Group, research partner, 

strategic planning activities)

• Targeted enforcement efforts (Principle 3)

− Identifying highest priority people and 

places and sharing this information with all 

stakeholders (e.g., shoot reviews)

− Ensuring successful prosecution of the 

highest priority cases (increased 

prosecution resources, strategic decision 

making, enhanced investigation and 

evidence collection, efforts to enhance 

victim/witness cooperation)

• Prevention of additional violence (Principle 4)

− Focused deterrence efforts (e.g., offender 

notification/reentry programs, youth 

violence prevention efforts)

− Victim/witness-focused efforts

− Primary prevention (CPTED, community 

forums)

− Awareness-raising activities

• Accountability for results (Principle 5)

− Defining and monitoring outputs and 

intended outcomes

− Using such data to refine strategy

• Law enforcement outputs (e.g., 

evidence gathered, # arrests, 

clearance rates)

• Prosecution outputs (e.g., number 

of cases screened or referred)

• Number of individuals served 

through prevention efforts

• Number of community outreach 

events held

• Increased victim cooperation

• Increased community engagement

• Improved prosecution outcomes 

(e.g., convictions, sentence length)

• Reductions in violent crime 

(firearms violence)
Community context in TEAs

 Violent crime (especially firearm-related violence) 

increasing in all TEAs

– More youthful and gang-involved offenders

– In some sites, crime now becoming more disbursed geographically 

– Proliferation of guns in many communities

 Strained relationships between community members and 

law enforcement in most TEAs

– Has reduced victim/witness cooperation and clearance rates

 Influence of the pandemic

– Associated with the increase in crime

– Has impacted PSN activities (formerly in-person activities either 

suspended or implemented virtually)
USAO leadership (Principle 1)

 In almost all sites, USAO leadership is perceived to be 

very strong and central to the district’s violent crime 

reduction strategy

– Key activities: convening stakeholders/partners, mediating disputes 

among partners, identifying and leveraging federal resources, 

guiding (but not dictating) strategic planning

 Main sources of variation 

– PSN coordinator’s tenure with the initiative and time available for 

the position

– U.S. Attorney commitment to PSN

– In some sites, very little institutional knowledge of PSN and or 

limited bandwidth 
Partnerships (Principle 2) 

 In almost all sites, partnerships with law enforcement 

agencies (federal and local) perceived to be very strong

– Examples: inter-agency task forces, leveraging shared 

resources, data sharing agreements

 In almost all sites, partnerships with community-based 

organizations were perceived to be much weaker

– A number of challenges to developing formal partnerships were 

identified

– Enhancing these partnerships was identified as a priority in some 

sites

– A few sites already had very strong partnerships with communitybased organizations
Targeted enforcement efforts (Principle 3)

 Most sites have robust data-/tech-driven processes to 

identify the most significant offenders, groups, and places

– Typically done by local LEA crime analysts

 Most sites have structured processes in place to share and 

discuss this information with other PSN stakeholders 

– Example: shooting reviews

– Main sources of variation: whether community-based organizations 

are included in these reviews, extent of mitigation efforts (e.g., 

outreach/resource provision to victims and witnesses, place-based 

interventions) resulting from reviews

 Decision making re: the level of prosecution is typically less 

structured, involving informal communication with state and 

federal prosecutors
Prevention of additional violence (Principle 4)

 Most common activity: offender notification programs

– notifying high risk offenders (usually reentering) of negative 

consequences and providing services and resources to help

– But these activities occur infrequently in most sites

 Other activities

– victim/witness-focused interventions to prevent further violence 

and increase cooperation with prosecution

 very intensive efforts in some sites

– social media monitoring for prevention

– place-based interventions (e.g., CPTED)

– community awareness raising activities re: PSN

– agency involvement in existing community activities (e.g., youth 

violence prevention, community forums)
Accountability (Principle 5)

 In most sites, activities under this principle were limited to 

sharing of data 

– Examples: sharing daily crime reports, number of cases prosecuted

 Uncommon for sites to use such data to refine the PSN 

strategy or foster accountability for outcomes

– Some activities of the research partner (e.g., tracking outputs and 

outcomes) were considered to provide accountability

 Many stakeholders noted the difficulty of identifying 

appropriate indicators of effectiveness 

– Examples: long-term impact of their strategic decisions on safety, 

whether the highest risk individuals are changing their behavior

– Crime rates dependent on many other factors
Challenges to implementing key PSN principles

 Effectively engaging the community (formalized 

partnerships)

– Specific challenges: identifying and vetting partners, finding 

meaningful involvement, limited resources, lack of community 

interest in having USAO involved

 Limited resources and personnel

 Turnover in key roles (e.g., police chiefs, PSN 

coordinator, U.S. Attorney) 

– creates instability and threatens collaborations

 COVID-related restrictions on in-person activities

 The number of overlapping violence reduction initiatives 

in the district
Perceived innovations

 Collaboration among stakeholders was noted in most 

sites as the most innovative aspect of their PSN activities

 Other, site-specific activities were noted:

– victim/witness coordinator position

– mentoring component

– hospital-based victim program

– NIBIN program

– co-locating certain personnel

– county crime analysis centers 

– USAO in-house investigators
Concluding thoughts from first round of site visits

 PSN activities are taking place under very challenging 

conditions in each case study TEA

 The diversity in community context and PSN strategies among 

the case study sites will allow for very interesting findings 

 USAO leadership in violence reduction strategies is very 

strong across the case study sites

– The role of the PSN coordinator is critical to success

 Most districts have very strong targeted enforcement efforts to 

focus investigation and prosecution resources on the most 

significant offenders and groups 

 Most districts could benefit from developing stronger 

community partnerships

– This will likely be addressed as USAOs refine their PSN strategy 

to reflect new design pillars
RTI International is a registered trademark and a trade name of Research Triangle Institute. www.rti.org

Preliminary Findings from the National 

Assessment of PSN

Lynn Langton, RTI
Goal and Key Research Questions

Goal: assess the nationwide implementation and impact of the 

enhanced PSN model

Key Research Questions:

How were the 5 PSN principles implemented across districts?

How did PSN impact violent crime rates nationwide?

To what extent were violent crime outcomes related to how the 5 

principles were implemented?

Goal and Key Research Questions
Goal and Key Research Questions

Unit of Analysis: US Attorney Districts (excluding territories)

 District-level focus is driven by 

underlying goals of PSN and 

availability of crime/prosecution data

 Some districts have multiple Target 

Enforcement Areas (TEAs), which 

are aggregated for analysis

Time Periods of Interest:

 2015 - 2017 – Baseline period (competitive PSN funding)

 2018 - 2020 – Initial Intervention period (PSN 2.0) 

 Mid-2021 - 2023 – Revised PSN

Unit and Time Period of Analysis
Measuring PSN Implementation

The implementation assessment will focus on central tenets of PSN 

 Interested in ALL PSN activities, not just those funded by DOJ grants

Analysis of reports and information collected by DOJ

• Semi-annual reports (FY18 – 1st half FY19)

• Strategic plans

Annual interviews with PSN Coordinators

• Beginning in 2022

Annual web-based surveys of key PSN stakeholders (Task Force 

members/partners) in each District

• Following the PSN Coordinator Interviews
Implementation Assessment Methodology

Descriptive statistics on variations in implementation of the 5 principles

Within each principle, using Item Response Theory (IRT) factor analysis to 

determine how best to group measures/responses related to each PSN 

principle

Using higher order LCA to identify groups of districts with similar 

configurations of implementation

• Moving beyond individual implementation elements to view the various 

PSN approaches more holistically 
Implementation Analysis: Select Findings

Leadership by the US Attorney to convene all partners

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

Unreported Did not

meet

Daily Weekly Biweekly Monthly Quarterly Other

Percent of districts

Frequency of PSN Task Force Meetings, 2019

In majority of districts, PSN task forces met at least monthly. 
Implementation Analysis: Select Findings

Partnerships at all level of law enforcement and with the community

0 10 20 30 40 50 60 70 80 90 100

Tribal criminal justice agencies

Public services or some other entity

Child protective services

Training and technical assistance provider(s)

Businesses

Physical or mental healthcare or substance use providers

K–12 schools

Community-based victim services

Research partner

Crime prevention programs

State, local, or tribal leadership

Court or pretrial agencies

Community groups or service providers, faith-based groups,...

Corrections or community corrections

Federal law enforcement agencies

State law enforcement agencies

Local law enforcement agencies

State or local prosecutors

Percent of districts

PSN Partnerships, 2019
Implementation Analysis: Select Findings

Partnerships at all level of law enforcement and with the community

0

5

10

15

20

25

30

35

1-5 6-8 9-11 12-13 14 or more

Percent of districts

Average number of PSN partners, 2019
Implementation Analysis: Select Findings

Targeted and prioritized enforcement efforts that identify the 

offenders driving violent crime rates in the district

0 20 40 60 80 100

Other

Anti-Gang ordinances/civil injunctions

Truancy strategies

Public housing strategies

Gunshot detection technology

Juvenile gun violence reduction strategies

Drug courts

Domestic violence programs

Other technology-based strategies

Probation/parole sweeps

Violent fugitive roundups

Worst of the worst

Social network analysis

Directed patrols

Most violent offender list

Comprehensive crime gun tracing

Criminal incident review

Stricter prosecutorial policies for felons

Field interrogations

Case screening for federal prosecutions

Percent of districts

PSN Enforcement Activities/Strategies, 2019
Implementation Analysis: Select Findings

Targeted and prioritized enforcement efforts that identify the 

offenders driving violent crime rates in the district

0

5

10

15

20

25

30

35

40

5 or less 6-9 10-12 13-16 17+

Percent of districts

Average number of PSN enforcement 

activities/strategies used, 2019
Implementation Analysis: Select Findings

Prevention of additional violence through public awareness, offender 

notification, and local prevention and reentry efforts

0 10 20 30 40 50 60 70 80 90 100

University based programs

Hospital trauma center outreach

Prison-based strategies

Programs involving former gang members

Mentoring programs

Substance use programs

Employment or vocational training programs

Offender notification/call-ins

School-based prevention or youth programs

Working with or utilizing reentry programs

Law enforcement/community trust building

Crime awareness, self-defense, or other education programs

Outreach efforts/campaigns

Media strategies or PSAs

Neighborhood development or community building/engagement programs

Percent of districts

PSN Prevention Activities, 2019
Implementation Analysis: Select Findings

Prevention of additional violence through public awareness, offender 

notification, and local prevention and reentry efforts

0

5

10

15

20

25

30

0-2 3-4 5-7 8-10 10+

Percent of districts

Average number of prevention activities, 2019
Implementation Analysis: Select Findings

Accountability for results based on analysis of outcomes rather than 

outputs

0 10 20 30 40 50 60 70 80 90 100

Tribal law enforcement intelligence and data

Survey data

Systematic observations of places by trained observers

Client risk assessments

Focus group data

Community data

Social service provider data

Public health data

Youth-focused data

Internal agency complaint data

State, local, or tribal prosecution data

Corrections data

Court data

FBI UCR data

Federal prosecution data

Intelligence from confidential or other informants

National Integrated Ballistics Intelligence Network (NIBIN)

State law enforcement intelligence and data

Federal law enforcement intelligence and data

Local police call, crime, and arrest data

Percent of districts

Types of data used to track PSN outcomes, 2019
Implementation Analysis: IRT and LCA

 IRT analysis suggest a one- or two- factor solution for each of the 

individual principles

– Generally, in terms of partnerships, enforcement, prevention, and data 

analysis, districts are distinguished not by which types of 

efforts/activities they are engaged in but by how many

 Next steps: Running the higher order LCA model to identify classes 

of districts based on PSN implementation
Measuring PSN Outcomes

The outcome assessment will focus on the impact of PSN on violent 

crime rates nationwide

Primary Sources of Data

Public crime data

• 2015 - 2020 - FBI UCR Summary Reporting System (aggregated to 

District)

• 2021 - 2023 – NIBRS + ??

Prosecution data

• 2015 - 2023 – EOUSA filings and sentencing (PSN flag added in 2018)
Outcome Assessment Methodology

Using Multilevel longitudinal growth models (MLLGM) to examine trends 

in crime across several periods: Illustrative Crime Outcome Trajectories by Group and Phase

• 2015 – 2017 Baseline period

• 2018 – 2020 PSN 2.0 Intervention

• Initial analyses exclude 2020

• 2021 – 2023 PSN 2.1 Intervention

• Not reflected here

Splitting districts into two groups based on whether they received competitive PSN 

funding during the baseline period enables assessment of:

• Effect of PSN 2.0 on crime in districts that did NOT get federal PSN funding in 

baseline period (orange)

• Effect of PSN 2.0 on crime in districts that DID receive federal PSN funding in 

baseline period (blue)

• Difference in slopes of two groups in the intervention period (orange vs. blue)
Outcome Assessment Methodology

Preliminary Analyses:

• 2016-2019 monthly UCR SRS data 

• Excludes agencies that reported annually only (<10%)

• Focused on reporting from local police departments and sheriffs

• Agencies nested within districts (n=87, excluding territories)

• Based on counts rather than rates

• Select crime types 

• All violence – murder, rape/sexual assault, robbery, aggravated 

assault

• Murder

• Robbery with a firearm
Outcome Assessment: Preliminary Findings

0

1

2

3

4

5

6

7

8

9

10

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

2016 2017 2018 2019

All Violence

Enhanced only Competitive and enhanced

• Trend is flat during baseline period for both groups

• Significant decrease in intervention period for PSN 2.0 only group 

(orange)

• Nonsignificant decrease in intervention period for competitive funding 

group (p=.06) (blue)
Outcome Assessment: Preliminary Findings

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

2016 2017 2018 2019

Murder

Enhanced only Competitive and enhanced

• Trend is flat in baseline period for both groups 

• Significant decrease in intervention period for PSN 2.0 only group 

(orange)

• Nonsignificant decrease in intervention period for competitive funding 

group (blue)

• Significantly larger relative decrease during intervention period for PSN 

2.0 only group (1.75X)
Outcome Assessment: Preliminary Findings

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

2016 2017 2018 2019

Robbery with a firearm

Enhanced only Competitive and enhanced

• Significant decrease in both periods for both groups 

• Larger (but nonsignificant) relative decrease during intervention period 

for both groups

• 1.5X for PSN 2.0 only group (orange)

• 1.8X for Competitive funding group (blue)
National Assessment Next Steps

 Running additional models for other crime types, using crime rates 

rather than crime counts, considering moving averages 

 Modeling the effects of implementation on violent crime outcomes

– Outcome and implementation data analyzed together to understand 

impact of differences in PSN implementation on outcomes

 Incorporating community factors and characteristics into the 

models to control for variations across districts and over time

 Incorporating federal PSN prosecutions and sentence lengths into 

models

 Continuing to assess how best to measure PSN outcomes given the 

retirement of the FBI’s UCR Summary Reporting System
RTI International is a registered trademark and a trade name of Research Triangle Institute. www.rti.org

Data Challenges: Implications of 

Changes in Crime Data Reporting in 

the FBI’s UCR Program 

Roger Przybylski, JRSA
Importance of crime data for the PSN evaluation

 Reductions in violent crime and firearms violence are key 

outcome goals for PSN

 Measures of both are needed for the national 

assessment as well as for each case study

– Geographically, data are needed at multiple levels of 

aggregation

 District, Target Enforcement Area(s), and comparison areas

– Temporally, data are needed pre- and post- enhanced PSN

 2015 through at least year-end 2022

 Data pertaining to other crimes not targeted by PSN are 

needed as well 
Data collection decision making

 Decision making about the best data sources and 

collection methods to use are influenced by several 

factors

– Evaluation project timing and resource constraints

 Data collection must be economical and efficient

– Data quality and completeness are paramount concerns

 Consistently reported data over time to support interrupted time 

series analysis 

 Consistently reported data across jurisdictions and multiple levels 

of aggregation to support comparative analyses
Data sources to be used

 Incident-level crime data will be obtained directly from 

local law enforcement agencies for the case study sites 

– Through secure FTPs or RTI’s CFS Analytics platform

 Using a similar approach for the national assessment is 

not feasible

– 18K+ local law enforcement agencies nationwide

 Best alternative is to use data collected through the FBI’s 

Uniform Crime Reporting (UCR) program

– Single, central source of data 

– Theoretically consistent reporting across time and 

jurisdictions
Characteristics of UCR crime data

 Administered by the FBI since1930

 Nationwide, cooperative effort of more than 18,000 law 

enforcement agencies to voluntarily report data on 

crimes brought to their attention

– Serious violent and property crime

– Not all crimes are reported to law enforcement

 Nationwide reporting

 With few exceptions, consistent definitions used across 

time and jurisdictions

 Can be aggregated/disaggregated to multiple levels
Characteristics of UCR crime data

 Traditionally reported as summary statistics (SRS)

– “Part 1” violent and property crimes

– Time lag between reporting and when data are released

 Blueprint for transition to National Incident Based 

Reporting System (NIBRS) rolled out in late 1980s

 Key differences between NIBRS and SRS

– Incident rather than summary reporting

– Far more detailed data on crimes known to law enforcement

 Characteristics of the crime incident, victims and offenders 

(if known)

 Small number of states and law enforcement agencies 

began NIBRS reporting in 1990s, more have followed in 

subsequent years
FBI imposed sunset of the SRS

 FBI stopped accepting SRS data at year-end 2020

– Only NIBRS data accepted beginning January 2021

 Only about one half of the nation’s 18K+ LE agencies were 

capable of reporting NIBRS data in January 2021

– Accounts for about 55% of the nation’s law enforcement 

agencies and 50% of the nation’s population

– Several major cities currently are not accounted for in NIBRS

 New York City, Los Angeles, Chicago, Philadelphia, Phoenix 
Implications for the PSN evaluation

 The sunset of SRS presents a major challenge for the 

PSN evaluation

– Interrupts the series of data needed for the national 

assessment outcome evaluation

– The number of additional law enforcement agencies 

reporting NIBRS will increase, but incrementally

 FBI UCR program cannot be used a a single source of 

crime data unless numerous USA Districts and/or TEAs 

are left out of the national assessment 
Addressing the Challenge

 Two key lines of inquiry are being pursued to 

potentially mitigate the impact of the UCR change

– In-depth analysis of NIBRS coverage to determine precisely 

which PSN TEAs are covered by NIBRS reporting agencies

 205 TEAs; 65 covered (32%), 98 not covered (48%), 42 maybe 

(20%)

– Inventory of state UCR programs to determine which will 

continue to accept SRS data from agencies in their state

 Some state programs will continue to accept SRS data from 

agencies that have not yet fully transitioned to NIBRS
Addressing the challenge

 By using NIBRS data available through the FBI and any 

SRS data available through state UCR programs we 

hope to close the “missing” data gap

– NIBRS data can be converted to SRS data to support trend 

analysis

 Using the FBI and state UCR programs as data sources 

keeps crime data collection practical and economical

– Keeps data sources limited

– Expands geographic and temporal coverage beyond what is 

available when relying solely on NIBRS coverage
The challenge of non-fatal shooting data

 Accessing valid data on non-fatal shootings presents 

another challenge

– Both SRS and NIBRS data identifies whether a firearm was 

involved a crime, but not whether an actual shooting 

occurred

– Several sources of non-fatal shooting data exist, but the 

available data does not meet the evaluation’s temporal or 

geographic requirements

– Non-fatal shooting data does exist at the local hospital/ER 

level, but using these as sources nationwide is not feasible

 The evaluation team continues to explore emerging 

datasets with the fallback of using firearm involvement in 

a crime as a limited proxy 
RTI International is a registered trademark and a trade name of Research Triangle Institute. www.rti.org

National Evaluation of 

Project Safe Neighborhoods –

Looking Ahead 

November 29, 2021
Looking Ahead – Challenges & Solutions

Obtaining crime data after UCR-SRS

“sunsetted” at end of 2020

• Explore two-pronged approach

• Use NIBRS data from FBI where 

available

• Augment with data from states

Addressing 2020 “shock”

• Explore options, e.g., comparing 

results with & without 2020 in analysis 

Incorporating 2021 shift in key elements

• Analyze crime trends in 3 segments

• 2015-17, 2018-20, & 2021 on
Looking Ahead – Data Collection & Analysis

Case Studies

• Continue to obtain incident-based crime data 

• Explore conducting resident surveys

• Analyze crime data & resident survey data, if 

conducted

• Conduct qualitative analysis of implementation 

& use to help explain outcomes

National Assessment

• Finalize initial analyses

• Obtain & analyze crime data

• Using Census data for contextual factors

• Conduct PSN Coordinator interviews & 

stakeholder surveys & analyze data

• Add “middle tier” to make use of incidentbased data where available

• To supplement National Assessment & 

Case Studies
Looking Ahead – Reporting & Dissemination

Reports to DOJ & Case Study Sites

• Site-specific reports to case study sites

• Reports to DOJ on findings & 

recommendations

Broader Dissemination 

• Practitioner & research publications & 

conferences

• Future events such as this

Archive Data (with safeguards)

• To support additional analysis & provide added 

value to NIJ’s investment
RTI International is a registered trademark and a trade name of Research Triangle Institute. www.rti.org

National Evaluation of 

Project Safe Neighborhoods

November 29, 2021

Discussant

Ron DeWald,

Assistant US Attorney &

PSN Coordinator

Northern District of Illinois
In Closing

Thank you!

 DOJ & NIJ for supporting the evaluation & hosting this 

webinar

 People involved in PSN – USAO staff & their partners –

for providing their insights through interviews & surveys

 YOU for attending today

 If you would like to contact the evaluation team, please 

email us at

PSNEvaluation@rti.org