Improving Estimates of Arrest 

Related Deaths with Machine 

Learning

October 25, 2018

Government Advances in Statistical Programming (GASP!) 

Workshop

Peter Baumgartner (Data Scientist @ RTI International)
This project was conducted with funds 

awarded by the Bureau of Justice 

Statistics, Office of Justice Programs, U.S. 

Department of Justice. The views, 

opinions, and content expressed in this 

exhibition do not necessarily reflect the 

views, opinions, or policies of the Bureau 

of Justice Statistics (BJS), the Office of 

Justice Programs (OJP), or the U.S. 

Department of Justice (DOJ).
History

2000: Death in Custody Reporting Act (DICRA) passed

2003-2009: Bureau of Justice Statistics (BJS) collects Arrest 

Related Death (ARD) statistics.

2011: BJS publishes ARD statistics for 2003-2009

2014: BJS conducts a coverage assessment of the ARD 

program. Results indicate ARD data collection is not a 

census of relevant incidents.

2015: BJS begins a pilot study to test a revised data 

collection approach that includes multiple methods of 

identifying and confirming arrest-related deaths.

https://www.bjs.gov/index.cfm?tid=82&ty=tp
2016 

https://fivethirtyeight.com/features/the-best-and-worst-data-stories-of-2016/
Hybrid Approach to Data 

Collection

Phase 1: Identification

• Review news alerts and other public sources for potential arrestrelated deaths.

• Track the efficiency and coverage of various approaches for 

identifying arrest-related deaths.

• Compile quarterly lists of potential arrest-related deaths.

Phase 2: Agency survey

• Conduct follow up with law enforcement agencies and medical 

examiner/coroner’s offices to confirm identified deaths and 

collect more information about the circumstances surrounding 

the deaths.

• Survey agencies with identified deaths and others to assess and 

adjust Phase 1 identification methods.

Focus of this talk
The Process

A typical month...

1,200,000 articles

245,000 articles

8,750 articles

135 events

Article Reduction:

99.2%
Keyword Matched Articles

• Keywords are a 

noun and verb 

combination 

indicating law 

enforcement and 

relevant action.

• e.g. “officer” & 

“lethal force”

• Use word 

embedding models 

(word2vec) to 

expand keywords.
Unique Articles

• Remove exact 

duplicates

• Remove certain 

sources

• Deduplicate based 

on Title similarity

• Deduplicate based 

on Text similarity

• Deduplicate based 

on Entity similarity
Relevant Articles

• Train article 

classification model 

based on labeled 

articles

• Update article 

classification model 

as coders label

• Evaluate model on 

randomly sampled 

articles
Unique Events

With a web interface:

• A team of coders 

labels articles 

meeting the 

definition of ARD 

• Articles about the 

same event are 

linked

• Coders enter details 

about incidents

• QC identifies data 

issues
Model Updating
Results

https://www.bjs.gov/content/pub/pdf/ardprs1516pf.pdf

Average number of 

arrest-related deaths 

identified per month in 

2011 through the 

previous ARD program 

methodology
Thanks to...

Bureau of Justice Statistics

Duren Banks

Erin Kennedy

Jason Nance

Paul Ruddle
Thank you
Appendix
Behind the scenes...
Text Similarity Detector

1,200,000  515,000  245,000 articles/month

Source: NewsOne

Source: Washington Post
Web Front End