DOI: 10.3886/icpsr38644.v1
title: Using Sentiment Analysis and Topic Modeling in Assessing the Impact of Police "Signaling" on Investigative and Prosecutorial Outcomes in Sexual Assault Reports, Cleveland, Ohio, 1993-2011
title: Version 1

description: The Cuyahoga County (Ohio, USA) Sexual Assault Kit (SAK) Initiative, led by the Cuyahoga County Prosecutor's Office, was launched in 2013 to test and follow up on previously untested sexual assault/rape kits that were collected as evidence from sexual assault victims. Rape reports typically include an incident report taken by the responding officer(s) who is tasked with gathering pertinent facts and evidence and then forwarding the report to an investigator (detective) for follow-up, as well as a summary of the investigative activity on the case as noted by the investigator, which can include the decision of the assigned prosecutor to file or not file charges.
Signaling is defined as information conveyed by responding officers in the narratives of police reports regarding a victim's creditability and rape-myth adherence. The goal was to better understand if and how responding officers' written reports in sexual assault cases impact investigating officers' decision-making and how cases proceed (or fail to proceed) in the criminal justice process. The objective of the study was to explore the first step in the investigative process to elucidate facilitators and barriers to sexual assault cases reaching a successful disposition.
The research team employed text mining and machine learning methods using natural language processing and advanced statistical analyses to evaluate the narratives of 5,638 police reports of sexual assaults where victims had sexual assault kits collected in Cuyahoga County over the span of nearly two decades (primarily 1993 through 2011). These reports were analyzed using topic modeling and sentiment analysis. The team addressed three research questions:

To what extent did "sentiments" in the responding officers' narratives reveal positive or negative signaling of victims' credibility?
To what extent were the "topics" and sentiments in the responding officers' narratives different in cases with increased investigative activity compared to those with less?
To what extent were both the topics and sentiments in the responding officers' narratives different for cases that were successfully investigated and prosecuted compared to those that were not?
This collection includes a quantitative dataset (DS1) and a qualitative dataset (DS2). The report-level quantitative dataset contains calculated sentiment scores, categorical variables describing the incident and outcome, and demographic variables of the victim(s) and suspect(s) for all reports analyzed (n=5,639). The full text for all reports is available in a CSV file that can be merged with the main data file. The qualitative data is a subset of reports from the main dataset (n=18) with high, medium, and low sentiment scores that were manually coded by the research team.
description: The purpose of this study was to identify signaling in narratives of police officers' sexual assault reports that might have affected subsequent decision-making, case flow, and attrition.
description: The research team extracted Cleveland Division of Police rape reports in PDF format from the Cuyahoga County Prosecutor's Office electronic management database, which produced a list of 6,353 potential reports. Of the total reports that were eligible for extraction, 6,071 (95.6 percent) were extracted. Reasons for not extracting included duplication, erroneous labeling, technical issues with extraction, or if the report was associated with a very small number of victims who reported a large number of rapes determined to be unfounded.
Reports were then converted from PDF to text using pdfMiner and optical character recognition programs. A portion of files (n=320) at this stage required manual conversion via typing or using Dragon Naturally Speaking. These were files with parts of the investigative narrative missing, handwritten reports, or narratives that did not correctly convert to text. A total of 5,638 reports were eligible to proceed to quality control, where a research team member manually reviewed each text file for inaccuracies.
Researchers manually coded case outcomes based on the report's closing language and information from each report's discrete fields (e.g., dates, victim name, suspect name, location of incident). Textual analysis was limited to the incident reports taken by responding officers. The research team trained machine-learning methods to classify the police reports, for which sentiment scores were assigned. For context and validation, the team also conducted human-detected sentiment and thematic analyses on portions of the data. For each sentiment measure (subjectivity, polarity, and overall sentiment), two of the highest sentiment analysis scored reports were selected at random from a list of the top 20, the median, and from the bottom 20 for hand-coding.
description: <strong>Sentiment score:</strong> indicates how positive or negative the words comprising the narrative are (higher score = more positive words)
<strong>Polarity score:</strong> indicates how negative or positive the sentiment is in the text (higher score = more positive)
<strong>Subjectivity score:</strong> indicates the amount of personal opinion and factual information in the text (higher score = more personal opinion) 
<strong>Demographics:</strong> victim and suspect race/ethnicity (does not include a multi-racial option), gender, age
<strong>Case characteristics:</strong> if suspect was fully named, if suspect/victim criminal history was mentioned, if victim was not believed, if victim did not follow-up with investigators, year of report, number of victims, case outcome, offense type
description: Datasets: 
DS0: Study-Level Files
DS1: Incident Report Quantitative Data
DS2: Incident Report Qualitative Text Data
description: Sexual assault reports from individuals who had sexual assault kits collected in Cleveland, Ohio primarily from 1993 through 2011.
description: record abstractsUsers will note that the full report text in the quantitative data (DS1) has been partially de-identified. The research team performed this work prior to depositing at ICPSR.The Sexual Assault Kit Initiative's focus was on previously untested kits from 1993 through 2011. However, the data includes some reports prior to 1993 and after 2011. If an offender was linked to a sexual assault outside of the 1993-2011 time frame, the report was included with the untested kit's investigation and prosecution.