To Protect and To Serve? Analyzing Entity-Centric Framing of Police Violence (2021)

Abstract: Framing has significant but subtle effects on public opinion and policy. We propose an NLP framework to measure entity-centric frames. We use it to understand media coverage on police violence in the United States in a new Police Violence Frames Corpus of 82k news articles spanning 7k police killings. Our work uncovers more than a dozen framing devices and reveals significant differences in the way liberal and conservative news sources frame both the issue of police violence and the entities involved. Conservative sources emphasize when the victim is armed or attacking an officer and are more likely to mention the victim’s criminal record. Liberal sources focus more on the underlying systemic injustice, highlighting the victim’s race and that they were unarmed. We discover temporary spikes in these injustice frames near high-profile shooting events, and finally, we show protest volume correlates with and precedes media framing decisions.

## introduction the normative standard in american journalism is for the news to be neutral and objective, especially regarding politically charged events @xcite . despite this expectation, journalists are unable to report on all of an event's details simultaneously. by choosing to include or exclude details, or by highlighting salient details in a particular order, journalists unavoidably induce a preferred interpretation among readers @xcite . this selective presentation is called framing @xcite . framing influences the way people think by "telling them what to think about" @xcite . in this way, frames impact both public opinion @xcite @xcite @xcite 1 data and code available at: https://github.com/ gt-salt/framing-police-violence figure 1 : framing the murder of jordan edwards. our system automatically identifies key details or frames that shape a reader's understanding of the shooting. importantly, we can distinguish the victim's attributes from the descriptions of the officer, like killer in "killer cop." only the left-leaning article uses this morally-weighted term, killer, and also takes care to mention the victim's race. while the left-leaning article highlights a quote from the edwards' pastor, an unofficial source, the right-center article cites only official sources (namely the chief of police). both mention the victim's age and unarmed status. @xcite and policy decisions @xcite . prior work has revealed an abundance of politically effective framing devices @xcite @xcite @xcite , some of which have been operationalized and measured at scale using methods from nlp @xcite @xcite @xcite . while these works extensively cover issue frames in broad topics of political debate (e.g. immigration), they overlook a wide array of entity frames (how an individual is represented; e.g. a particular undocumented worker described as lazy), and these can have huge policy implications for target populations @xcite . in this paper, we introduce an nlp framework to understand entity framing and its relation to issue framing in political news. as a case study, we consider news coverage of police violence. though we choose this domain for the stark contrast between two readily-discernible entities (police and victim), our framing measures can also be applied to other major social issues @xcite , and salient entities involved in these events, like protesters, politicians, migrants, etc. we make several novel contributions. first, we introduce the police violence frame cor-pus that contains 82k news articles on over 7k police shooting incidents. see figure 1 for example articles with annotated frames. next, we build a set of syntax-aware methods for extracting 15 issue and entity frames, implemented using entity co-reference and the syntactic dependency parse. unlike bag-of-words methods (e.g. topic modeling) our entity-centric methods can distinguish between a white man and a white car. in this example, we identify race frames by scanning the attributive and predicative adjectives of any vic-tim tokens. such distinctions can be crucial, especially in a domain where officer aggression will have different ramifications than aggression from a suspected criminal. by exact string-matching, we can also extract, for the first time, differences in the order that frames appear within each document. we find that liberal sources discuss race and systemic racism much earlier, which can prime readers to interpret all other frames through the lens of injustice. furthermore, we quantify and statistically confirm what smaller-scale content analyses in the social sciences have previously shown @xcite @xcite , that conservative sources highlight law-and-order and focus on the victim's criminal record or their harm or resistance towards the officer, which could justify police conduct. finally, we rigorously examine the broader interactions between media framing and offline events. we find that high-profile shootings are correlated with an increase in systemic and racial framing, and that increased protest activity granger-causes or precedes media attention towards the victim's race and unarmed status. ## related work a large body of related work in nlp focuses on detecting stance, ideology, or political leaning @xcite @xcite @xcite @xcite . while we show a relationship between framing and political leaning, we argue that frames are often more subtle than overt expressions of stance, and cognitively more salient than other stylistic differences in the language of political actors, thus more challenging to be measured. specifically, we distinguish between issue frames @xcite and entity frames (van den @xcite . entity frames are descriptions of individuals that can shape a reader's ideas about a broader issue. the entity frames of interest here are the victim's age, gender, race, criminality, mental illness, and attacking/fleeing/unarmed status. one related work found a shooter identity cluster in their topic model that contained broad descriptors like "crazy" @xcite . however, their bag-of-words method would not differentiate a crazy shooter from a crazy situation. to follow up, there is need for a syntax-aware analysis. in a systematic study of issue framing, @xcite applied the policy frames codebook of @xcite to build the media frames corpus (mfc). they annotated spans of text from discussions on tobacco, same-sex marriage, and immigration policy with broad meta-topic framing labels like health and safety. @xcite built lexicons from the mfc annotations to classify issue frames in russian news, and @xcite extended this work with subframe lexicons to refine the broad categories of the mfc. some have considered the way moral foundations @xcite can serve as issue frames @xcite @xcite , and others have built issue-specific typologies @xcite . while issue framing has been well-studied @xcite , entity framing remains under-examined in nlp with a few exceptions. one line of work used an unsupervised approach to identify personas or clusters of co-occurring verbs and adjective modifiers @xcite @xcite . another line of work combined psychology lexicons with distributional methods to measure implicit differences in power, sentiment, and agency attributed to male and female entities in news and film @xcite @xcite . social scientists have experimentally manipulated framing devices related to police violence, including law and order, police brutality, or racial stereotypes, revealing dramatic effects on participants' perceptions of police shootings @xcite @xcite . criminologists have trained coders to manually annotate on the order of 100 news articles for framing devices relevant to police use of force @xcite . while these studies provide great theoretical insight, their manual coding schemes and small corpora are not suited for large scale real-time analysis of news reports nationwide. while many recent works in nlp have started to detect police shootings @xcite and other gun violence @xcite , we are the first to model entity-centric framing around police violence. ## police violence frame corpus to study media framing of police violence, we introduce police violence frame corpus (pvfc) which contains over 82,000 news reports of police shooting events. we now describe the corpus construction as it is shown in figure 2 . ## media frames extraction we are interested in both the issue and entity frames that structure public narratives on police violence. we will now present our computational framework for extracting both from news text. throughout this section, we cite numerous prior works from criminology and sociology to motivate our taxonomy, but we are the first to measure these frames computationally. as a preview of the system, figure 1 shows the key frames extracted from two articles on the murder of 15-year-old jordan edwards. notably, only the left-leaning article mentions the victim's race. most importantly, our system distinguishes the victim's attributes from descriptions of the officer. here, it is the officer who is described as a "killer," and not the victim. ## validating frame extraction methods one coder labeled 50 randomly-sampled news articles with indices to mark the order of frames present. against this ground truth, our binary frame extraction system achieves high precision and recall scores above 70%, with only race and unofficial sources at 66% and 65% precision. accuracy is no less than 70% for any frame (see ## the broader scope of media framing prior works have algorithmically tracked collective attention in the news cycle @xcite , and measured the correlation between media attention and offline political action @xcite @xcite . this section examines the broader scope of media framing via two studies. ## discussion and conclusion in this work, we present new tools for measuring entity-centric media framing, introduce the po-lice violence frame corpus, and use them to understand media coverage on police violence in the united states. our work uncovers 15 domainrelevant framing devices and reveals significant differences in the way liberal and conservative news sources frame both the issue of police violence and the entities involved. we also show that framing strategies co-evolve, and that protest activity precedes or anticipates crucial media framing decisions. we should carefully consider the limitations of this work and the potential for bias. since we matched age, gender and race directly with the mpv, we expect minimal bias, but acknowledge that our exact string-matching methods will miss context clues (e.g. drinking age), imprecise referents (e.g. "teenager"), and circumlocution. we also rely on lexicons derived from expert sources (e.g. u.s. courts 2021) or from data (empath), both of which are inherently incomplete. even the most straightforward keywords (e.g. armed, fleeing) are prone to error. however, biases could also appear in discriminative text classifiers. the advantage of our approach is that it is interpretable and extractive, allowing us to identify matched spans of text and quantify differences in frame ordering. furthermore, it is grounded heavy in the social science literature. similar methods could be applied to other major issues such as climate change @xcite and immigration @xcite , where entities include politicians, protesters, and minorities, and where race, mental illness, and unarmed status may all be salient framing devices (e.g. describing the perpetrator or victim of anti-asian abuse or violence; gover et al. 2020; chiang 2020; @xcite .