All Things Considered: Detecting Partisan Events from News Media with Cross-Article Comparison (2023)

Abstract: Public opinion is shaped by the information news media provide, and that information in turn may be shaped by the ideological preferences of media outlets. But while much attention has been devoted to media bias via overt ideological language or topic selection, a more unobtrusive way in which the media shape opinion is via the strategic inclusion or omission of \textit{partisan events} that may \textit{support} one side or the other. We develop a latent variable-based framework to predict the ideology of news articles by comparing multiple articles on the same story and identifying partisan events whose inclusion or omission reveals ideology. Our experiments first validate the existence of partisan event selection, and then show that article alignment and cross-document comparison detect partisan events and article ideology better than competitive baselines. Our results reveal the high-level form of media bias, which is present even among mainstream media with strong norms of objectivity and nonpartisanship. Our codebase and dataset are available at https://github.com/launchnlp/ATC.

## introduction news media play a critical role in society not merely by supplying information, but also by selecting and shaping the content they report @xcite @xcite . to understand how media bias affects media consumers @xcite , we must understand not just how media ideology affects the presentation of news stories on a surface level, such as the usage of partisan phrases or opinions, but also the less obvious process of content selection @xcite . content selection, such as what events that are related to the main story and should be included news story: biden pushes for gun legislation after visiting uvalde. the washington post (left): e1: [jaydien jaydien jaydien jaydien jaydien jaydien jaydien jaydien jaydien jaydien jaydien jaydien jaydien jaydien jaydien jaydien jaydien]arg0, . . ., [said said said said said said said said said said said said said said said said said]pred [he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: he asked the president: "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more "could you please make our schools safer and send more police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?" police, please?"]arg1 e2: @xcite arg0 . . . [noting]pred: "[you couldn't buy a cannon when the second amendment was passed]arg1." new york post (right): e1: [you]arg0 couldn't [buy]pred [a cannon]arg1 when the second amendment was passed. e2: biden has made that claim before, . . ., and they have been repeatedly [ :::::: declared declared declared declared declared declared declared declared declared declared declared declared declared declared declared declared declared]pred [ ::: false false false false false false false false false false false false false false false false false]arg1 [ :::::: by factby factby factby factby factby factby factby factby fact-by fact-by fact-by fact-by fact-by fact-by fact-by factby fact-:::::: checkers checkers checkers checkers checkers checkers checkers checkers checkers checkers checkers checkers checkers checkers checkers checkers checkers]arg0. in the report, has recently become a focus of study in political science. numerous studies point out that media selectively report information that is flattering to a particular political party or ideology, which may consequently shift audience beliefs and attitudes @xcite @xcite . however, most existing work either requires manual inspection of reported content @xcite , or relies on simple tools for coarse analyses, such as overall slant and topic emphasis @xcite . as a result, these studies are either limited to a short time period, or are unable to provide a detailed understanding of content selection bias. thus there remains a strong need for automatic tools that can analyze and detect how more complex content is selectively reported. rather than focusing on more superficial biases such as word, topic, or entity selection, we investigate here how media ideology affects their selection of which events to include for news reporting. events are the fundamental high-level components of the storytelling process @xcite , and their inclusion or omission shapes how a news story is perceived. in line with previous analysis of partisan selection bias in the literature @xcite , we define partisan events as selectively reported events that are favorable to a media organization's co-partisans or unfavorable to counter-partisans. when there are many potentially relevant events, which subset are included in an article fundamentally affects how readers interpret the story, and can reveal a media outlet's stance on that topic and their ideology @xcite @xcite . one example of event-selection bias is shown in fig. 1 , where a washington post article includes a survivor's request to impose gun control (pro-gun control), whereas a new york post article claims biden's statement as false (pro-gun rights). this paper has two major goals: (1) examining the relation between event selection and media ideology, and (2) formulating a task for partisan event detection in news articles and developing computational methods to automate the process. for the first goal, we verify the existence of partisan event selection by measuring how event selection affects the performance of media ideology prediction. specifically, we represent articles using triplets of 〈arg0, predicate, arg1〉, denoting the set of events they report with participating entities (e.g., in fig. 1 ). this representation is shown to be effective in narrative understanding @xcite . we conduct two studies. first, we compare article-level ideology prediction performance by using events within a single article vs. contextualizing them with events in other news articles on the same story but reported by media with different ideologies, inspired by the observation that biased content should be evaluated against other media @xcite . we show that the latter setup yields significantly higher f1 scores, suggesting that cross-article comparison can identify partisan events and thereby produce more accurate ideology prediction. second, we annotate an evaluation dataset of 50 articles that focus on two recent political issues, where in total we manually label 828 partisan events out of 1867 sentences from all articles. testing on this dataset, we show that removing partisan events from the articles hurts ideology prediction performance significantly more than removing similar amounts of randomly selected events. for the second goal, the most critical challenge in developing computational tools to identify partisan events is the lack of annotation, where manually labeling a large-scale dataset requires domain expertise and is highly time-consuming. for that reason, we use latent variables to represent whether an event is partisan or not, and propose to jointly infer partisan events and predict an article's ideology. our models are trained using article-level ideology labels only, which are easier to obtain, and they do not require any human annotation of partisan events. we compare two approaches @xcite to train latent variable models and explore two methods for further improvement: (1) steering the model toward events that are selected only by one side, which are more likely to be partisan, and (2) providing prior ideology knowledge with pretrained event representations. we conduct experiments on two existing news article datasets @xcite and our newly annotated data with partisan events (test only). results indicate that latent variable models outperform all competitive baselines on both partisan event detection and ideology prediction, where cross-article event comparison is shown to be critical for both tasks. analysis of the extracted partisan events reveals key challenges in detecting implicit nuanced sentiments and discerning event relations (e.g., main vs. background events), suggesting future research directions. to the best of our knowledge, this is the first time that computational methods are developed for studying media bias at the event selection level. it is also the first time that automatic models are investigated to detect partisan events. our results provide new insights into a high-level form of media bias that may be present even in apparently nonpartisan news, enabling a new understanding of how news media content is produced and shaped. while partisan coverage filtering has been studied in political science, detecting it requires human efforts to review all news content @xcite , making these methods unscalable and only applicable to short time periods. @xcite automate the process, but use predefined lists of phrases and simple topic models to determine the overall slant and topic of a news report, which cannot capture more tactful content selection like events. most recently, a contemporaneous work @xcite also explores the partisan events within news articles, but they mainly curate a larger-scaled annotated dataset to support fine-tuning models on the labeled events. compared to these works, we operate with more nuanced factual details than phrases and topics, and we treat partisan events as latent variables and automatically detect them from news articles with methods that are scalable to large quantities of news. another line of work that is similar to ours is the detection of informational bias @xcite , defined as "tangential, speculative, or background information that sways readers' opinions" @xcite . our work differs in two important aspects: first, their "informational bias" can occur in any text span, and detecting speculative information often requires complex inference and also depends on specific wording. by contrast, by focusing on the presence or absence of events, we target concrete units of potentially partisan information, which can be more easily validated and understood by readers. second, they train supervised models on annotated biased content, while our latent variable models do not need any labels on partisan events. ideology prediction with text. many computational models have been developed to predict ideology using textual data @xcite @xcite . recent work, for instance, leverages neural networks to incorporate phrase-level ideology @xcite , external knowledge from social media @xcite , and large-scale language model pretraining @xcite . however, most of this computational work focuses directly on ideology prediction, with little attention to the higher-level processes underlying media bias. in particular, ideology prediction may fail for many mainstream media outlets who eschew overtly ideological language, and instead may bias readers only via a more sophisticated information selection procedure at the event level. we demonstrate that incorporating story-level context enables global content comparison over political spectrum, and benefits both partisan event detection and ideology prediction. ## event selection effect study in this section, we verify the existence of partisan event selection by examining partisan events' influence on ideology prediction. using events extracted from articles, we design a model that predicts ideology with single-or multi-article context ( §3.1), based on the assumption that comparing events included by different media may reveal their ideological leanings. we then manually annotate a dataset with partisan events in news stories ( §3.2). using this dataset and two existing corpora, we show that cross-article content comparison can reveal potential partisan events and removing partisan events hurts ideology prediction ( §3.3). ## latent variable models for partisan event detection the general idea of our latent variable models for partisan event detection is that the detected partisan events should be indicative of article's ideology, the removal of which would lower models' prediction confidence, according to our study in §3. we adopt two methods that are originally developed to extract rationales of model predictions ( §4.2) for our task and further improve them by adding constraints on the usage of common events and adding prior knowledge of event-level ideology ( §4.3). ## experiments tasks and datasets. similar to §3.3, we train all models solely on allsides and evaluate on all-sides test set, basil, and our partisan event dataset (pevent). we measure ideology prediction performance on all three datasets and partisan event detection performance on pevent. evaluation metrics. for ideology prediction, we measure the macro f1 score at the article level. for partisan event detection, we measure the f1 score for the positive class, i.e., partisan event. baselines. we consider the following baselines: ## further analyses and discussions effect of varying k. we now explore the effect of k's values. we experiment with three multi-article models: base two-player, two-player with restriction and prior knowledge, and base three-player models. we train these models with @xmath0, 20, 30, 40, 50 and plot the performance of partisan event detection and ideology prediction on pevent in fig. 3 . for event detection, the model improves as k increases, but the improvement is moderate when @xmath1. for ideology prediction, the performance plateaus at @xmath2, which peaks at @xmath3. this suggests that only a subset of events reflect the article's ideology, and it is enough to make predictions based on them. error analysis. usage of the latent variable model and future directions. the latent variable model can be used as a stance analyzer, which would come in handy in practice as well, especially for generating rationales out-of-the-box in different settings. firstly, being trained as an ideology predictor, it has the potential for future extensions to multi-modal ideology analysis, as suggested in @xcite . secondly, working on articles on the same topics, it enables exploration of how different media outlets title: at the nra convention, people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shoo-people blame mass shootings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns tings on everything but guns the nation has been plunged into despair and mourning a party a party a party a party a party a party a party a party a party a party a party a party a party a party a party a party a party . . . two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled two messages emerged from the assembled throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 throngs and the doting politicians in attendance, just 300 miles from uvalde: 1) ::::: people people people people people people people people people people people people people people people people people must :::::::::::: continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy continue to enjoy :::::::::::::::::::::::::::::::::::: the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose the right to acquire any damn firearm they choose, without meddling from the state; and 2) :::::::::::: the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had the massacre had :::::::::::::::::::::::::::::::::::::::::: ## conclusion partisan event selection is an important form of media bias which may exist in even the most apparently nonpartisan news, but which is especially hard to detect without extensive cross-article comparison. we first verify the existence of partisan event selection by inspecting the impact of partisan events on the performance of ideology prediction. we then jointly detect partisan events and predict article's ideology using latent variable models. experiments show that detected partisan events reasonably align with human judgement, and our models using cross-article event context outperforms the counterpart that only uses single-article context on both two tasks. our analysis also suggests future directions for identifying interactions among entities in an article and for resolving event coreference across articles. title: biden calls for assault weapons ban, making gun manufacturers liable for shootings president biden on thursday made an emotional appeal for ambitious new gun laws, including a ban on military-style rifles . . . on the other side of the aisle, republicans republicans republicans republicans republicans republicans republicans republicans republicans republicans republicans republicans republicans republicans republicans republicans republicans bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-bristled at democrats' equating support for the second a-