Sentence-level Media Bias Analysis Informed by Discourse Structures (2022)

Abstract: As polarization continues to rise among both the public and the news media, increasing attention has been devoted to detecting media bias. Most recent work in the NLP community, however, identify bias at the level of individual articles. However, each article itself comprises multiple sentences, which vary in their ideological bias. In this paper, we aim to identify sentences within an article that can illuminate and explain the overall bias of the entire article. We show that understanding the discourse role of a sentence in telling a news story, as well as its relation with nearby sentences, can reveal the ideological leanings of an author even when the sentence itself appears merely neutral. In particular, we consider using a functional news discourse structure and PDTB discourse relations to inform bias sentence identification, and distill the auxiliary knowledge from the two types of discourse structure into our bias sentence identification system. Experimental results on benchmark datasets show that incorporating both the global functional discourse structure and local rhetorical discourse relations can effectively increase the recall of bias sentence identification by 8.27% - 8.62%, as well as increase the precision by 2.82% - 3.48%.

## introduction news media play a vast role not only by providing information, but also by selecting, packaging, and organizing the information to shape public opinions @xcite . multiple studies showed that the media outlets are becoming more partisan and polarized, with great potential to influence public's political stance @xcite , which presents the necessity to develop novel models to detect the media bias. most recent research work focus on detecting the media bias either at the level of media outlet 1 the code link: https://github.com/yuanyuanlei-nlp/ bias_sentence_discourse_emnlp_2022 @xcite , or at the level of individual articles @xcite @xcite . however, each article itself comprises multiple sentences, which vary in their ideological bias @xcite . in this paper, we focus on sentence-level media bias analysis to identify bias sentences that, as interpreted by @xcite @xcite , provide the supportive or background information to shift opinion in an ideological direction, though that may be done via selective inclusion or omission as well as overt ideological language. the identified bias sentences can illuminate and explain the overall bias of the entire article. this is a difficult task, however, considering that ideological bias tends to be implicit and subtle, and a bias sentence itself can appear merely neutral. while sophisticated semantic reasoning may be needed to determine if a sentence induces bias, we observe that understanding the discourse role of a sentence in news story telling can inform bias sentence identification. specifically, we observe that sentences describing the main news event are less likely to carry bias, in contrast, certain types of supportive contents are more likely to induce bias, such as sentences describing reactions of various parties toward the main event or main entities. this is related to the selective reporting problem in news production @xcite , where journalists select what to include from among many materials that are all relevant to the main event to some extent, and the selected content can reveal the organization or journalist's leaning and stance on the main event or main entities. in particular, we choose to incorporate discourse roles predicted by our recent system for news discourse profiling 2 (choubey and huang, 2021). the news discourse profiling task distinguishes three types of contents in a news article, main contents, context-informing contents and additional supportive contents, and labels each sentence with one of 2 the system link: https://github.com/prafulla77/ discoure_profiling_rl_emnlp21findings eight subtypes reflecting common discourse roles of a sentence in telling a news story. specifically, 1). main contents have two subtypes, main event (m1) and consequence (m2), and cover sentences that describe the main event and their immediate consequences which are often found inseparable from main events. 2). context-informing contents have two subtypes, previous event (c1) and current context (c2), and cover sentences that explain the context or cause of the main event, including recent events and general circumstances, and 3). additional supportive contents have four subtypes, describing past events that precede the main event in months and years (historical event (d1)) or unverifiable situations that are often fictional or personal accounts of incidents of an unknown person (anecdotal event (d2)), or opinionated contents including reactions from immediate participants, experts, known personalities as well as journalists or news sources (evaluation (d3)), except speculations and projected consequences that are labeled as expectation (d4). numerical analysis on two datasets ( in addition to global discourse roles in news story telling, we observe that local discourse relations with nearby sentences, the causal relation and the comparison relation in particular, can inform bias sentence identification as well. causal relation implication is commonly used in journalism to attribute responsibility @xcite . interestingly, we found that the strong contrast semantics indicated by a comparison discourse relation can influence readers' perceptions of the related events or entities. in the example of we design a knowledge distillation model @xcite to distill the auxiliary knowledge from the global functional discourse role predictor and local rhetorical discourse relation predictors into our bias sentence detection system. an extra distillation loss is designed for guiding the detection model to learn from both global and local discourse structure predictors, so that the system can learn to take both discourse structures into account for building sentence representations. specifically, a framework for response-based multiteacher knowledge distillation is implemented, in which the student model takes predicted probability from the teacher model as learning material and aims to mimic teacher's behavior, as well as digests and integrates knowledge from multiple teachers. experiments on two benchmark datasets @xcite show that the knowledge distilled from both the global functional structure teacher and the local rhetorical structure teacher can increase the bias sentence identification recall by 8.27% -8.62%" as well as the precision by 2.82% -3.48%, on top of a strong baseline system. ## the distillation model in this section, we will elaborate on the bias sentence detection model distilling two types of discourse in detail. the model takes a whole news article consisting of n sentences (s 1 , s 2 , . . . , s n ) as input, and outputs the predicted probability of each sentence containing bias (p bias 1 , p bias 2 , . . . , p bias n ). a framework of response-based multi-teacher knowledge distillation is designed, shown in figure 1 . response, which is the soft probability predicted from the teacher model, is used as the learning materials for the bias detection model. an extra distillation loss is designed to guide the model to mimic the discourse teachers' response, so that the sentence embedding can be updated with two types of discourse informed as auxiliary knowledge. ## limitations one major limitation is that we only experimented on english datasets. while both the global news discourse structure and local discourse relations we identified useful for bias detection in english news articles may also be useful for analyzing news articles written in other languages (more future work is needed to verify this), we acknowledge that it will not be easy to obtain the discourse structure teacher models our approach requires because there may not be relevant annotated corpora existing for other languages. to the best of our knowledge, the news discourse corpus does not have any non-english version yet, researchers have started to create pdtb-style discourse annotations for other languages, but the languages considered are still limited to several resource rich languages such as arabic, hindi and chinese. ## ethical considerations as our evaluation shows, the presented bias sentence detection system has not achieved a satisfactory level of performance and may make false bias predictions when deployed.