Context in Informational Bias Detection (2020)

Abstract: Informational bias is bias conveyed through sentences or clauses that provide tangential, speculative or background information that can sway readers’ opinions towards entities. By nature, informational bias is context-dependent, but previous work on informational bias detection has not explored the role of context beyond the sentence. In this paper, we explore four kinds of context for informational bias in English news articles: neighboring sentences, the full article, articles on the same event from other news publishers, and articles from the same domain (but potentially different events). We find that integrating event context improves classification performance over a very strong baseline. In addition, we perform the first error analysis of models on this task. We find that the best-performing context-inclusive model outperforms the baseline on longer sentences, and sentences from politically centrist articles.

## introduction informational bias is conveyed through sentences or clauses that provide tangential, speculative or background information that can sway readers' opinions towards entities @xcite . a natural place to look for informational bias is in news texts, where journalists use background information to place newsworthy events in a broader context. examples of informational bias include quotations of opinions from third parties about the target entity, allusions to what may have motivated the target entity to act as they did, and mentions of previous statements and actions of the same entity. what separates informational bias from other kinds of bias is that it can be expressed in a completely factual and neutral way. while some instances of bias are recognisable outside of their context (e.g. quotations from third parties that contain opinions), others are mere statements of facts that do not raise suspicions of bias outside of their context. consider example instance 1.3 in while previous work has performed informational bias detection on a token level and a sentence level, we are the first to involve context beyond the sentence. we integrate four kinds of context: neighboring sentences (direct textual context), the full article (article context), articles on the same event (event context) and articles from the same domain (domain context). 1 our model for leveraging event context significantly improves over the baseline. in an error analysis, we find that the best-performing context- idx sentence inf src id 1.1 former arkansas gov. mike huckabee announced tuesday he is running for president [. . . ]. 0 fox 53fox00 1.2 mr. huckabee opposes same-sex marriage, suggesting as recently as february that homosexuality is a lifestyle choice akin to drinking or swearing. 1 nyt 53nyt10 1.3 he was the longest-serving arkansas governor, from 1996 to 2007. 1 hpo 53hpo15 2.1 "trump says he wants to run the nation like he's run his business," mr. bloomberg said on wednesday. 0 nyt 82nyt01 2.2 bloomberg contrasted his business history with trump's, saying "i've built a business, and i didn't start it with a million-dollar check from my father." 1 fox 82fox05 2.3 "but trump's business plan is a disaster in the making." 1 hpo 82hpo21 3.1 the states of nebraska and oklahoma filed a federal lawsuit in the u.s. supreme court thursday [. . . ]. 0 hpo 21hpo00 3.2 "while colorado reaps millions from the sale of pot, nebraska taxpayers have to bear the cost." 1 fox 21fox09 3.3 [sheriff adam hayward of deuel county, neb.] has complained that marijuana arrests have strained his jail budget. ## nyt 21nyt09 our contributions are: framing. informational bias can be considered a type of framing with a focus on entities. the framing of entities has been studied for the construction of the english basil corpus of lexical and informational bias @xcite , which is the corpus our models are tested on. @xcite emphasize that, unlike more commonly studied kinds of bias, informational bias label assignments depend very heavily on context. the sentence-level and span-level basil annotations were thus provided by human annotators who saw sentences in their article context. however, the computational models in @xcite , which are based on bert @xcite , only treat sentences in isolation. framing of entities is also studied by @xcite , who examined how english-speaking news frames events through casts of characters, and van den @xcite , who studied the effect of naming and titling on the perception of entities in english and german. most framing research focuses not on entities but on the framing of topics and events. the study of topic framing in news has a long history in social science @xcite @xcite and has begun to attract attention from the natural language processing community @xcite @xcite . for topic framing research there exists the media frames corpus of news annotated for the framing of same-sex marriage, smoking, and immigration @xcite and the gun violence frame corpus @xcite annotated for framing in news on gun violence. computational analysis and classification experiments have been done on framing in russian news @xcite , on detecting frames in english headlines @xcite , and on detecting frames in a multilabel, multi-lingual setting @xcite . ## method we experiment with different kinds of context to assess which one or which ones are helpful for informational bias detection. we define four types of context: direct textual context, article context, event context and domain context. ## conclusion we explore the impact of including four kinds of context in informational bias detection. we integrate direct textual context, article context, context from other articles on the same event (event context), and domain context into sentence classification methods and test performance on the basil corpus of informational bias. we find that direct textual context and domain context are difficult to integrate in a way that boosts performance beyond the strong roberta baseline. our proposed context-inclusive model, however, outperforms roberta signficantly when using event context (evcim). error analysis shows that evcim performs better than the baseline on longer sentences, and sentences from politically centrist articles. furthermore, both models perform better on basil's fox news instances than its new york times or huffington post instances, and both models are better at recognising bias in quotes. future work could explore domain-adaptation to unlabeled data from the same population of articles that the basil corpus was drawn from. given the differences in performance on fox news articles compared to other sources, domain-adaptation to specific sources is also a promising avenue. in addition, future datasets may need to ensure a balance of sources that represent different layers and sections of society. future work could also extend context-inclusion experiments to token classification.