Generative Entity-to-Entity Stance Detection with Knowledge Graph Augmentation (2022)

Abstract: Stance detection is typically framed as predicting the sentiment in a given text towards a target entity. However, this setup overlooks the importance of the source entity, i.e., who is expressing the opinion. In this paper, we emphasize the imperative need for studying interactions among entities when inferring stances. We first introduce a new task, entity-to-entity (E2E) stance detection, which primes models to identify entities in their canonical names and discern stances jointly. To support this study, we curate a new dataset with 10,641 annotations labeled at the sentence level from news articles of different ideological leanings. We present a novel generative framework to allow the generation of canonical names for entities as well as stances among them. We further enhance the model with a graph encoder to summarize entity activities and external knowledge surrounding the entities. Experiments show that our model outperforms strong comparisons by large margins. Further analyses demonstrate the usefulness of E2E stance detection for understanding media quotation and stance landscape as well as inferring entity ideology.

## introduction news media often employ ideological language to sway their readers, including criticizing entities they disagree with, and praising those conforming to their values @xcite . however, in many cases, the sources do not directly express their sentiments, in part due to the norm that "objective" news media should restrict their role to narrating events and quoting others. in the realm of political news, many reported events consist of individuals or groups who themselves are engaged in praise or blame. the seemingly neutral act of choosing who to quote, and about what, as illustrated in fig. 1 , may be shaped by ideology and have significant effects on readers @xcite . trump's rhetoric, including calling central americans trying to enter the united states "an invasion," and his hard-line immigration policies have exposed him to condemnation since the el paso shooting. "how far is it from trump's saying this 'is an invasion' to the shooter in el paso declaring 'his attack is a response to the hispanic invasion of texas?' not far at all," biden was due to say, according to an advance copy of his speech. "in both clear language and in code, this president has fanned the flames of white supremacy in this nation." there thus exists a pressing need to examine these expressions of support and opposition in news articles @xcite in order to understand how even apparently nonpartisan media can bias readers via the selective inclusion of stances among entities. recognizing stances among political entities and events is also important in its own right: if copartisans are more likely to be positive towards each other and vice versa for counter-partisans, this allows us to (1) propagate partisanship and ideology through the signed network @xcite , (2) infer ideology not just for politicians, but for events or objects (e.g., a new bill) that may inherently support the positions of specific groups @xcite , and (3) illuminate the implicit ideology of a journalist or media outlet @xcite . as a first step towards these goals, this paper presents the first study on solving the task of entityto-entity (e2e) stance detection in an end-to-end fashion: given a target sentence and its surrounding context, we extract a sequence of stance triplets that can be inferred from the input. each triplet consists of a source entity and their sentiment towards a target entity, with entities in their canonical forms. existing stance detection methods are largely designed to infer an author's overall sentiment towards a given entity @xcite or topic @xcite from a text. e2e stance detection, by contrast, presents a number of new challenges. first, entities can be involved in multiple and even conflicting sentiments within a sentence, foot_0 as demonstrated in fig. 1 , suggesting the need to develop a model that can disentangle entity interactions. second, entities are mentioned in various forms, e.g., full names or pronouns. simply extracting the mentions would cause ambiguity for downstream applications. canonical names that can be identified via knowledge bases @xcite are thus preferred, which further requires the model to consider contextual information and global knowledge. in this work, we first collect and annotate an e2e stance dataset, seesaw 2 (stance between entity and entity supplemented with article-level viewpoint), based on 609 news articles crawled from allsides. 3 seesaw contains 10,619 stance triplets annotated at the sentence level, drawn from 203 political news stories, with each "story" consisting of 3 articles by media of different ideological leanings, as collected, coded, and aligned by allsides. our entities cover people, organizations, events, topics, and other objects. we then present a novel encoder-decoder generative framework to output stance triplets in order. we first enhance the text encoder with a graph model @xcite encoding a semantic graph that summarizes global entity interactions in the context, using relations extracted by an open information extraction (openie) system @xcite . on the decoder side, we improve the transformer decoder block @xcite with a task-customized joint attention mechanism to combine textual and graph representations. finally, external knowledge, such as party affiliation or employment, is injected into the graph encoder by adding extra nodes initialized with pretrained representations from wikipedia. this allows the system to better characterize entity relations. we conduct experiments on the newly collected seesaw to evaluate models' capability of generating stance triplets, and additionally evaluate on a task of stance-only prediction when pairs of entities are given. our model outperforms competitive baselines on e2e stance detection by at least 21% (relatively, accuracy of 11.32 vs. 13.74), demonstrating the effectiveness of adding knowledge from context and wikipedia via graph encoding. our best model also outperforms its pipeline counterpart which first extracts entities and then detects sentiment. this highlights the end-to-end prediction capability of generative models. finally, we demonstrate the usefulness of e2e stances for media stance characterization and entity-level ideology prediction. notably, we find that 1) both leftand right-leaning media tend to quote more from the republican politicians; and 2) there appears a symmetrical asymmetry in expressed stances: the left is balanced while the right is biased in terms of expressed positivity, but the reverse holds for negativity. ## seesaw collection and annotation annotation process. we use allsides news stories collected by @xcite , where each story contains 1-3 articles on the same event but reported by media of different ideology. the stories span from 2012 to 2021. we only keep news stories with 3 articles and that are pertinent to u.s. politics. we manually inspect and select stories to cover diverse topics. the resulting seesaw contains 52 topics (full list in table a1 ). we further clean articles by removing boilerplate and noisy text. we hired six college students with high english proficiency to conduct annotation tasks. each person is asked to read all articles written on the same story before annotating. we summarize the main steps below, with detailed protocol in appendix a. 1. they start with reading the article, and then identify entities of political interests that are involved in sentiment expressions. an entity can have a type of person, organization, place, event, topic, or religion. annotated entities are categorized into main and salient entities. foot_2 2. for each sentence, stance is annotated between entities in the triplet format, i.e., <source, sentiment, @xmath0. we use author as the source entity, if no clear source entity is found. also, someone is used to replace source or target entities of no clear political interest, e.g., "a neighbor". we then convert the annotations on sentiment and ideological leaning into 3-way and 5-way, respectively. finally, we conduct cross-document entity resolution by linking annotations to their canonical names according to wikipedia, e.g., "this president" becomes "donald trump" as in fig. 1 . a quality control procedure is designed to allow annotators to receive timely feedback and improve agreement over time. details are in appendix c. importantly, over 60 randomly sampled articles, the average overlap of annotated entities between pairs of coders is 55.5%. when both source and target entities match, the sentiment agreement level is 97%, indicating the high quality of the dataset. statistics. seesaw includes 609 news articles from 203 stories, published by 24 different media outlets (9 left, 6 central, and 9 right). table a2 lists all the media outlets. on average, each article contains 28 sentences and 647 words. 44% of sentences per article have at least one stance annotation, among which 29% have at least two. in total, there are 10,619 stance annotations in seesaw, covering 1, 757 distinct entities. 62.4% of the triplets have negative sentiment. entity types in seesaw cover people (49.6%), organization (12.8%), place (4.2%), event (12.0%), topic (17.4%), religion (0.2%), and others (3.8%), showing its diversity of entity annotations. it is worth noting that the source entity @xmath1.1% and 12.5% of the annotations. meanwhile, the number for target entity @xmath2.3%. in terms of entity ideology, the portions of entities annotated as liberal, moderate, and conservative is 31.0%, 15.9%, and 34.6%, respectively. 5 our annotated article leanings align with allsides' media-level labels for 76.7% of the time. ## generative e2e stance detection fig. 2 depicts the end-to-end generative framework that reads in a document and produces stance triplets in an auto-regressive fashion, by leveraging multiple knowledge sources using graph augmentation. we use bart @xcite , a large pretrained encode-decoder model, as the backbone. taking as input a target sentence from a document d, our model first constructs a semantic graph g ( §4.1), which is encoded via the graph encoder ( §4.2). the contextualized representations of tokens and nodes are denoted as h t and h g . stance triplets are generated by our decoder using improved in-parallel attention and information fusion mechanisms ( §4.3). moreover, we inject wikipedia knowledge to support the identification of relations between entities ( §4.5). below we describe the main components, with additional formulation and implementation details in appendix b. text encoder document entities graph encoder e 1 : medicaid e 2 : republican … … entity linking e 3 p 1 e 2 p 2 e 1 lm head self attn cross attn cross attn ffnn information fuse × m h g h t z t z g z … <@xmath3... z' <@xmath4, in the format of "@xmath5...". besides token and position embeddings, we add a third embedding to indicate a token's type: preceding context, target text, succeeding context, or entities. our decoder implements in-parallel crossattention ( §4.3) to attend both text (h t ) and node (h g ) representations concurrently. fused representations are obtained through the information fusion layer. ## further analysis in seesaw, we are able to identify the partisanship of 204 politicians (democrat vs. republican) based on voteview. 9 this subset accounts for more than 60% of person mentions in the dataset. we further include democrat and republican as two separate entities, since they are also frequently mentioned in news. analyses in this section are done based on this entity set (henceforth analysis set). ## conclusion we present and investigate a novel task: entityto-entity (e2e) stance detection, with the goal of extracting a sequence of stance triplets from a target sentence by jointly identifying entities in their canonical names and discerning stances among them. to support this study, we annotate a new dataset, seesaw, with 10,619 sentence-level annotations. we propose a novel end-to-end generative framework to output stance triplets. specifically, we enhance standard encoder-decoder models with a semantic graph to capture entity interactions within context. we further augment our model with external knowledge learned from wikipedia, yielding the best overall performance. we conduct further analyses to demonstrate the effectiveness of e2e stances on media landscape characterization and entity ideology prediction.