MEAN : Multi-head Entity Aware Attention Networkfor Political Perspective Detection in News Media (2021)

Abstract: The way information is generated and disseminated has changed dramatically over the last decade. Identifying the political perspective shaping the way events are discussed in the media becomes more important due to the sharp increase in the number of news outlets and articles. Previous approaches usually only leverage linguistic information. However, news articles attempt to maintain credibility and seem impartial. Therefore, bias is introduced in subtle ways, usually by emphasizing different aspects of the story. In this paper, we propose a novel framework that considers entities mentioned in news articles and external knowledge about them, capturing the bias with respect to those entities. We explore different ways to inject entity information into the text model. Experiments show that our proposed framework achieves significant improvements over the standard text models, and is capable of identifying the difference in news narratives with different perspectives.

## introduction the perspectives underlying the way information is conveyed to readers can prime them to take similar stances and shape their world view @xcite . given the highly polarized coverage of news events, recognizing these perspectives can help ensure that all point of view are represented by news aggregation services, and help avoid "information echo-chambers" in which only a single view point is represented. past work studying expression of bias in text has focused on lexical and syntactic representations of bias @xcite @xcite . expressions of bias can include the use of the passive voice (e.g., "mistakes were made"), or references to known ideological talking points and framing decisions @xcite @xcite @xcite (e.g., "pro-life" vs. "pro-choice"). however, bias in news media is often more nuanced, expressed through informational choices @xcite , which highlight different aspects of the news story, depending on the entity or relation being discussed. for example, consider the following articles, discussing the same news story from different perspectives. ## related work the problem of perspective identification is originally studied as a text classification task @xcite @xcite , in which a classifier is trained to differentiate between specific perspectives. other works use linguistic indicators of bias and expressions of implicit sentiment @xcite @xcite . recent work by @xcite aims to characterize content relevant for bias detection. unlike their work which relies on annotated spans of text, we aim to characterize this content without explicit supervision. in the recent semeval-2019, a hyperpartisan news article detection task was suggested 1 . many works attempt to solve this problem with deep learning models @xcite . we build on these works to help shape our 1 https://pan.webis.de/semeval19/semeval19-web/ text representation approach. several recent works also started to make use of concepts or entities appearing in text to get a better representation. @xcite treats the extracted concepts as pseudo words and appending them to the original word sequence which is then fed to a cnn. the kcnn @xcite model, used for news recommendation, concatenates entity embeddings with the respective word embeddings at each word position to enhance the input. we take a different approach, and instead learn a document representation with respect to each entity in the article. using auxiliary information to improve text model was studied recently. tang et al. proposes user-word composition vector model that modifies word embeddings given author representations in order to capture user-specific modification to word meanings. other works incorporate user and product information to compute attentions over different semantic levels in the context of sentiment classification of online review @xcite . in this work, we learn the entity embedding based on external knowledge source (i.e. wikipedia) or text, instead of including them in the training of bias prediction task. therefore, we are able to capture rich knowledge about entities from various sources. another series of work that is closely related to ours is aspect based sentiment analysis. it aims at determining the sentiment polarity of a text span in a specific aspect or toward a target in the text. many neural network based approaches have been proposed @xcite @xcite to incorporate the aspect term into the text model. recently, several works @xcite designed their model based on bert @xcite . unlike these works, we are not trying to determine the sentiment toward each entity mentioned in text. instead, we are interested in identifying the underlying political perspective through the angles of these entities. ## model the problem of political perspective detection in news media can be formalised as follows. given a news article d, where d consists of sentences s i , @xmath0, l], and each sentence s i consists of words w it , @xmath1, t ]. l and t are the number of sentences in d and number of words in s i respectively. the goal of this task is to predict the political perspective y of the document. given different datasets, this can either be a binary classification task, where @xmath2, 1} (hyperpartisan or not), or a multi-class classification problem, where @xmath3, 1, 2} (left, center, right). to inject knowledge about entities and relations, which would help solve the above classification problem, we first extract entities from the data corpus, and then learn knowledge representations for them using both external knowledge and the text corpus itself. in the second part, we describe how the learned aspect representations can be used in our multi-head entity aware attention network. the overall architecture of our model is shown in figure 1 . it includes two sequence encoders, one for word level and another for sentence level. our model learns a document representation with respect to each entity or relation in the document. the hidden states from an encoder are combined through a multi head entity-aware attention mechanism such that the generated sentence and document vectors will consider not only the context within the text but also the knowledge about the target entity (e.g. their political affiliation, or stance on controversial issues) or relation. we explain the acquisition of entity and relation knowledge representation and the structure of mean in details below. ## conclusion in this work, we propose an entity-centric framework for political perspective detection. entity and relation representations learnt from external knowledge source and text corpus are utilized to compute attention at both word and sentence levels. a document representation with respect to each aspect in the article is then generated for prediction. empirical experiments on two recent news article datasets show that our model achieve significantly better performance in bias detection comparing to traditional text models and other strategies of incorporating entity information. in fact, relations are highly dependent on individual entities. we intend to extend this work to learn better relation representations given entity embeddings based on description of entity interactions in text. moreover, we would like to weigh the importance of each aspect toward the overall perspective of an article instead of having all of them contribute equally to the final prediction.