Hierarchical Multi-head Attentive Network for Evidence-aware Fake News Detection (2021)

Abstract: The widespread of fake news and misinformation in various domains ranging from politics, economics to public health has posed an urgent need to automatically fact-check information. A recent trend in fake news detection is to utilize evidence from external sources. However, existing evidence-aware fake news detection methods focused on either only word-level attention or evidence-level attention, which may result in suboptimal performance. In this paper, we propose a Hierarchical Multi-head Attentive Network to fact-check textual claims. Our model jointly combines multi-head word-level attention and multi-head document-level attention, which aid explanation in both word-level and evidence-level. Experiments on two real-word datasets show that our model outperforms seven state-of-the-art baselines. Improvements over baselines are from 6% to 18%. Our source code and datasets are released at https://github.com/nguyenvo09/EACL2021 .

## introduction the proliferation of biased news, misleading claims, disinformation and fake news has caused heightened negative effects on modern society in various domains ranging from politics, economics to public health. a recent study showed that maliciously fabricated and partisan stories possibly caused citizens' misperception about political candidates @xcite during the 2016 u.s. presidential elections. in economics, the spread of fake news has manipulated stock price @xcite . for example, $139 billion was wiped out when the associated press (ap)'s hacked twitter account posted rumor about white house explosion with barack obama's injury. recently, misinformation has caused infodemics in public health @xcite and even led to people's fatalities in the physical world @xcite . to reduce the spread of misinformation and its detrimental influences, many fact-checking systems have been developed to fact-check textual claims. it is estimated that the number of factchecking outlets has increased 400% in 60 countries since 2014 @xcite . several factchecking systems such as snopes.com and politifact.com are widely used by both online users and major corporations. facebook (cnn, 2020) recently incorporated third-party fact-checking sites to social media posts and google integrated factchecking articles to their search engine @xcite . these fact-checking systems debunk claims by manually assess their credibility based on collected webpages used as evidence. however, this manual process is laborious and unscalable to handle the large volume of produced false claims on communication platforms. therefore, in this paper, our goal is to build an automatic fake news detection system to fact-check textual claims based on collected evidence to speed up fact-checking process of the above fact-checking sites. to detect fake news, researchers proposed to use linguistics and textual content @xcite @xcite . since textual claims are usually deliberately written to deceive readers, it is hard to detect fake news by solely relying on the content claims. therefore, multiple works utilized other signals such as temporal spreading patterns @xcite , network structures @xcite @xcite and users' feedbacks @xcite @xcite . however, limited work used external webpages as documents which could provide interpretive explanation to users. several recent work @xcite @xcite started to utilize documents to fact-check textual claims. @xcite used word-level attention in documents but treated all documents with equal im-portance whereas @xcite only focused on which documents are more crucial without considering what words help explain credibility of textual claims. observing drawbacks of the existing work, we propose hierarchical multi-head attentive network which jointly utilizes word attention and evidence attention. overall semantics of a document may be generated by multiple parts of the document. therefore, we propose a multi-head word attention mechanism to capture different semantic contributions of words to the meaning of the documents. since a document may have different semantic aspects corresponding to various information related to credibility of a claim, we propose a multi-head document-level attention mechanism to capture contributions of the different semantic aspects of the documents. in our attention mechanism, we also use speakers and publishers information to further improve effectiveness of our model. to our knowledge, our work is the first applying multi-head attention mechanism for both words and documents in evidence-aware fake news detection. our work makes the following contributions: ## related work many methods have been proposed to detect fake news in recent years. these methods can be placed into three groups: (1) human-based fact-checking sites (e.g. snopes.com, politifact.com), (2) machine learning based methods and (3) hybrid systems (e.g. content moderation on social media sites). in machine-learning-based methods, researchers mainly used linguistics and textual content @xcite @xcite , temporal spreading patterns @xcite , network structures @xcite @xcite , users' feedbacks @xcite and multimodal signals @xcite . recently, researchers focus on fact-checking claims based on evidence from different sources. @xcite and @xcite fact-check claims using subject-predicate-object triplets extracted from knowledge graph as evidence. chen et al. (2020) assess claims' credibility using tabular data. our work is closely related to fact verification task @xcite @xcite which aims to classify a pair of a claim and an evidence extracted from wikipedia into three classes: supported, refuted, or not enough info. for fact verification task, nie et al. (2019) used elmo (peters et al., 2018) to extract contextual embeddings of words and used a modified esim model (chen et al., 2017). soleimani et al. (2020) used bert model (devlin et al., 2018) to retrieve and verify claims. zhou et al. (2019) used graph based models for semantic reasoning. our work is different from these work since our goal is to classify a pair of a claim and a list of relevant evidence into true or false. our work is close to existing work about evidence-aware fake news detection @xcite @xcite . @xcite used an average pooling layer to derive claims' representation to attend to words in evidence, @xcite focused on words and sentences in each evidence, and @xcite proposed a semantic entailment model to attend to important evidence. however, to the best of our knowledge, our work is the first jointly using multi-head attention mechanisms to focus on important words in each evidence and important evidence from a set of relevant articles. our attention mechanism is different from these work since we use multiple attention heads to capture different semantic contributions of words and evidence. ## problem statement we denote an evidence-based fact-checking dataset c as a collection of tuples (c, s, d, p) where c is a textual claim originated from a speaker s, @xmath0. note, |d| = |p|. our goal is to classify each tuple (c, s, d, p) into a pre-defined class (i.e. true news/fake news). ## framework in this section, we describe our hierarchical multihead attentive network for fact-checking (mac) which jointly considers word-level attention and document-level attention. our framework consists of four main components: (1) embedding layer, (2) multi-head word attention layer, (3) multi-head document attention layer and (4) output layer. these components are illustrated in fig. 1 where we show a claim and two documents as an example. ## conclusions in this paper, we propose a novel evidence-aware model to fact-check textual claims. our mac is designed by hierarchically stacking two attention layers. the first one is a word attention layer and the second one is a document attention layer. in both layers, we propose multi-head attention mechanisms to capture different semantic contributions of words and documents. our mac outperforms the baselines significantly with an average increase of 6% to 9% over the best results from baselines with a maximum improvements of 18%. we conduct ablation studies to understand the performance of mac and provide a case study to show the effectiveness of the attention mechanisms. in future work, we will further examine other data types such as images to improve the performance of our model.