Automatic Fake News Detection: Are Models Learning to Reason? (2021)

Abstract: Most fact checking models for automatic fake news detection are based on reasoning: given a claim with associated evidence, the models aim to estimate the claim veracity based on the supporting or refuting content within the evidence. When these models perform well, it is generally assumed to be due to the models having learned to reason over the evidence with regards to the claim. In this paper, we investigate this assumption of reasoning, by exploring the relationship and importance of both claim and evidence. Surprisingly, we find on political fact checking datasets that most often the highest effectiveness is obtained by utilizing only the evidence, as the impact of including the claim is either negligible or harmful to the effectiveness. This highlights an important problem in what constitutes evidence in existing approaches for automatic fake news detection.

## introduction misinformation is spreading at increasing rates @xcite , particularly online, and is considered a highly pressing issue by the world economic forum @xcite . to combat this problem, automatic fact checking, especially for estimating the veracity of potential fake news, have been extensively researched @xcite @xcite @xcite @xcite . given a claim, most fact checking systems are evidencebased, meaning they utilize external knowledge to determine the claim veracity. such external knowledge may consist of previously fact checked claims @xcite , but it typically consists of using the claim to query the web through a search api to retrieve relevant hits. while including the evidence in the model increases the effectiveness in this work we investigate if fact checking models learn reasoning, i.e., provided a claim and associated evidence, whether the model determines the claim veracity by reasoning over the evidence. if the model learns reasoning, we would expect the following proposition to hold: a model using both the claim and evidence should perform better on the task of fact checking compared to a model using only the claim or evidence. if a model is only given the claim as input, it does not necessarily have the information needed to determine the veracity. similarly, if the model is only given the evidence, the predictive signal must come from dataset bias or the differences in the evidence obtained from claims with varying veracity, as it otherwise corresponds to being able to provide an answer to an unknown question. in our experimental evaluation on two political fact checking datasets, across multiple types of claim and evidence representations, we find the evidence provides a very strong predictive signal independent of the claim, and that the best performance is most often obtained while entirely ignoring the claim. this highlights that fact checking models may not be learning to reason, but instead exploit an inherent signal in the evidence itself, which can be used to determine factuality independent of using the claim as part of the model input. this highlights an important problem in what constitutes evidence in existing approaches for automatic fake news detection. we make our code publicly available foot_0 . ## related work automatic fact checking models include deep learning approaches, based on contextual and noncontextual embeddings, which encode the claim and evidence using rnns or transformers @xcite @xcite @xcite , and non-deep learning approaches @xcite , which uses hand-crafted features or bag-of-word representations as input to traditional machine learning classifiers such as random forests, svm, and mlp @xcite @xcite . generally, models may learn to memorize artifact and biases rather than truly learning @xcite @xcite , e.g., from political individuals often leaning towards one side of the truth spectrum. additionally, language models have been shown to implicitly store world knowledge @xcite , which in principle could enhance the aforementioned biases. to this end, we design our experimental setup to include representative fact checking models of varying complexity (from simple term-frequency based representations to contextual embeddings), while always evaluating each trained model on multiple different datasets to determine generalizability. overview. in the following we describe the different models used for the experimental compari-son (section 4), which consists of models based on term frequency (term-frequency weighted bag-ofwords @xcite ), word embeddings (glove word embeddings @xcite ), and contextual word embeddings (bert @xcite ). these representations are chosen as to include the typical representations most broadly used among past and current nlp models. term-frequency based random forest. we construct a term-frequency weighted bag-of-words representation per sample based on concatenating the text content of the claim and associated evidence snippets. we train a random forest @xcite as the classifier using the gini impurity measure. in the setting of only using either the claim or evidence snippets as the input, only the relevant part is used for constructing the bag-of-words representation. glove-based lstm model. we adapt the model by @xcite , which originally was proposed for multi-domain veracity prediction. using a pretrained glove embedding @xcite foot_1 , claim and snippet tokens are embedded into a joint space. we encode the claim and snippets using an attention-weighted bidirectional lstm @xcite : lastly, the model is trained using cross entropy as the loss function. evidence is used, then an attention weighted sum of the evidence snippet encodings is used in eq. 5 instead of o i . ## tuning details in the following, the best overall parameter configurations are underlined. the best configuration is chosen based on the average of the micro and macro f1 foot_3 . for rf, we tune the number of trees from @xcite @xcite , the minimum number of samples in a leaf from @xcite @xcite , and the minimum number of samples per split from @xcite @xcite . for the lstm model, we tune the learning rate from [1e-4,5e-4,1e-5], batch size @xcite , number of lstm layers from [1,2], dropout from [0, 0.1], and fix the number of hidden dimensions to 128. for the bert model, we tune the learning rate from [3e-5, 3e-6, 3e-7] and fix the batch size to 8. ## conclusion we investigate if fact checking models for fake news detection are learning to process claim and evidence jointly in a way resembling reasoning. across models of varying complexity and evaluated on multiple datasets, we find that the best performance can most often be obtained using only the evidence. this highlights that models using both claim and evidence are inherently not learning to reason, and points to a potential problem in how evidence is currently obtained in existing approaches for automatic fake news detection.