Automatic Fake News Detection: Are current models “fact-checking” or“gut-checking”? (2022)

Abstract: Automatic fake news detection models are ostensibly based on logic, where the truth of a claim made in a headline can be determined by supporting or refuting evidence found in a resulting web query. These models are believed to be reasoning in some way; however, it has been shown that these same results, or better, can be achieved without considering the claim at all – only the evidence. This implies that other signals are contained within the examined evidence, and could be based on manipulable factors such as emotion, sentiment, or part-of-speech (POS) frequencies, which are vulnerable to adversarial inputs. We neutralize some of these signals through multiple forms of both neural and non-neural pre-processing and style transfer, and find that this flattening of extraneous indicators can induce the models to actually require both claims and evidence to perform well. We conclude with the construction of a model using emotion vectors built off a lexicon and passed through an “emotional attention” mechanism to appropriately weight certain emotions. We provide quantifiable results that prove our hypothesis that manipulable features are being used for fact-checking.

## introduction recent events such as the last two u.s. presidential elections have been greatly affected by fake news, defined as "fabricated information that disseminates deceptive content, or grossly distort actual news reports, shared on social media platforms" @xcite . in fact, the world economic forum 2013 report designates massive digital misinformation as a major technological and geopolitical risk @xcite . as daily social media usage increases @xcite , manual fact-checking cannot keep up with this deluge of information. automatic fact-checking models are therefore a necessity, and most of them function using a system of claims and evidence @xcite . given a specific claim, the models use external knowledge as evidence. typically, a web search query is treated as the claim, and a subset of the top search results is treated as the evidence. there is an implicit assumption that the fact-checking models are reasoning in some way, using the evidence to confirm or refute the claim. recent research @xcite found this conclusion may be premature; current models can show improved performance when considering evidence alone, essentially fact-checking an unasked question. while this might seem reasonable given that the evidence is conditioned on the claims by the search engine, this can be exploited as illustrated in figure 1 , which shows that evidence returned using a ridiculous claim can still appear reasonable if we view the evidence alone without the claim. furthermore, textual entailment requires both a text and a hypothesis; if we have a result without a hypothesis, we are performing a different, unknown task. this finding indicates a problem with current automatic fake news detection, signaling that the models rely on features in the evidence typical to fake news, rather than using entailment. since most automated fact-checking research is primarily concerned with the accuracy of the results, rather than addressing how the results are achieved, we propose a novel investigation into these models and their evidence. we use a variety of pre-processing steps, including neural and non-neural ones, to attempt to reduce the affectations common in evidence: we also develop our own bert-based model as an extension of the emocred system (giachanou figure 1 : an example of why evidence alone does not suffice in identifying fake news, despite the evidence being conditioned on the claim as a search-engine query. although the returned evidence appearing reputable, it is clear that it has little relevance to deciding the veracity of the claim that "all canadians have eaten at least one bear." et al., 2019), adding an "emotional attention" layer to weight the most relevant emotional signals in a given evidence snippet. we make our code publicly available. 1 with each of these methods, we focus on scores where the models perform better using both the claims and the evidence combined, s c&e , rather than with the evidence alone, s e . going forward, we will refer to the difference between these dataset combinations as the delta of the pre-processing step, where @xmath0. a positive delta score indicates that the claim was useful and helped yield an increase in performance. since we are removing indicators that the current models rely on, some of the models perform worse at the task than they did previously. however, a surprising result is that many improved, and the need to consider the claim and the evidence together is a sign of using reasoning rather than manipulable indicators. under current fact-checking models, adversarial data can subvert these detectors. paraphrasing can be performed by inserting fictitious statements into otherwise truthful evidence with little effect on the model's output. for example, an article titled "is the gop losing walmart?", could have "walmart" substituted with "apple," and the predictions are nearly identical despite the news now being fictitious @xcite . 1 ## models the most common automatic fact-checking nlp models are based on term frequency, word embeddings, and contextualized word embeddings, using random forests, lstms, and bert (hassan et al., 2017). we limit our experimentation to the bert model, as it is the highest performing state-of-the-art model and was thoroughly tested in @xcite . this bert model with no pre-processing is our baseline model. for the style transfer model we use the styleformer model @xcite , a transformer-based seq2seq model. we also develop our own bert-based model using the emolexi and emoint implementation of the emocred system by adding an emotional attention layer to emphasize certain emotion representations for a given claim and its evidence @xcite . there is also a snippet attention layer at-tending to which evidence itself should be weighted most heavily for the given claim. 5 experiments 5.1 non-neural pre-processing our goal is to separate affect-based properties from factual content of the text. toward this, we run a large number of permutations of the following four simple pre-processing steps (see figure 4 in appendix b for results). these steps were chosen as they have been shown to facilitate affective tasks such as sentiment analysis, emotion classification, and sarcasm detection @xcite . in some cases we used a modified form -such as removing adverbs for pos pre-processing. ## results surprisingly, the four top-performing models with the snopes dataset include two non-neural models and two neural models. all four achieve greater f1 macro scores than the baseline bert model without pre-processing (see figure 2 ). pos and stop yield the biggest delta between s c&e vs. s e , followed by emoint and informal style transfer. however, emoint yields the highest f1 macro, followed by pos, informal, and stop. in politifact, none of the pre-processing steps achieve a delta greater than zero for s c&e versus s e . the combination of pos+stop steps come closest to parity, followed by emoint, then pos and stop. for the best f1 macro scores overall, emoattention's two forms (i.e., emoint and emolexi) were the two best, followed by stop and pos. all of these pre-processing steps achieve higher f1 macro scores than the baseline bert model. further, they yield better deltas for s c&e versus s e , implying that the model now requires the claims to reason. ## conclusion many pre-processing steps increase both the model's f1 scores and its need for claims and evidence, validating our hypothesis that signals in style and tone have become a crutch for factchecking models. rather than doing entailment, they are leveraging other signals -perhaps similar to sentiment analysis -and relying on a "gut feeling". emoattention generates our best predictions and deltas, confirming our suspicion that the models rely on emotionally charged style as a predictive feature. this is further narrowed to emotional intensity: the emoint intensity score-based model performs much better than its count-based counterpart emolexi. thus, evidence containing emotions associated with fake news will be considered more when scoring the claim. one surprising result is the effectiveness of the simple pos and stop pre-processing steps. pos only included nouns, verbs, and adjectives (i.e., a superset of stop). this could explain why it has the best delta between s c&e vs. s e . future research could investigate if stopwords, which are often discarded, actually contain signals such as anaphora: a repetitive rhetoric style which can affect nlp analyses @xcite . as an example, donald trump makes heavy use of anaphora in his 2017 inauguration speech: "together, we will make america strong again. we will make america wealthy again. we will make america proud again. we will make america safe again. and, yes, together, we will make america great again." @xcite by removing stopwords "we", "will" and "again", the model relies less on the text's rhetoric style and more on the entailment we are seeking. we propose further study on the effects of stop and pos, as well as experimenting with different emotional vectors and emoattention to make factchecking models more robust. automatic fake news detection remains a challenging problem, and unfortunately, current fact-checking models can be subverted by adversarial techniques that exploit emotionally charged writing.