Cross-lingual Evidence Improves Monolingual Fake News Detection (2021)

Abstract: Misleading information spreads on the Internet at an incredible speed, which can lead to irreparable consequences in some cases. Therefore, it is becoming essential to develop fake news detection technologies. While substantial work has been done in this direction, one of the limitations of the current approaches is that these models are focused only on one language and do not use multilingual information. In this work, we propose a new technique based on cross-lingual evidence (CE) that can be used for fake news detection and improve existing approaches. The hypothesis of the usage of cross-lingual evidence as a feature for fake news detection is confirmed, firstly, by manual experiment based on a set of known true and fake news. Besides, we compared our fake news classification system based on the proposed feature with several strong baselines on two multi-domain datasets of general-topic news and one newly fake COVID-19 news dataset showing that combining cross-lingual evidence with strong baselines such as RoBERTa yields significant improvements in fake news detection.

## introduction after the manipulation of opinions on facebook during the 2016 u.s. election @xcite , the interest in the topic of fake news has increased substantially. unfortunately, the distribution of fakes leads not only to misinformation of readers but also to more severe consequences such as shooting in washington pizzeria @xcite that was caused by the spreading of fake news about hillary clinton leading a child sex trafficking. also, due to the global pandemic in 2020, there was a simultaneous emergence of infodemic @xcite that could lead to an even worse epidemiological situation and harm people's health dramatically. as a result, fake news received tremendous public attention, as well as drawn increasing interest from the academic community. multiple supervised fake news detection models were proposed based on linguistic features @xcite ; deep learning models @xcite @xcite ; or signals from social networks @xcite . one of the directions of the supervised approaches is to use additional information from the web @xcite @xcite . however, in these works only monolingual signals were taken into account. in our work, we assume that viral spreading of (fake) information may naturally hit the "language barrier" and cross-checking of facts across media in various languages (supposed to be strongly independent) could yield an additional signal. we aim to close this gap and perform an exploration of cross-lingual web features to fake news detection. the contribution of our work is a new crosslingual evidence feature for fake news detection based on multilingual news verification. 1 we conduct a manual experiment based on cross-lingual dataset markup to evaluate if the user can use such a feature for misinformation identification. after that, we implement the proposed feature showing that adding cross-lingual evidence consistently improves the results of strong baselines including large pre-trained transformers. we release publicly all code and data. 2 2 related work firstly, several datasets have been collected for different sub-tasks of fake news detection pipeline: seeking information via some search engine to find evidence is a quite natural feature motivated by real users' behaviour. several studies tried to figure out how users authenticate the information from the web. jr. et al. @xcite showed that individuals rely on both their judgment of the source and the message, and when this does not adequately provide a definitive answer, they turn to external resources to authenticate news. the intentional and institutional reaction was seeking confirmation from institutional sources (some respondents answered simply "google"). moreover, participants that received messages across different media plat-forms @xcite and different perspectives of the information @xcite showed greater awareness about news evidence. consequently, the information from the external search is an important feature for news authenticity evaluation and evidence seeking. while the idea of multilingualism was already explored for hate speech (aluru et al., 2020) and rumors @xcite detection, however, previous works did not fully use multilingual information of fake news detection. in our study, we explore fake news spread on the web for different languages and extend evidence retrieval to cross-lingual news verification. ## detection of fake news using cross-lingual evidence (ce) our approach is based on the following hypothesis: if the news is true, then it will be widespread in different languages and also across media with different biases, and the facts mentioned should be identical. on the other hand, if it is fake news, it will receive a lesser response in the foreign press than true news. the step-by-step process, schematically represented in figure 1 , is as follows: step 1. text extraction: as a new article arrives, title and content are extracted from it. ## experiment 1: manual verification first, we conducted a manual experiment on a small dataset to test the hypothesis in "ideal conditions". ## experiment 2: automatic verification we implemented cross-lingual evidence (ce) feature, as described below. we tested its performance on fake news detection on three multi-domain datasets comparing it with strong baselines. ## conclusion we presented an approach for fake news detection based on cross-lingual evidence (ce) which provides a different perspective on the event across languages verified in two experiments. a fake news classification model with ce significantly improves performance over various baselines and compares favorably to sota. besides, the ce is interpretable as a user can check in which and how many languages a piece of given news was found. a promising direction to explore is to increase the number of languages used for cross-lingual information retrieval. in addition to this, the general distribution of news in the world should be taken into account -for instance, us news tend to be covered in european presses more than european news are covered in the us press. also, in our work the language of original news was english. the analogous experiments for other original languages of news should be conducted. original news: lottery winner arrested for dumping @xmath0200,000 of manure on exboss' lawn - https: //worldnews dailyreport. com/lotterywinnerarrested-fordumping-200000-ofmanure-onex-bosslawn/ a man from illinois was arrested for getting @xmath1125 million at the lottery and quit his job. 54-year old brian morris, from the small town of clarendon hills in dupage county, bought over 20,000 tons of manure and asked for it to be dumped on his former boss' property, pretending it was his residence. dozens of trucks filled with manure showed up in front of the house around 6:00 this morning and began dumping their smelly cargo over the property's lawn. george fitzgerald, mr. morris' former employer, was awakened by the sound of the vehicles on his property and rapidly called the police. unfortunately, it took the police more than 15 minutes to arrive on the site, and more than 10,000 tons of manure had already been dumped in the meantime. brian morris was standing right across the street and laughing when the police arrived, and he rapidly came over to confess his responsibility and explain his motivations. lieutenant frank meyers, a spokesman of the clarendon hills police department, met the press a few hours later to explain the motivations behind this strange crime. englsih query https://www.google.com/search ? cd_min:1/1/2018,cd_max: 1/1/2019&@xmath4,000 of manure on ex-boss' lawn https://worldnewsdailyreport.com/lottery-winner-arrested-fordumping-200000-of-manure-on-ex-boss-lawn/ title title in en link text of the content content in en do you think it supports original news? answer: 1 (support), 0 (refute), -1 (not enough info) any comments 0 lottery winner arrested for dumping @xmath3224,000 worth of manure dumped on his former employer's property, only two weeks after he won $125 million at the lottery and quit his job. 54-year old brian morris, from the small town of clarendon hills in dupage county, bought over 20,000 tons of manure and asked for it to be dumped on his former boss' property, pretending it was his residence. dozens of trucks filled with manure showed up in front of the house around 6:00 this morning and began dumping their smelly cargo over the property's lawn. george fitzgerald, mr. morris' former employer, was awakened by the sound of the vehicles on his property and rapidly called the police. unfortunately, it took the police more than 15 minutes to arrive on the site, and more than 10,000 tons of manure had already been dumped in the meantime. brian morris was standing right across the street and laughing when the police arrived, and he rapidly came over to confess his responsibility and explain his motivations. lieutenant frank meyers, a spokesman of the clarendon hills police department, met the press a few hours later to explain the motivations behind this strange crime.