Improving Fake News Detection of Influential Domain via Domain- and Instance-Level Transfer (2022)

Abstract: Social media spreads both real news and fake news in various domains including politics, health, entertainment, etc. It is crucial to automatically detect fake news, especially for news of influential domains like politics and health because they may lead to serious social impact, e.g., panic in the COVID-19 pandemic. Some studies indicate the correlation between domains and perform multi-domain fake news detection. However, these multi-domain methods suffer from a seesaw problem that the performance of some domains is often improved by hurting the performance of other domains, which could lead to an unsatisfying performance in the specific target domains. To address this issue, we propose a Domain- and Instance-level Transfer Framework for Fake News Detection (DITFEND), which could improve the performance of specific target domains. To transfer coarse-grained domain-level knowledge, we train a general model with data of all domains from the meta-learning perspective. To transfer fine-grained instance-level knowledge and adapt the general model to a target domain, a language model is trained on the target domain to evaluate the transferability of each data instance in source domains and re-weight the instance’s contribution. Experiments on two real-world datasets demonstrate the effectiveness of DITFEND. According to both offline and online experiments, the DITFEND shows superior effectiveness for fake news detection.

## introduction with the rapid popularization of the internet, more and more people tend to acquire news through social media platforms, such as weibo 1 and twitter 2 . due to the above phenomenon, fake news has spread widely all over the world. during the 2016 u.s. presidential election campaign, the top 20 frequently discussed fake election stories generated 8,711,000 engagements on facebook, which is larger than the total of 7,367,000 for the top 20 most-discussed election stories posted by 19 major news websites @xcite . the wide spread of fake news may break the authenticity balance of the news ecosystem @xcite , and it not only misleads many people but also leads to the social mobs and social panic @xcite . therefore, fake news detection is of significant importance. real-world news platforms categorize news pieces by topic into various domains, e.g., politics, health, and entertainment. although it is necessary to detect fake news in every domain, the societal effects of fake news from some specific domains are more serious. for example, during the u.s. election, fake news in the political domain may have dictated election results @xcite , and @xcite find that false political news travels deeper more quickly and more broadly, reaches more people, and is more vital than other categories of false information. in the covid-19 infodemic @xcite , thousands of fake news pieces have caused social panic @xcite ) and weakened the effect of pandemic countermeasures @xcite . therefore, it is important to detect fake news for these specific news domains, e.g., politics @xcite , health @xcite . some researchers pay attention to fake news detection in the influential news domains by modeling each domain separately @xcite @xcite @xcite @xcite . however, a single-domain dataset could only contain limited data, e.g., the politifact dataset of the political domain only has 948 samples @xcite . fortunately, news pieces in different domains are correlated @xcite , so it is promising to exploit data of other domains to improve performance for a target domain. recently, some works @xcite @xcite simultaneously model various domains to improve the overall performance of all domains. however, these multidomain methods suffer from a serious seesaw phenomenon which could cause the performance of some domains to be improved at the cost of hurting the performance of other domains @xcite . for example, eann @xcite performs quite well in military (with f1-score of 0.9274), whereas the results is not satisfactory in politics (with f1-score of 0.8705) @xcite . moreover, it is hard to guide these multi-domain models to improve the performance of a specific target domain due to the lack of a target-oriented design. in this paper, we focus on exploiting news pieces of other domains to improve the detection performance of a certain target domain, called cross-domain fake news detection, which can bring additional gains for target domains compared to multi-domain methods. the differences among single-domain, multi-domain, and cross-domain fake news detection are shown in figure 1 . to solve cross-domain fake news detection, we adopt two key ideas: transfer domain-level knowledge. it is necessary to transfer knowledge from multiple domains because news pieces in different domains are correlated. moreover, to alleviate the seesaw phenomenon, improving the generalization ability of a multi-domain model, which can adapt fast to the target domain is necessary. transfer instance-level knowledge. the transferability varies from instance to instance. for example, "a politician claimed covid-19 is less lethal than flu" and "new york officials welcome immigrants, legal or illegal" are two different news pieces of politics. the former is more relevant to health while the latter is irrelevant. in other words, the former is more transferable. therefore, it is important to quantify the transferability of source instances, in order to decrease the impact of irrelevant instances. along this line, we propose a domain-and instance-level transfer framework for fake news detection (ditfend). to transfer domain-level knowledge, we exploit data from multiple domains to train a general model from the meta-learning perspective, which contains common knowledge and can adapt fast to a specific domain. to transfer instance-level knowledge, we first learn a domainadaptive language model which is endowed with characteristics of the target domain. to weigh the contribution to the target domain of every instance in source domains, we adopt an index, perplexity of the domain-adaptive language model, to quantify the transferability of these instances. because in information theory perplexity can measure how well a probability model predicts an instance, in other words, low perplexity on a news piece indicates that the instance is highly related to the knowledge contained in the language model. finally, we adapt the general model with instances from the target domain and weighted instances from source domains, in order to achieve satisfying performance on the target domain. the main contributions of this paper can be summarized as follows: ## experiments in this section, we aim to answer the following evaluation questions: