WSDMS : Debunk Fake News via Weakly Supervised Detection of Misinforming Sentences with Contextualized Social Wisdom (2023)

Abstract: Fake news debunking primarily focuses on determining the truthfulness of news articles, which oversimplifies the issue as fake news often combines elements of both truth and falsehood. Thus, it becomes crucial to identify specific instances of misinformation within the articles. In this research, we investigate a novel task in the field of fake news debunking, which involves detecting sentence-level misinformation. One of the major challenges in this task is the absence of a training dataset with sentence-level annotations regarding veracity. Inspired by the Multiple Instance Learning (MIL) approach, we propose a model called Weakly Supervised Detection of Misinforming Sentences (WSDMS). This model only requires bag-level labels for training but is capable of inferring both sentence-level misinformation and article-level veracity, aided by relevant social media conversations that are attentively contextualized with news sentences. We evaluate WSDMS on three real-world benchmarks and demonstrate that it outperforms existing state-of-the-art baselines in debunking fake news at both the sentence and article levels.

## introduction misinformation, such as fake news, poses tremendous risks and threats to contemporary society. the detection of fake news entails various technical challenges @xcite , and one of them is accurately identifying false elements within news articles. this challenge arises due to the blending of authentic and fabricated content by creators * jing ma is the corresponding author. fake news detection aims to determine the veracity of a given news article @xcite . previous analysis has revealed that users often share comments and provide evidence about fake news on social media platforms @xcite , which has led to a growing stream of research that leverages these social engagements, along with the content of news articles, to aid in fake news detection @xcite @xcite . this approach bears analogies to rumor detection, where the focus is on assessing as a specific statement rather than an entire news article @xcite @xcite @xcite @xcite . many studies in this domain aims to train supervised classifiers using features extracted from the social context and the content of the claim or article. however, the existing fake news detection models predominately focus on coarse-level classification of the entire article, which oversimplifies the problem. misinformation can be strategically embedded within an article by manipulating portions of its content to enhance its credibility @xcite @xcite ) therefore, we target a fine-grained task that aims to identify sentences containing misinformation within an article, which can be jointly learned with article-level fake news detection. figure 1 shows an illustrative example of a fake news article titled "nasa will pay 100,000 usd news title: nasa will pay you 100,000 usd to stay in bed for 60 days! ## related work early studies on fake news detection have attempted to exploit various approaches to extract features from news content and social context information, including linguistic features @xcite , visual clues @xcite , temporal traits @xcite , user behaviors and profiles @xcite @xcite . subsequent studies have employed neural networks to automatically learn deep feature representations from similar sources of data @xcite @xcite @xcite . furthermore, researchers have incorporated external knowledge sources @xcite @xcite and combined multi-modal data @xcite @xcite @xcite to enhance learning and improve fake news detection performance. notably, social context information has played a crucial role in debunking fake news and rumors @xcite @xcite @xcite . the utilization of social context structures has spurred the development of graph neural networks (gnns) such as kernel graph attention networks (kgat) @xcite and graphaware co-attention networks (gcan) @xcite , which have demonstrated effectiveness in various fake news-related tasks. however, existing approaches @xcite @xcite generally aim to detect article-level fake news, which lack the capability to tell which specific sentences contain misinformation. mil is a weakly supervised approach that infers instance-level labels (e.g., sentence or pixel) when training data is annotated with bag-level labels (e.g., document or image) @xcite . several mil variants have been developed based on threshold-based mil assumption @xcite and weighted collective mil assumption @xcite , successfully applied in various downstream tasks such as recommendation systems @xcite sentiment analysis @xcite , keywords extraction @xcite , community question answering @xcite , and more recently joint detection of stances and rumors @xcite . we adopt the weighted collective mil assumption @xcite to incorporate a weight function over the sentence space to calculate the article veracity probability. this assumption allows us to achieve a more robust prediction, as it avoids bias introduced by less important instances. ## problem definition we define a fake news dataset as a set of news articles {a}, where each article consists of a set of n sentences @xmath0. for each article, we assume there is a set of m social conversation trees relevant to it denoted as @xmath1, where t j is the j-th conversation tree containing posts (i.e., nodes) and message propagation paths (i.e., edges) which can provide the social context information for a. our task is to predict the veracity of information at both sentence level and article level in a unified model: ## wsdms: our mil-based model detecting more nuanced instances of misinformation at the sentence level solely based on article content is challenging @xcite . previous studies have demonstrated that social media posts contain valuable opinions, conjectures, and evidence that can be leveraged to debunk claimlevel misinformation, such as rumors @xcite @xcite , where claims, typically presented as short sentences, share similar characteristics with sentences in news articles. we hypothesize that the detection of misinforming sentences can be done by incorporating relevant information from social context associated with the article. we try to establish connections between social conversations and specific news sentences in the article, enabling the contextualization of social wisdom to enrich the representation of sentences, in order to better capture the veracity of sentences. the architecture of our mil-based weakly supervised model wsdms is illustrated in figure 2 . wsdms consists of four closely coupled components: input embedding, sentence and conversation tree linking, misinforming sentence detection, and article veracity prediction. we describe them with detail in this section. ## conclusion and future work we propose a mil-based model called wsdms to debunk fake news in a finer-grained manner via weakly supervised detection of misinforming sentences with only article veracity labels for model training. wsdms uses the attention mechanism to associate news sentences with their relevant social news conversations to identify misinforming sentences and determine the article's veracity by aggregating sentence-level predictions. wsdms outperforms a set of strong baselines at the article level and sentence level on three datasets. in the future, we will incorporate more intersentence features, such as discourse relations, to detect composition-level misinformation.