Two Heads Are Better Than One: Improving Fake News Video Detection by Correlating with Neighbors (2023)

Abstract: The prevalence of short video platforms has spawned a lot of fake news videos, which have stronger propagation ability than textual fake news. Thus, automatically detecting fake news videos has been an important countermeasure in practice. Previous works commonly verify each news video individually with multimodal information. Nevertheless, news videos from different perspectives regarding the same event are commonly posted together, which contain complementary or contradictory information and thus can be used to evaluate each other mutually. To this end, we introduce a new and practical paradigm, i.e., cross-sample fake news video detection, and propose a novel framework, Neighbor-Enhanced fakE news video Detection (NEED), which integrates the neighborhood relationship of new videos belonging to the same event. NEED can be readily combined with existing single-sample detectors and further enhance their performances with the proposed graph aggregation (GA) and debunking rectification (DR) modules. Specifically, given the feature representations obtained from single-sample detectors, GA aggregates the neighborhood information with the dynamic graph to enrich the features of independent samples. After that, DR explicitly leverages the relationship between debunking videos and fake news videos to refute the candidate videos via textual and visual consistency. Extensive experiments on the public benchmark demonstrate that NEED greatly improves the performance of both single-modal (up to 8.34% in accuracy) and multimodal (up to 4.97% in accuracy) base detectors.

## introduction the dissemination of fake news has become an important social issue which poses real-world threats to politics @xcite , finance (el-boghdady, 2013), public health (naeem and bhatti, 2020), etc. recently, the prevalence of short video platforms has spawned a lot of fake news videos, which are more convincing and easier to spread compared to textual fake news @xcite . the cyberspace administration of china reported that five of the seven core rumors circulating in the china eastern airlines crash incident originated from short video platforms (cyberspace administration of china, 2022). statistics from another study also reveal the powerful propagation of fake news videos, which reports that only 124 tiktok fake news about covid-19 gained more than 20 million views and 2 million likes, comments and shares, causing negative influences on millions of people @xcite . therefore, developing automatic detection techniques for fake news videos is urgent to mitigate their negative impact. in view of the practicality of fake news video detection, previous works @xcite @xcite @xcite leverage the heterogeneous multimodal information of an individual news video for corroboration. however, fake news is intentionally created to mislead consumers @xcite and thus the multimodal components show few abnormities after deliberate fabrication. in addition, fake news videos typically contain real news videos where only some frames or the textual description has been maliciously modified to alter their meanings @xcite . the above characteristics demonstrate that the deliberate fabrication and malicious modification are inconspicuous in a single video, leading to low effectiveness of independent detection by existing works. in real-world scenarios, when a news event emerges, multiple related videos from different perspectives are posted, including fake news, real news, and debunking videos. such news videos contain complementary or contradictory information, which can be used to evaluate each other mutually. as shown in figure 1 , on the one hand, the fake news video contains conflict information with the real one (i.e., different locations: "anhui" province v.s. "henan" province). furthermore, debunking videos also exist in some events, and can easily detect the corresponding fake news by providing fact-based refutations. in a newly released dataset @xcite based on short video platforms, 54% of events containing fake news videos also have debunking videos, but 39% of events with debunking videos still had fake news videos spread after the debunking videos were posted. to some extent, these statistics reveal the universality and insufficient utilization of debunking videos. based on the above observations, we conjecture that the relationship among videos of the same event can be modeled to enhance the fake news video detection and rectify the detection results via factual information. to this end, we introduce the new cross-sample paradigm for fake news video detection and propose a corresponding novel framework, neighbor-enhanced fake news video detection (need), which integrates the neighborhood relationship both explicitly and implicitly for better detection. need is a model-agnostic framework, which can easily incorporate various single-sample detectors to yield further improvement. thus, we first obtain the feature representa-debunking videos are videos that use factual evidence to refute widely circulated fake news, usually posted by experts. to compensate for the insufficient information in a single video, we organize the news videos in the same event in the form of graph to aggregate the neighborhood information (graph aggregation). specifically, we leverage the attention mechanism on the event graph @xcite to model the correlations between different nodes and dynamically aggregate these features. furthermore, as mentioned before, there exists explicit relation between debunking and fake news videos, i.e., refutations. consequently, debunking videos can be adopted to rectify the false negative predictions, spotting the "hidden" fake news videos (debunking rectification). specifically, we formulate a new inference task to discriminate whether the given debunking video can refute the given candidate video. for a given video pair, the refutations commonly exist in the textual descriptions of the same visual scenes, which inspires us to detect the textual conflict of the same visual representation. to fulfill the discrimination, we take the visual representations from the video copy detector to obtain visual consistency, and fuse it with the textual feature from the textual conflict detector via the attention mechanism. then the fusion feature is used to classify the refutation relationship between the debunking and candidate videos. given the proposed graph aggregation and debunking rectification modules, need can significantly improve the performance of base single-sample detectors trained with single-modal or multimodal data. our contributions are summarized as follows: derived from such a paradigm, we propose the need framework, which exploits the neighborhood relationship explicitly and implicitly to enhance the fake news video detection. to this end, we formulate a new multimodal inference task and propose a novel model that utilizes the consistency from both the textual and visual perspectives to identify whether the given debunking video can refute the given candidate video. • need is versatile and can be applied to various single-sample detectors. extensive experiments on the public benchmark demonstrate that need can yield significant improvement with both single-modal and multimodal base detectors. ## related work to defend against fake news, researchers are mainly devoted to two threads of techniques: fake news detection methods commonly use non-factual multimodal signals such as linguistic patterns @xcite , image quality @xcite , multimodal inconsistency @xcite , user response @xcite , and propagation structure @xcite , to classify the given news post as real or fake. with the prevalence of short video platforms, detecting fake news videos draws more attention in the community. recent works mainly leverage deep neural networks to extract the multimodal features and model the cross-modal correlations @xcite @xcite . for example, @xcite use the cross-attention transformer to fuse news content features of different modalities including text, keyframes, and audio, and use the self-attention transformer to fuse them with social context features including comments and user. however, existing works in fake news video detection identify each target news independently, without considering the neighborhood relationship in an event. in view of the practicality of the eventlevel process, @xcite construct a crossdocument knowledge graph and employ a heterogeneous graph neural network to detect misinformation. nonetheless, this work is performed on the synthetic dataset where each fake news document originates from a manipulated knowledge graph, which cannot be readily applied to real-world scenarios with unpredictable noises in information extraction. moreover, they only consider the implicit relation among news texts while ignoring the explicit refutations between debunking information and fake news. fact-checking methods commonly rely on retrieved relevant factual information from reliable sources such as wikipedia @xcite and webpages @xcite to judge the veracity of the given check-worthy claim @xcite . a recent thread is to determine whether a claim has been previously factchecked before retrieving evidence @xcite . this task is commonly framed as a ranking task, ranking fact-checking articles based on the similarities to the given claim. compared to textual fact-checking, multimodal verification is underexplored. @xcite treat the verification as a multimodal entailment task, where the model needs to classify the relationship between the given reliable document (text with associated image) and check-worthy claim (text with associated image). inspired by these works, the debunking rectification module in need focuses on rectifying the wrong predictions of previously fact-checked news videos by identifying the refutation relationship between the given debunking and candidate news video. in summary, fake new detection methods leverage non-factual patterns learned from large-scale data to give timely judgments for newly emerging events, while fact-checking techniques provide more reliable judgments benefiting from the factual information but only work for a part of events limited by the coverage of external sources. our work combines the merits of these two approaches: ## experiments in this section, we conduct experiments to evaluate the effectiveness of need. specifically, we aim to answer the following evaluation questions: ## conclusion we proposed a novel framework, namely need, to utilize the neighborhood relationship in the same event for fake news video detection. we designed the graph aggregation and debunking rectification modules to assist existing single-sample fake news video detectors. experiments show the effectiveness of need in boosting the performance of existing models. we also drew insights on how the graph aggregation and debunking rectification contribute to fake news video detection.