FACTIFY 3 M : A benchmark for multimodal fact verification with explainability through 5 W Question-Answering (2023)

Abstract: Combating disinformation is one of the burning societal crises - about 67% of the American population believes that disinformation produces a lot of uncertainty, and 10% of them knowingly propagate disinformation. Evidence shows that disinformation can manipulate democratic processes and public opinion, causing disruption in the share market, panic and anxiety in society, and even death during crises. Therefore, disinformation should be identified promptly and, if possible, mitigated. With approximately 3.2 billion images and 720,000 hours of video shared online daily on social media platforms, scalable detection of multimodal disinformation requires efficient fact verification. Despite progress in automatic text-based fact verification (e.g., FEVER, LIAR), the research community lacks substantial effort in multimodal fact verification. To address this gap, we introduce FACTIFY 3M, a dataset of 3 million samples that pushes the boundaries of the domain of fact verification via a multimodal fake news dataset, in addition to offering explainability through the concept of 5W question-answering. Salient features of the dataset include: (i) textual claims, (ii) ChatGPT-generated paraphrased claims, (iii) associated images, (iv) stable diffusion-generated additional images (i.e., visual paraphrases), (v) pixel-level image heatmap to foster image-text explainability of the claim, (vi) 5W QA pairs, and (vii) adversarial fake news stories.

## factify 3m: an illustration we introduce factify 3m (3 million), the largest dataset and benchmark for multimodal fact verification. figure 1 : a tweet referring to the sports personality and known aids victim magic johnson with a photo that was taken a decade before covid and, moreover, is not real evidence of blood donation. consider the example in fig. 1 . a widely distributed image of the sports legend magic johnson with an iv line in his arm was accompanied by the claim that he was donating blood, implicitly during the covid-19 pandemic. if true, this is troubling because magic johnson is a well-known victim of aids and is prohibited from donating blood. the picture predated the covid-19 epidemic by a decade and is related to his treatment for aids. textual claim: the text associated with the claim in fig. 1 purports that the tweet's author took this photo and assisted magic johnson in donating promptfake3m at a glance entailment classes textual support visual/image support no. of claims no. of paraphrased claims no. of images no. of stable diffusion generated images 5wqa pairs no. of evidence documents adversarial opt-generated news story support_multimodal texts are supporting each other ∼similar news images are supporting each other 232,000 882,000 232,000 927,000 858,400 232,000 support support_text texts are supporting each other ∼similar news images are neither supporting nor refuting 174,000 609,000 169,000 661,000 852,600 174,000 ## related works: data sources and compilation automatic fact verification has received significant attention in recent times. several datasets are available for text-based fact verification, e.g., fever @xcite , snopes (vo and lee, 2020), politifact (vo and lee, 2020), faviq @xcite , hover @xcite , x-fact @xcite , creak @xcite , feverous @xcite , etc. multimodal fact verification has recently started gaining momentum. defactify workshop series at aaai 2022 @xcite and 2023 @xcite has released factify 1.0 @xcite and 2.0 @xcite with 50k annotated data each year, which we have embedded as well as part of factify 3m. fact verification datasets are mainly classified into three major categories: (i) support, (ii) neutral, and (iii) refute. while it is relatively easier to collect data for support and neutral categories, collecting largescale refute category fake news claims is relatively challenging. fever @xcite proposed an alternative via manual imaginary claim generation, but is complex, lacks scalability, and may generate something unrealistic. therefore, we decided to merge available datasets, at least for the refute category. it is wise to have all these datasets into one, and further, we have generated associated images using stable diffusion. while selecting datasets, we only chose datasets with evidence claim documents as we are interested in 5w qa-based explanation. other datasets only with fake claims were discarded for this reason. furthermore, we use opt @xcite to generate adversarial fake news documents of text based on the refute claims as prompts. we have adopted an automatic method to compile a list of claims for support and neutral categories. it is often seen that the same event gets reported by two different media houses separately on the same day -therefore, one can be treated as support for the other. with this strategy in mind, we have collected and annotated large-scale data automatically (cf. appendix a for details). fig. 6 visualizes how much data from each dataset are compiled. figure 6 : distribution of our dataset delineating its constituent components. there are only a handful of other previous works, @xcite , @xcite , @xcite , @xcite , @xcite , @xcite , that have discussed multimodal fact verification. none of them generated large resources like ours and did not discuss qa-based explanation, heatmap-based image explainability, and adversarial assertion. ## paraphrasing textual claims a claim may have multiple diverse manifestations depending on the style and manner in which it was reported. specifically, the textual component (i.e., prose) may have variations as highlighted in fig. 5a . we seek to echo such heterogeneity to ensure the real-world applicability of our benchmark (cf. examples in fig. 3 and more examples in the appendix o). manual generation of possible paraphrases is undoubtedly ideal but is timeconsuming and labor-intensive. on the other hand, automatic paraphrasing has received significant attention in recent times @xcite @xcite @xcite . our criteria for selecting the most appropriate paraphrasing model was the linguistic correctness of the paraphrased output and the number of paraphrase variations. to achieve this, we propose the following processlet's say we have a claim c, we generate a set of paraphrases of c. textual paraphrase detection is a well-studied paradigm, there are much state-ofthe-art (sota) systems @xcite @xcite . we pick the best model available mostly trained based on the resources available from snli @xcite . next, we use the entailment model @xcite to choose the right paraphrase candidate from the generated set, by doing a pairwise entailment check and choosing only examples which exhibit entailment with c. we empirically validated the performance of (a) pegasus @xcite , (b) t5 (flan-t5-xxl variant) @xcite , and (c) gpt-3.5 (gpt-3.5-turbo-0301 variant) @xcite ) models for our use-case and found that gpt-3.5 outperformed the rest (appendix c for details on evaluation along three dimensions: (i) coverage, (ii) correctness, and (iii) diversity). ## visual paraphrase: stable diffusion-based image synthesis while textual diversity in claims seen in the wild is commonplace, typically the visual components -particularly, images -also show diversity. the concept of ai-based text-to-image generators has been around for the past several years, but their outputs were rudimentary up until recently. in the past year, text prompt-based image generation has emerged in the form of dall-e @xcite , imagegen @xcite , and stable diffusion @xcite . while these new-age systems are significantly more powerful, they are a double-edged sword. they have shown tremendous potential in practical applications but also come with their fair share of unintended use cases. one major caution is the inadvertent misuse of such powerful systems. to further this point, we have utilized stable diffusion 2.0 @xcite to generate a large amount of fake news data. stable diffusion @xcite is a powerful, open-source text-to-image generation model. the model is not only extremely capable of generating high-quality, accurate images to a given prompt, but this process is also far less computationally expensive than other text-conditional image synthesis approaches such as @xcite @xcite . stable diffusion works on stabilizing the latent diffusion process which has an aspect of randomization, as a result, it generates a different result each time. moreover, quality control is a challenge. we have generated 5 images for a given claim and then further ranked them, discussed in the next section (cf. appendix n for examples). ## automatic 5w qa pair generation a false claim is very likely to have some truth in it, some correct information. in fact, most fake news articles are challenging to detect precisely because they are mostly based on correct information, deviating from the facts only in a few aspects. that is, the misinformation in the claim comes from a very specific inaccurate statement. so, given our textual claim and image claim, we generate 5w claim: 'after april 11, 2020, there was a fatality rate of over 1.61 in malaysia during the coronavirus pandemic' srl model who what when why where answer: 'after april 11, 2020' we leverage the method of using 5w srl to generate qa pairs @xcite and verify each aspect separately to detect 'exactly where the lie lies'. this, in turn, provides an explanation of why a particular claim is refutable since we can identify exactly which part of the claim is false. ## injecting adversarial assertions the rise of generative ai techniques with capabilities that mimic a human's creative thought process has led to the availability of extraordinary skills at the masses' fingertips. this has led to the proliferation of generated content, which is virtually indistinguishable as real or fake to the human eye, even for experts in some cases. this poses unparalleled challenges to machines in assessing the veracity of such content. as one of the novelties of our work, we address this by introducing synthetically generated adversarial fake news documents for all the refute claims using opt @xcite , a large language model. in doing so, we attempt to confuse the fact verification system by injecting fake examples acting as an adversarial attack. to draw a parallel in a real-world scenario, this could mean the proliferation of such fake news articles online via social media, blog posts, etc. which would eventually lead to a fact-verification system being unable to make a concrete decision on the trustworthiness of the news. such a scenario would lend itself as a natural manifestation of an adversarial attack by virtue (rather, the "vice") of the fake news articles confusing the fact verification system. we analyze the impact on the performance of our fact verification system in ## experiments: baselines & performance in this section, we present baselines for: (i) multimodal entailment, (ii) 5w qa-based validation, and (iii) results of our models after adversarial injections of generated fake news stories. 7.1 multimodal entailment: support or refute? in this paper, we model the task of detecting multimodal fake news as multimodal entailment. we assume that each data point contains a reliable source of information, called document, and its associated image and another source whose validity must be assessed, called the claim which also contains a respective image. the goal is to identify if the claim entails the document. since we are interested in a multimodal scenario with both image and text, entailment has two verticals, namely textual entailment, and visual entailment, and their respective combinations. this data format is a stepping stone for the fact-checking problem where we have one reliable source of news and want to identify the fake/real claims given a large set of multimodal claims. therefore the task essentially is: given a textual claim, claim image, text document, and document image, the system has to classify the data sample into one of the five categories: support_text, support_multimodal, insufficient_text, insufficient_multimodal, and refute. using the google cloud vision api (google, 2022), we also perform ocr to obtain the text embedded in images and utilize that as additional input. text-only model: fig. 16 shows our text-only model, which adopts a siamese architecture focussing only on the textual aspect of the data and ignores the visual information. to this end, we generate sentence embeddings of the claim and document attributes using a pretrained mpnet sentence bert model @xcite (specifically the all-mpnet-base-v2 variant). next, we measure the cosine similarity using the generated embeddings. the score, thus generated, is used as the only feature for the dataset, and classification is evaluated based on their f1 scores. multimodal model: information shared online is very often of multimodal nature. images can change the context of a textual claim (and vice versa) and lead to misinformation. as such, to holistically glean information from the available data, it is important that we consider both the visual and textual context when classifying the claims. our multimodal architecture (fig. 8 ), adopts a siamese architecture and utilizes both modalities. as we utilize an entailment-based approach, features from both the claim and document imagetext pairs must be extracted. to this end, we utilize the pretrained mpnet sentence bert model @xcite (specifically the all-mpnet-base-v2 variant) as our text embedding extractor and a pretrained vision transformer (vit) model @xcite (specifically the vit-base-patch16-224-in21k variant) as our vision embedding extractor. the cosine similarity score is computed between both the claim and document image features. furthermore, we also compute the cosine similarity for the text embeddings, similar to the text-only model. ## conclusion and future avenues we are introducing factify 3m, the largest dataset and benchmark for multimodal fact verification. we hope that our dataset facilitates research on multimodal fact verification on several aspects -(i) visual qa-based explanation of facts, (ii) how to handle adversarial attacks for fact verifications, (iii) whether generated images can be detected, and (iv) 5w qa-based help journalists to fact verify easily for complex facts. factify 3m will be made public and open for research purposes.