FACTIFY3M: A benchmark for multimodal fact verification with explainability through 5W Question-Answering
Authors: Megha Chakraborty
Khushbu Pahwa
Anku Rani
Shreyas Chatterjee
Dwip Dalal
Harshit Dave
Ritvik G
Preethi Gurumurthy
Adarsh Ashok Mahor
Samahriti Mukherjee
Aditya Pakala
Ishan Paul
Janvita Reddy
Arghya Sarkar
Kinjal Sensharma
Aman Chadha
Amit P. Sheth
Amitava Das
Venue: EMNLP 2023 Main (2023)
Keywords: Multimodality
Fact Verification
Disinformation
Explainability

Abstract: Combating disinformation is one of the burning societal crises - about 67% of the American population believes that disinformation produces a lot of uncertainty, and 10% of them knowingly propagate disinformation. Evidence shows that disinformation can manipulate democratic processes and public opinion, causing disruption in the share market, panic and anxiety in society, and even death during crises. Therefore, disinformation should be identified promptly and, if possible, mitigated. With approximately 3.2 billion images and 720,000 hours of video shared online daily on social media platforms, scalable detection of multimodal disinformation requires efficient fact verification. Despite progress in automatic text-based fact verification (e.g., FEVER, LIAR), the research community lacks substantial effort in multimodal fact verification. To address this gap, we introduce FACTIFY 3M, a dataset of 3 million samples that pushes the boundaries of the domain of fact verification via a multimodal fake news dataset, in addition to offering explainability through the concept of 5W question-answering. Salient features of the dataset include: (i) textual claims, (ii) ChatGPT-generated paraphrased claims, (iii) associated images, (iv) stable diffusion-generated additional images (i.e., visual paraphrases), (v) pixel-level image heatmap to foster image-text explainability of the claim, (vi) 5W QA pairs, and (vii) adversarial fake news stories.

Factify 3M
A benchmark for multimodal fact verification with explainability through
5W Question-Asswering
Megha Chakraborty' Khushbu Pahwa? Anku Rani' Shreyas Chatterjee' Dwip Dalal'
Harshit Dave" Ritvik G Preethi Gurumurthy! Adarsh Mahor Samahriti Mukherjee
Aditya Pakala* Ishan Paul Janvita Reddy" Arghya Sarkar' Kinjal Sensharma Aman Chadha Amit P. Sheth' Amitava Das!
University of South Carolina, USA 2 UCLA. USA Amazon AI, USA 4 Stanford University, USA meghac@email.co edu
amitava@mai lbox. edu
Abstract
1 FACTIFY 3M: An Illustration
Combating disinformation is one burn- ing socictal crises about 67% of the Amer- ican population believes that disinformation produces lot of uncertainty, and 10% of them knowingly propagate disinformation. Disin- formation can manipulate democracy, public opinion, disrupt markets, and cause panic or even fatalities. Thus, swift detection and possi- ble prevention disinformation are vital, espe- cially with the daily flood of 3.2 billion images and 720,000 hours of videos on social media platforms, necessitating efficient fact verifica- tion. Despite progress in automatic text-based fact verification (e.g., FEVER, LIAR), the re- search community lacks substantial effort in multimodal fact verification. To address this gap, we introduce FACTIFY 3M, a dataset of 3 million samples that pushes the boundaries of the domain of fact verification via a mul- timodal fake news dataset, in addition to of- fering explainability through the concept of 5W question- answering. Salient features of the dataset are: (i) textual claims, (ii) GPT3.5- generated paraphrased claims, (iii) associated images, (iv) stable diffiusion generated tional images (i.e., visual paraphrases), (v) pixel-leve image heatmap to foster image-text explainability of the claim, (vi) 5W QA puirs, and (vii) adversarial fake news stories.
We introduce FACTIFY 3M (3 million), the largest dataset and benchmark for multimodal fact verifi- cation.
pai wark sagic aohisean moart whenhe herpita Blied
Her vudye not rive dow sant metrrin preciss
1ook this phobal
Figure 1: A tweet referring to the sports personality and known AIDS victim Magic Johnson with photo that was taken decade before COVID and, moreover, not real evidence of blood donation.
Consider the example in Fig. 1. A widely dis- tributed image of the sports legend Magic Johnson with IV line in his arm was accompanied by the claim that he was donating blood, implicitly during the COVID-19 19 pandemic. If true, this is troubling because Magic Johnson
well-known victim of AIDS and is prohibited from donating blood. The picture predated the COVID-19 epidemic by decade and is related to his treatment for AIDS.
tiffusiion-eeneratedd addi-
Textual claim: The text associated with the claim in Fig. purports that the tweet's 's author took this photo and assisted Magic Johnson in donating
Work does not relate to the position at Amazon.

---

Rezalitant damee
Voteal Sepport
Vinssinage Support
of calza
Nt. peprrptrand
greestal leage
etentie diftakn
SuQ4 pun
OPigrnmttal
Adversttll
appsiing
222100
MS2160
20.00
123300
156430
222000
Supper
Stppant_eet Jest
Toat zs sppaating
beggraer "ppering
Beaffiiiat Malionaiaal
segoatal lox ofaad
Isen at FeIle
Imsgo aCc seppating ond
166
54,00
513000
175000
Swaffeiiat Text
regeotell foe infird eaag
Tric
Irage aciber
135
555600
175000
4000000
$11.000
170000
Papprrng eoi resung
Befatr
Jlace
315/60
1093
100.00
916-000
1337,00
104
115/63
941 K0
505000
3301,00
12000
Table I: A top-level view of FACTIFY 3M: (i) classes and their respective textual/visual support specifics, (ii) number of claims, paraphrased claims, associated images, generated images, 5W pairs, evidence documents, and adversarial stories.
Sports star Magic Jeénson camer in the locepial lust mooth to donate blood to suppoct the COVID-19 crisis.
SW QA based Explainahility
Who cialiss
What claiiss
W When clains
Where claims
Why claimss
QQ: Who wost a thr
morth, Magic Jolmaan, famous athlenc, visited hospital to give blood to help with the COVID pandemic.
Prphr I:
Q1 What def Mogic
And Maye Jiinuce
Jvhanan phun?
Magic
Ql: worked
Jokesse hengh..
Prphr Magic Johnson, sports celebrity. recently visited a hospital and donuted blood to contribure tn the COVID-19 crisis.
Are Nost
the enther with
Q2.
Whse pracees
Jobnee sill hinqied.
Jahusos todouata Mand
Mlggt Mitocc
gg: mhs Aust dhe
part of?
Magic Jahnoun
mooeth fives Auged
of the poot Sept.
Prphr Recenty. Magic Johnson, well-known spocts star, went to a hospital and gave blood to support the COVID-19 19 crisss.
Ane author
COVID-19 crisis, Migic Johnson, a sports celebrity. come hospital last month to doaute blood.
Prphr
suppoct
i coution
verified false
! caution Evidence
Bot veriflable
veriftied false
Pephr S an tolprrte--uppett the COVID-19 crisss, Magic Johmson, hoepical last month to donate blood.
nend
Music Jakrars Stats. Dorss
spurts figurs, visited
The
SChoptiad.n
'Robe Swey' He Do,
witdd Blood
need will Maric sawsl- ofl - Made + newsl will Megio
Sdance vists benkeels Jobeson sisti haddisd isl Atracc Shzza Doem
uPt
ofiiica te topiat on oftik SSthoggial on
Filee Suo! St Do- rassl Tovdl
Mayie
Des 10, 2019
Vec 10, 2012
Figure 3: Claims paraphrased using GPT3.5 (Brown et al., 2020) to foster their textual di- versity.
alb for
Addu' desuatc Hend
vorieed Nivens
Magic
Ielence
Johason
No information anil.
sclated awayy
plll cn 5db 2013
Unavenay tine.
Howard Uhebrrdy tkn- plal F 2013
hete COVID-19
dunate boodl
att about ofe bode ploto.
Sports star Magic Johnson came to the bospi- tal last month donate blood support the COVID-19 crviis.9eriiis. crisis. He shared his experience on social media, expressing appreciation the medical staff at the hospital and encouraging others donate blood. He said, "I just donated blood at the hospital to help those affected by the comnavinu pandemic. want thank all the ductors, nurses, and medical stuff for their hard work and dedication. I urge everyune step up and donate blood if they are able. Evvery donation makes difTerence."
LEV 50
Figure 2: An illustration of the proposed SW QA basdd explainable fact verification system. This example illustrates the false claim shown in Fig. A typical semantic role laheling (SRL.) system processes a sentence and identifies verb verb-specific semantic roles. Therefore, for the specified example, we have 3 sentences: sentence 1 has two main verbs work and come, sentence 2 has one verb meet, and sentence 3 has one verb take. For each verb, 5W QA pair will be automatically generated (4 x 5= 20 sets of QA pairs in total for this example). Furthermore, all those 20 5W aspects will be fact-checked. some aspects end up having neutral entailment verdict, possible relevant documents with associated URLs will be listed for the end user to read further and assess. In addition, a reverse image search result will be shown to aid human fact-checkers further.
Figure 4: An example of OPT (Zhang et al., 20/22 generated fake news that confirms the Magic Johnson blood donation incident.
blood. The further implicit declaration is that he is a medical worker and possibly works for hospital that Magic Johnson visited for blood donation. ..r33.-paraphrased claims: To emulate the textual diversity in how news publishing houses re- port day-to-day happenings, is important to pro-
duce data representative of such variety. Varicty in text is embodied through difference in narrative styles, choice of words, ways of presenting factual information, etc. For e.g., Fig. 5a shows a claim and document that address the same topic but dif- fer in their textual prose. To mimic this aspect, we

---

Ted
Similar Ioo
Stable Diffusion-generateed additional images a.k.a. visual paraphrases: Referring to Fig. 5a, the diversity of images associated with the claim and document are apparent. Specifically, the image associated with the claim is that of a frontal face, while that associated with the document, while still the same person, is one of a not-so-visible face but includes multiple other entities. Such diversity is commonly seen in the wild when different news media houses cover the same news. As such, we try to emulate this aspect in our work by harnessing the power of the latest text-to-image generation. We generate additional images using Stable Diffusion (Rombach et al., 2021). Fig. 5b shows the diversity of generated images in terms of the camera angle, subject, entities, etc., which in turn offers enhanced visual diversity for multimodal claim.
nt to twvnrtpattee warterr
allopety cerrrppne quord
saafee dewayy E3I aola rafted
degee haye oeter
wit of spader (ile)
iicuttio, willin
(a) Another example of covering the same news event by two news media houses. Here the same alleged lady is visible in both images, but altogether two images are different, and the text is paraphrused differently.
(b) Stable Diffussion output for the above claim.
Pixel-Ievel image heatmap: To clearly delin- eate and hence explain which aspect of the image being referred to in the various components of the text caption, we generate pixel-level attribu- tion heatmaps to foster explainability. For e.g., referring to Fig. 5c, the heatmap highlights gothic- architecture buildings for the word capitol which the building is known for, and highlights the hu- man figure for the term investigating. A dataset of this kind would be very helpful in designing ex- plainable multimodal fact verification and possibly visual question-ansee answer-based fact verification. SWQA: The process of fact verification is inher- ently intricate, with several questions representing the components within the underlying claim that need answers to reach verdict on the veracity of the claim. Referring to the example in Fig. 1, such questions may include: (a) who donated blood? (b) when did he donate blood? (c) can Magic Johnson donate blood? (d) what can go wrong if this claim is false? Manual fact-checking can be labor-intensive consuming several hours or days (Hassan et al., 2015; Adair et al., 2017).
(c) DAAM (Tang et al., 2022) explanation ahove claim. Figure 5: An example from PromptFake 3M dataset. Multi- modal fact verification is challenging endeavor considering the intricacies of real-life data, where the entailment of images requires understanding the nuances of day-to-day life situa- tions. As such, multimodal entailment is research paradigm with limited progress; current SoTA systems lack the finesse needed to handle the complexity portrayed in these previous examples adeptly.
adopted GPT3.5 as paraphraser and generated claims Fig. 3. Associated images: The image included as part of the claim (refer Fig. 1 for the image embed- ded the tweet and Fig. 5a for images included as part of the claim) improves its trustworthiness perception since humans tend to believe visual in- put much more than mere text prose. Moreover, the text and image components together provide holistic claim assertion, similar to how news arti- cles convey world updates.
Contemporary automatic faet-checking systems focus on estimating truthfulness using numerical scores, which are not human-intens iitmannnterpretablee. Others extract explicit mentions of the candidate's facts

---

in the text as evidence for the candidate's facts, which can be hard to spot directly. Only two re- cent works (Yang et al., 2022; Kwiatkowski al., 2019) propose question answering as a proxy to fact verification explanation, breaking down au- tomated fact-checking into several steps and pro- viding a more detailed analysis of the decision- making processes. Question-answering answering- based fact explainability is indeed very promising direc- tion. However, open-ended QA for a fact can be hard to summarize. Therefore, we refine the QA-based explanation using the 5W framework (who, what, when, where, and why). Journalists follow an established practice for fact-checking, verifying the so-called 5Ws (Mott, 1942), (Stofer et al., 2009), (Silverman, 2020), (Su et al., 2019), (Smarts, 2017), @Wiki_Artice 2023). This directs verification search and, moreover, identifies miss- ing content in the claim that bears on its validity. One consequence of journalistic practice is that claim rejection is not a matter of degree (as con- veyed by popular representations such as a number of Pinocchios or crows, or true, false, half true, half false, pants on fire), but the rather specific, substantive explanation that recipients can them- selves evaluate (Dobbs, 2012). Please refer Fig. 2 to look at the possible 5W QA questionnaire for the claim in Fig.
between Al and a human-in-the-loop) has been an area of recent focus. While detecting machine-generated text from server-side models (for instance, GPT-3 (Brown et al., 2020), which is primarily utilized through an API, uses tech- niques like watermarking (Wiggers, 2022b)) is still topic of investigation, being able to do so for the plethora of open-source LLMs available online is herculean task. Our adversarial dataset will offer a testbed so that such detection advances can be measured against with the ultimate goal of curbing the proliferation of Al-generated fake news.
2 Related Works: Data Sources and Compilation
Automatic fact verification has received signifi- cant attention in recent times. Several datasets are available for text-based fact verification, e.g., FEVER (Thorne et al., 2018), Snopes (Vo and Lee, 2020), PolitiFact (Vo and Lee, 2020), FavIQ (Kwiatkowski et al., 2019), HoVer (Jiang al., 2020), X-Fact (Gupta and Srikumar, 2021), CREAK (Onoe et al., 2021), FEVEROUS (Aly et al., 2021), etc.
Multimodal fact verification has recently started gaining momentum. DEFACTIFY workshop se- ries at AAAI 2022 (Mishra, 2022) and 2023 (Suryavardhan, 2023) has released FACTIFY 1.0 (Mishra et al., 2022) and 2.0 (Mishra et al., 2023) with 50K annotated data each year, which have embedded well as part of FACTIFY 3M. Fact verification datasets are mainly classified into three major categories: (i) support, (ii) neutral, and (iii) refute. While it is relatively easier to collect data for support and neutral categories, collecting large- scale refute category fake news claims is relatively challenging. FEVER (Thorne al., 2018) pro- posed an alternative via manual imaginary claim generation, but is complex, lacks scalability, and may generate something unrealistic. Therefore, we decided to merge available datasets, at least for the refute category. It is wise to have all these datasets into one, and further, we have generated associated images using stable diffusion. While
Adversarial fake news: Fact verification sys- tems are only as good as the evidence they can ref- erence while verifying a claim's authenticity. Over the past decade, with social media having mush- roomed into the masses' numero-uno choice of ob- taining world news, fake news articles can be one of the biggest bias-inducers towards the world. To this end, using the SoTA lan- guage model, we generate adversarial news stories to offer a new benchmark that future researchers can utilize to certify the performance of their fact verification systems against adversarial news. Programmatic detection of Al Al-generated writing (where Al is the sole author behind the arti- cle) and its more challenging cousin- Al-assisted writing (where the authorship of the article split
person's outlook

---

selecting datasets, only chose datasets with evi- dence claim documents as we are interested in 5W QA-based explanation. Other datasets only with fake claims were discarded for this reason. Further- more, we use OPT (Zhang et al., 2022) to generate adversarial fake news documents of text based on the refute claims as prompts.
(i.e., prose) may have variations highlighted in Fig. 5a. We seek to echo such heterogeneity to ensure the real-world applicability of our bench- mark (cf. examples in Fig. 3 and more examples in the Appendix O). Manual generation of possi- ble paraphrases is undoubtedly ideal but is time- consuming and labor-intensive. On the other hand, automatic paraphrasing has received significant at- tention in recent times (Sancheti al., 2022; Xue al., 2022: Bandel al., 2022; Garg et al., 2021; Goyal and Durrett, 2020). Our criteria for select- ing the most appropriate paraphrasing model was the linguistic correctness of the paraphrased out- put and the number of paraphrase variations. To achieve this, we propose the following process let's say we have claim c, we generate a set of paraphrases of c. Textual paraphrase detection is a well-studied paradigm, there much state-of- the-art (SoTA) systems (Wang al., 2021, 2019; Tay et al., 2021). We pick the best model avail- able mostly trained based on the resources avail- able from SNLI (Bowman et al., 2015). Next, we use the entailment model (Wang et al., 2019) to choose the right paraphrase candidate from the gen- erated set, by doing a pairwise entailment check and choosing only examples which exhibit entail- ment with c. We empirically validated the per- formance of (a) Pegasus (Zhang et al., 2020), (b) T5 (Flan-t5-xxl variant) (Chung et al., 2022), and (c) GPT-3.5 (gpt-3.5-turbo 5-tturoo-o000 0000 variant) (Brown et al., 2020) models for our use-case and found that GPT-3.5 outperformed the rest (Ap- pendix C for details on evaluation along three di- mensions: (i) coverage, (ii) correctness, and (iii) diversity).
We have adopted automatic method to compile list of claims for support and neutral categories. is often seen that the same event gets reported by two different media houses separately on the same day therefore, one can be treated as support for the other. With this strategy in mind, we have col- lected and annotated large-scale data automatically (cf. Appendix A for details). Fig. 6 visualizes how much data from each dataset are compiled.
23%
GCOOO
19900)
102%
47.52%
14,28%
12000)
VTCC
Sropps
FEVER
Factlyy
Figure 6: Distribution of our dataset delincating its constituent components.
Table offers statistical description of the five entailment classes, their definitions in terms of tex- tual/visual support, an approximate count of the claims, paraphrased claims, images, 5W QA pairs, evidence documents, and adversarial stories for the Refute class. Furthermore, to solidify the idea behind the above categories, Fig. 5 offers walk- through of example from the proposed dataset.
There are only handful of other previous works, (Yao et al., 2022), (Abdelnabi et al., 2022), (Roy and Ekbal, 2021), (Nielsen and McConville, 2022), (Jin et al., 2017). (Luo et al., 2021). that have dis- cussed multimodal fact verification. None of them generated large resources like ours and did not dis- cuss QA-based explanation, heatmap-bart based image explainability, and adversarial assertion.
Visual Paraphrase: Stable Diffusion-based Image Synthesis
While textual diversity in claims seen in the wild commonplace, typically the visual components particularly, images also show diversity. The concept of Al-based text-to-image generators has been around for the past several years, but their outputs were rudimentary up until recently. In the
3 Paraphrasing Textual Claims
claim may have multiple diverse manifestations depending on the style and manner in which was reported. Specifically, the textual component

---

past year, text prompt-based image generation has emerged in the form of DALL-E (Ramesh al., 2021), ImageGen (Saharia et al., 2022), and Stable Diffusion (Rombach et al.. 2021). While these new-age systems are significantly more powerful, they are a double-edged sword. They have shown tremendous potential in practical applications but also come with their fair share of unintended use cases. One major caution is the inadvertent misuse of such powerful systems. To further this point, we have utilized Stable Diffusion 2.0 (Rombach et al., 2021) to generate large amount of fake news data. Stable Diffusion (Rombach et al., 2021) is powerful, open-source text-to-image generation model. The model is not only extremely capable of generating high-quality, accurate images to given prompt, but this process is also far less com- putationally expensive than other text-conditional image synthesis approaches such as (Ding al., 2021; Nichol et al., 2021; Zhou et al., 2021; Gafni et al., 2022). Stable diffusion works on stabilizing the latent diffusion process which has an aspect of randomization, as a result, it generates a different result each time. Moreover, quality control is challenge. We have generated 5 images for given claim and then further ranked them, diseussed in the next section (cf. Appendix N for examples). 4.1 Re-ranking of Generated Images In order to quantitatively assess and rank the im- ages generated by the stable diffusion model, we leverage the CLIP model (Radford et al., 2021 to obtain the best image conditioned on the prompt. We use CLIP-Score based re-ranking to select the best image corresponding the prompt. The CLIP- Score denotes the proximity between the final im- age encodings and the input prompt encoding.
denoising subnetwork. We adapt the official code available on Github (Castorini, 2022) to obtain the attribution maps in the generated images for each word in the cleaned prompt (pre-provdd processee prompt after removal of stop-words, links, etc.). See Fig. 5c and Appendix N for examples.
4.3 Quality Assessment of Synthetically Generated Images
While SD has received great acclaim owing to its stellar performance for variety of use cases, to our knowledge, to knowledge, are the first to adopt it for fake news generation. As such, to assess the quality of generated images in the context of the fake news generation task, we utilize two evaluation metrics.
We use Fréchet inception distance (FID) (Heusel et al., 2017) which captures both fidelity and diver- sity and has been the de-facto standard metric for SoTA generative models (Karras al., 2018, 2019; Ho et al., 2020; Brock et al., 2018). The process we adopted to compute the FID score to quantita- tively assess the quality of SD-generated images detailed in E.1. For chosen set of 500 claims ( (100 per category), we obtained an FID score. We obtained an FID score of 8.67 (lower is better) between the set of real images and SD-generated images for the same textual claims.
As our secondary metric, utilized Mean Opin- ion Score (MOS) at the claim category level which numerical measure of the human-judged per- ceived quality of artificially generated media (cf. Appendix E.2 for process details). Results of the conducted MOS tests are summarized in Fig. 17.
5 Automatic SW QA Pair Generation
false claim very likely to have some truth in it, some correct information. In fact, most fake news articles are challenging to detect precisely because they are mostly based on correct information, de- viating from the facts only in a few aspects. That is, the misinformation in the claim comes from very specific inaccurate statement. So, given our textual claim and image claim, we generate 5W
4.2 Pixel-level Image Heatmap
(Tang ct al., 2022) perform a text-image attribution analysis on Stable Diffusion. To produce pixel- level attribution maps, authors propose Diffusion Attentive Attribution Maps (DAAM), a novel in- terpretability method based on upscaling and ag- gregating cross attention activations in the latent

---

Ascara
anism to map these PropBank arguments to 5W semantic roles. The conversion table 4 and neces- sary discussion can be found in Appendix F.
5.2 Automatic 5W QA Pair Generation We present system for generating 5W aspect-based questions generation using language model (LM) that is fed claims as input and uses the SRL outputs as replies to produce 5W questions with respect to the 5W outputs. We experimented with variety of LMs: BART (Lewis et al., 2019) and ProphetNet (Qi et al., 2020), eventually settling on ProphetNet (see Fig. 7) based on empirically figuring out the best fit for our use-case (cf. Appendix G for details).
Mar Age"". 250
300
con unlir pantertl
Figure 7: 5W QA Generation Pipeline using PropheiNet.
We then create answers using the evidence from the questions generated using ProphetNet by run- ning them through T5 (Yamada et al., 2020) - So'TA QA model, using the 5W-based generated questions. See Section 7.2 for details.
question-answer answer pairs by doing semantic role la- beling on the given claim. The task is now based on the generated QA pairs, a fact-checking system can extract evidence sentences from existing au- thentic resources verify or refute the claim based on each question- Who, What, When, Where, and Why. Example the figure 19 and in Appendix O.
6 Injecting Adversarial Assertions
The rise of generative AI techniques with capabili- ties that mimic human's 's creative thought process has led to the availability of extraordinary skills at the masses' fingertips. This has led to the pro- liferation of generated content, which is virtually indistinguishable as real or fake the human eye, even for experts in some cases. This poses un- paralleled challenges to machines in assessing the veracity of such content.
We leverage the method of using 5W SRL to generate QA pairs (Rani et al., 2023) and verify each aspect separately to detect 'exactly where the lie lies'. This, in turn, provides an explanation of why a particular claim is refutable since can identify exactly which part of the claim is false.
5.1 5W Semantic Role Labelling
Identification of the functional semantic roles played by various words or phrases in a given sen- tence is known as semantic role labeling (SRL). SRL is a well-explored area within the NLP com- munity. There are quite few off-the-shelf tools available: (i) Stanford SRL (Manning al., 2014), (ii) AllenNLP (AIlenNLP, 2020), etc. A typical SRL system first identifies verbs in a given sen- tence and then marks all the related words/phrases haven relational projection with the verb and as- signs appropriate roles. Thematic roles are gen- erally marked by standard roles defined by the Proposition Bank (generally referred to as Prop- Bank) (Palmer et al., 2005), such as: ArgO, Argl, Arg2, and on. We propose mapping mech-
As one of the novelties of our work, we address this by introducing synthetically generated adver- sarial fake news documents for all the refute claims using OPT (Zhang al., 2022), large language model. In doing so, we attempt confuse the fact verification system by injecting fake examples act- ing as an adversarial attack. To draw a parallel in a real-world scenario, this could mean the prolifer- ation of such fake news articles online via social media, blog posts, etc. which would eventually lead to verfficcfii system being unable to make concrete decision the trustworthiness: of the news. Such scenario would lend itself as a natural mani- festation of an adversarial attack by virtue (rather, the "vice") of the fake news articles confusing the

---

fact verification system. We analyze the impact on the performance of our fact verification system in table 2. Our goal in offering these adversarial arti- cles as one of our contributions is to provide future researchers a benchmark using which they can mea- sure (and hence, improve) the performance of their fact verification system.
image, the system has to classify the data sample into one of the five categories: Support_Text, Insufficient_ Insufficient_ ietiiieettttlimmdaall and Refute. Using the Google Cloud Vision API (Google, 2022), we also perform OCR to obtain the text embedded in images and utilize that as additional input.
6.1 Accuracy of Text Generation
Text-only model: Fig. 16 shows our text-only model, which adopts siamese architecture fo- cussing only on the textual aspect of the data and ignores the visual information. To this end, we generate sentence embeddings of the claim and document attributes using a pretrained MPNet Sentence BERT model (Reimers and Gurevych, 2019a) (specifically the all
We assess the quality of text generation using per- plexity as an evaluation metrics. Perplexity measure of the likelihood of the generated sentence on a language model. We use a pre-trained GPT-2 model to evaluate text perplexity. A lower value is preferred. We have used the GPTZero detector to evaluate our perplexity score (Tian, 2023). Check- ing for paraphrased text generated over 50 claims report aver- age perplexity score of 129.06 for original claims and 175.80 for adversarial claims (Appendix J).
vari- ant). Next, we measure the cosine similarity using the generated embeddings. The score, thus gener- ated, is used the only feature for the dataset, and classification evaluated based on their Fi scores. Multimodal model: Information shared online is very often of multimodal nature. Images can change the context of textual claim (and vice versa) and lead to misinformation. As such, to holistically glean information from the available data, it is important that we consider both the visual and textual context when classifying the claims. Our multimodal architecture (Fig. 8), adopts siamese architecture and utilizes both modalities. As we utilize an entailment- approach, fea- tures from both the claim and document image- text pairs must be extracted. To this end, we uti- lize the pretrained MPNet Sentence BERT model (Reimers and Gurevych, 2019a) (specifically the text embed-
(25 original and 25 adversarial),
Experiments: Baselines & Performance In this section, we present baselines for: (i) mul- timodal entailment, (ii) 5W QA-based validation, and (iii) results of our models after adversarial injections of generated fake news stories.
7.1 Multimodal entailment: support refute? In this paper, we model the task of detecting multimodal fake news as multimodal entailment. We assume that each data point contains a reliable source of information, called document, and its associated image and another source whose validity must be assessed, called the claim which also contains a respective image. The goal to identify if the claim entails the document. Since we are interested in a multimodal scenario with both image and text, entailment has two verticals, namely textual entailment, and visual entailment, and their respective combinations. This data format is stepping stone for the fact-checking problem where we have one reliable source of news and want to identify the fake/real claims given a large set of multimodal claims. Therefore the task essentially is: given a textual claim, claim image, text document, and document
all-mpnet-base-v> variant) ding extractor and pretrained Vision Transformer (ViT) model (Dosovitskiy al., 2020) (specifi- cally the vit-base- sattaasepaachh6--24--11 as our vision embedding extractor. The cosine sim- ilarity score computed between both the claim and document image features. Furthermore, we also compute the cosine similarity for the text em- beddings, similar to the text-only model.
variant)
Table 2 shows the F1 score for the unimodal (i.e., text-only) and multimodal approaches (cf.

---

Suppars "att
Support Natiicaa)
teafffiterr "wat
trauTiccter natcinoal
Rafuse
Ncerage
Touely
Neteoald
Tosente
Tri
Fashocsed aack Ei)
989
E244
Bib
Sai
Table 2: Results of the text-only and multimodal haselines pre- and post-adwersarial attack.
pre-adversarial attack row in table 2) trained us- ing their respective feature sets. The multimodal model shows a distinct improvement in perfor- mance compared the text-only model, indicating the value-addition of the visual modality.
Stack Overflow (Makyen and Olson, 1969), while scientific conferences like ACL (Chairs, 2023) and ICML (Foundation, 2023) have released new poli- cies deterring the usage of ChatGPT for scientific writing. Indeed, the detection of Al-generated text has suddenly emerged as a concern that needs immi- nent attention. While watermarking as a potential solution to the problem is being studied by OpenAl (Wiggers, 2022b), handful of systems that detect Al-generated text such GPT-2 output detector (Wiggers, 2022a), GLTR (Strobelt et al., 2022), GPTZero (Tian, 2022), has recently been seen in the wild. Furthermore, these tools typically only produce meaningful output after a minimum (usu- ally, 50+) number of tokens. We tested GPTZero on randomly selected set of 100 adversarial sam- ples, equally divided into gennrrate text and Al generated text. Our results indicate that these systems are still in their infancy (with meager 22% accuracy). It is inevitable that Al-generated text detection techniques such as watermarking, perplexity, etc. will emerge as important paradigms in generative AI in the near future, and FACTIFY 3M will serve the community as a benchmark in order to test such techniques for fact verification. Table 2 shows the F1 score post adversarial attack for the unimodal (i.e., text-only) and multimodal ap- proaches proving that injecting adversarial news can confuse fact-checking very easily.
Chara
Decerce
Figure& &: Multimodal baseline model which takes as input: (i) claim text, (ii) claim image, (iii) document text, and (iv) document image.
7.2 SW QA-based Validation
We generate 5W Question-Answe Answer pairs for the claims, thus providing explainability along with evidence in the form of answers generated. To this end, we use the SoTA T5 (Raffel et al., 2020) model for question answering (QA) in this work. It is trained using a modified version of BERT's masked language model, which involves predicting masked words and entities in an entity- annotated corpus from Wikipedia.
After Apel 11 2303 these sou lusaity durig over 1.0 manderis. The leakly nás is the dioase o populttonn copioded as perentape the number beality ven is messin the 2333 death
Cold
Answer: 1itte Apd 11, 2027
8 Conclusion and Future Avenues
1933 dectts on awerge in
We introducing FACTIFY 3M, the largest dataset and benchmark for multimodal fact verifi- cation. We hope that our dataset facilitates research on multimodal fact verification on several aspects- (i) visual QA-based explanation facts, (ii) how to handle adversarial attacks for fact verifications, (iii) whether generated images can be detected, and (iv) 5W QA-based help journalists to fact ver- ify easily for complex facts. FACTIFY 3M will be made public and open for research purposes.
Contect
T5
Answer: Since
"Won COME-11
Quistion
Figure 9: T5-hased question answering framework.
7.3 Adversarial Attack
As the risk of Al-generated content has reached an alarming apocalypse. ChatGPT has been declared banned by the school system in NYC (Rosenblatt, 2023), Google ads (Grant and Metz, 2022), and

---

Discussion and Limitations
ii) text with metaphoric twists for example, "It's all about business a patent infringement case against Pfizer by a rival corporate reveals they knew about COVID in one way!" and "Oh my god COVID is not enough now we have to deal with HIV blood in the name of charity!". It is worthy seeing how in-domain adaptation could be made for SD image synthesis, inspired from (Ruiz et al., 2022).
In this section, we self-criticize a few aspects that could be improved and also detail how we (tenta- tively) plan to improve upon those specific aspects-
8.1 Paraphrasing Claims
Manual generation of possible paraphrases is un- doubtedly ideal but time-consuming and labor- intensive. Automatic paraphrasing is good way to scale quickly. but there could be more complex variations of meaning paraphrases hard to generate automatically. For example - "It's all about busi- ness patent infringement case against Pfizer by rival corporate reveals they knew about COVID in one way!" and "Oh my god COVID is not enough now we have to deal with HIV blood in the name of charity!".
8.3 5W SRL
Semantic role labeling is a well-studied sub- discipline, and the mapping mechanism we pro- posed works well in most cases except in elliptic situations like anaphora and cataphora. In the fu- ture, would like to explore how anaphora and coreference resolution (Joshi et al., 2019) can aid an improvement.
An ideal for this shortcoming would be to man- ually generate a few thousand paraphrase samples and then fine-tune language models. On the other hand, paradigm in-context Learning is gain- ing momentum (Xun et al., 2017). In-context learn- ing has been magical in adapting language model to new tasks through just few demonstration ex- amples without doing gradient descent. There are quite a few recent studies that demonstrate new abilities of language models that learn from a hand- ful of examples in the context (in-context learning ICL for short). Many studies have shown that LLMs can perform series of complex tasks with ICL, such solving mathematical reasoning prob- lems (Wei al., 2022). These strong abilities have been widely verified as emerging abilities for large language models (Wei et al., 2022). From prompt enginecring to chain of thoughts, we are excited to do more experiments with the new paradigm of in-context learning for automatically paraphrasing claims.
8.4 5W QA Pair Generation
5W semantic role-based question generation is one of the major contributions of this paper. While automatic generation aided in scaling up the QA pair generation, it also comes with limitations of generating more complex questions covering mul- tiple Ws and how kinds of questions. For example "How Moderna going to get benefited if this Pfizer COVID news turns out be a rumor?". For the betterment of FACTIFY benchmark, we would like to generate few thousand manually generated abstract QA pairs. Then will proceed towards in- context Learning (Xun et al., 2017).
Abstractive question-answwriigg answering has received momentum (Zhao et al., 2022), (Pal al., 2022) recently. We want to explore how we can gen- erate more abstract QA pairs for the multimodal fact-verification task.
8.5 QA System for the 5W Questions
Generated performance measures attest the pro- posed QA model needs a lot more improvement. This is due to the complexity of the problem and we believe that will attract future researchers to try this benchmark and conduct research on multi- modal fact verification.
8.2 Image Synthesis using Stable Diffusion Although, in general, the quality of the image syn- thesized by Stable Diffusion is great, it does not perform well two cases i) very long text (more than 30 words or so, multiple sentence claim, etc.),

---

It has been realized by the community that rele- vant document retrieval is the major bottleneck for fact verification. Recent work, such as Hypothet- ical Document Embeddings (HyDE) (Gao et al., 2022), introduced a fresh perspective to the prob- lem and applied a clever trick even if the wrong answer is more semantically similar to the right answer than the question. This could be an inter- esting direction explore and examine how that could aid in retrieving relevant documents and an- swers.
Christodoulopoulls, Oana Cocarascu, and Arpit Mittal, 2021. FEVEROUS: Fact extraction and VERification over unstructured and structured information.
Elron Bandel, Ranit Aharonov, Michal Shmueli- Scheuer, lya Shnayderman, Noam Slonim, and Liat Ein-Dor. 2022. Quality controlled paraphrase gener- ation. arXiv preprint arXiv:2203. 100:20000..
Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evalu- ation measures for machine translation and/or sum- marization, pages 65-72.
8.6 Adversarial Attack
Precisely, we are the first to formally introduce an adversarial attack for fact verification and intro- ducing large-scale data. While it is a hot topic of discussion how systems identify AI AT-generated text, there is breakthrough so far. We would like to explore more in this direction more, specifically for multimodal fact verification.
Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processngg with Python: analyzing text with the natural language toolkit. O'Reilly Media. Inc.".
Ali Borji. 2022. Generated faces in the wild: Quantita- tive comparison of stable diffusion, mid journey and dall-e arXiv preprint 3xXi::2200000500
Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. large anno- tated corpus for learning natural language inference. arXiv preprint arXiv:/508. 300.0055...0336..
References
Sahar Abdelnabi, Rakibul Hasan, and Mario Fritz. 2022. Open-domain, content-based, multi-modal fact-checking of out-of-context images via online resources. In Proceedings of the JEFF/CVF Confer- ence on Computer Vision and Pattern Recognition. pages (4400-19499
Andrew Brock, Jeff Donahue, and Karen Simonyan. 2018. Large scale gan training for high fi- delity natural image synthesis. arXiv preprint arXiv:1809 1009.
Bill Adair, Chengkai Li, Jun Yang, and Cong Yu. 2017. Progress toward the holy grail "; The continued quest to automate fact-checking.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1377-9999
AFPI AFP India. 2022. Afp india. [Online; accessed 2023-01-02).
AFPU AFP USA. 2022. Afp usa. [Online; accessed 2023-01-02).
Castorini Castorini. 2022. Castorini/daam: Diffusion attentive attribution maps for interpreting stable dif- fusion.
Kalim Ahmed, Shinjinee Majumder, Abhishek Kumar, Arjun Sidharth, and Vansh Shah. 2020. [link].
Program Chairs. 2023. Acl 2023 policy on ai writing assistance.
AllenNLP. 2020.
Allennlp semantic role la- beling. https:/demo.alleemi rogssemantic-role- labeling. [Online; accessed 2023-01-02).
Pierre Chambon, Christian Bluethgen, Curtis P Lan- glotz. and Akshay Chaudhari. 2022. Adapt- ing pretrained vision-language foundational mod- els to medical imaging domains. arXiv preprint ar.:v:2210.04133.
Rami Aly, Zhijiang Guo, Michael Sejr Schlichtkrull, James Thorne, Andreas Vlachos, Christos

---

Hyung Won Chung, Hou, Shayne Longpre, Barret Zoph, Yi Tay. William Fedus, Yunxuan Li, Xuezhi Wang. Mostafa Dehghani, Siddhartha Brahma, Al- bert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowd- hery, Alex Castro-Ros, Marie Pellat, Kevin Robin- son, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, An- drew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le, and Jason Wei. 2022. Scaling cnaruuciionfiiettune language models.
Google. 2022.
Cloud https://clou..gogg com/vvssio.. cessed 2023-01-02].
Vision
Api.
[Online; ac-
Tanya Goyal and Greg Durrett. 2020. Neural syntac- tic preordering for controlled paraphrase generation. arXiv preprint arXiv:2005.0m 020000.
Nico Grant and Cade Metz. 2022. A new chat bot is "code red' for google's search business.
Ashim Gupta and Vivek Srikumar. 2021. X-FACT: New Benchmark Dataset for Multilingual Fact Checking. In Proceedings of the 59th Annual Meet- ing the Association for Computational Linguistics, Online, Association for Computational Linguistics.
Jacob Devlin, Ming- Wiig-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv: 1870.04805.
Yaru Hao, Zewen Chi, Li Dong, and Furu Wei. 2022. Optimizing prompts for text-to-image generation. arXiv preprint arXiv:2212.0966
Ming Ding, Zhuoyi Yang, Wenyi Hong. Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. 2021. Cogview: Master- ing text-to-image generation via transformers. Ad- vances Neural Information Processing Systems, 34:19822-19835.
Naeemul Hassan, Chengkai Li, and Mark Tremayne. 2015. Detecting check-worthy factual claims in presidential debates. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management, CIKM '15, page 1838, New York, NY, USA. Association for Computing Machinery.
Michael Dobbs. 2012. The rise of political fact- checking, how reagan inspired journalistic move- ment. New America Foundation, pages 5-5.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. De- noising diffusion probabilistic models.
Neural Information Processing Systems Foundation. 2023. Clarification on large language model policy 11m.
Newschecker home. 2022. [link].
IT India Today. 2022. Fact check news: Truth behind fake news images, videos. [Online; accessed 2023- 01-02].
Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman. 2022. Make-a-scena Scene-based text-to-image gen- eration with human priors. arXiv preprint arXiv:2203.13331.
Asian News International. Asian news international.
Yichen Jiang, Shikha Bordia, Zheng Zhong, Charles Dognin, Maneesh Singh, and Mohit Bansal. 2020. Hover: dataset many-hop fact extraction and claim verification. arXiv preprint arXiv:2011 1003088.
Luyu Gao, Xueguang Ma. Jimmy Lin, and Jamie Callan. 2022. Precise zero-shot dense retrieval without rele- vance labels. arXiv preprint arXiv:2212. 10496.
Zhiwei Jin, Juan Cao, Han Guo, Yongdong Zhang, and Jiebo Luo. 2017. Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In Proceedings of the 25th ACM international con- ference Multimedia, pages 795-816
Sonal Garg, Sumanth Prabhu, Hemant Misra, and G Srinivasaraghavan. 2021. Unsupervised con- textual paraphrase generation using lexical con- trol and reinforcement learning. arXiv preprint 1r3iv:2103.11777

---

Mandar Joshi, Omer Levy, Daniel Weld, and Luke Zettlemoyer. 2019. BERT for coreference resolution: Baselines and analysis. In Empirical Methods in Natural Language Processing (EMNLP).
et al. 2019. Mediapipe: A framework for perceiving and processing reality. In Third Workshop Com- puter Vision for AR/VR at IEEE Computer Vision and Pattern Recognition (CVPR), volume 2019.
Tero Karras, Samuli Laine, and Timo Aila. 2018. A style-based generator architecture for generative ad- versarial networks.
Grace Luo, Trevor Darrell, and Anna Rohrbach. 2021. Newsclippings: Automatic generation of out-of-context multimodal media. arXiv preprint arXiv:2104005822
Tero Karras, Samuli Laine, Miika Aittala, Janne Hell- sten, Jaakko Lehtinen, and Timo Aila. 2019. Ana- lyzing and improving the image quality of stylegan.
Pavithra K M. Harshavardhan Konda, Dilip Kumar Sri- pada, Varun Borugadda, Akhil Reddy, and Abhishek Mandadi. English archives.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Al- berti, Danielle Epstein, Illia Polosukhin, Jacob De- vlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wo ding-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: A benchmark for question swering research. Transactions of the Association for Computational Linguisties, 7:452 466.
Mod Makyen and Peter Olson, 1969. Temporary policy: Chatgpt is banned.
Christopher D. Manning. Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David Mc- Closky. 2014. The stanford corenlp natural language processing toolkit. An (System Demonstrations), pages 55 60. The Association for Computer Linguis- tics.
Christian Ledig, Lucas Theis, Ferene Huszár, Jose Ca- ballero. Andrew Cunningham, Alejandro Acosta, An- drew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. 2017. Photo-realistic single image super- resolution using generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recogmition, pages 468 4681-4690.
Shreyash Mishra. 2022. De-factify 2022. [Online; accessed 2023-01-02].
Shreyash Mishra, S Suryavardan, Amrit Bhaskar, Parul Chopra, Aishwarya Reganti, Parth Patwa, Amitava Das, Tanmoy Chakraborty, Amit Sheth, Asif Ekbal, and Chaitanya Ahuja. 2023. Factify 2: A multimodal fake news and satire news dataset. In proceedings of defactify 2: second workshop on Multimodal Fact- Checking and Hate Speech Detection, CEUR.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to peaseett-sequenee pre-training for nat- ural language generation, translation, and compre- hension. arXiv preprint arXiv: 10900.13461
Shreyash Mishra, S Suryavardan, Amrit Bhaskar, Parul Chopra, Aishwarya Reganti, Parth Patwa, Amitava Das, Tanmoy Chakraborty, Amit Sheth, Asif Ekbal, et al. 2022. Factify: A multi-modal fact verification dataset. In Proceedings of the First Workshop on Multimodal Fact- Checcinnn and Hate Speech Detec- tion (DE-FACTIFY).
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Yinhan Liu, Myle Ou, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint urXiv:1901 1692.
Frank Luther Mott. 1942. Trends in newspaper content. The Anmals of the American Academy of Political and Social Science, 219:60-65.
Cable News Network. Cable news network.
Boom Live. Fact check stories and news on politics, health, viral photos and social posts.
ABC News. Abs news.
Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Yong, Juhyun Lee,
Vishvas News. 2022. Vishvas news fact check hindi: Fact checking of fake and viral news. [Online; ac- cessed 2023-01-02).

---

Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Πla Sutskever, and Mark Chen. 2021. Glide: To- wards photorealistic image generation and editing with text- eext-guided diffusion models. arXiv preprint arXiv:2112, 10741.
Weizhen Qi. Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, and Ming Zhou. 2020. Prophetnet: Predicting future n-gram for sequence-to-sequeee pre-training. arXiv preprint arXiv:2001.0406
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021a. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748-8763. PMLR.
Dan S Nielsen and Ryan McConville, 2022. Mumin: large-scale multilingual multimodal fact-checked misinformation social network dataset. In Proceed- ings of the 45th International ACM SIGIR Confer- ence on Research and Development in Information Retrieval, pages 3141-3153.
Animesh Nighojkar and John Licato. 2021, Improving paraphrase detection with the adversarial paraphras- ing task. arXiv preprint 11x.:22006.0769/
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 202 learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748-8763. PMLR.
Tong Niu, Semih Yavuz, Yingbo Zhou, Nitish Shirish Keskar, Huan Wang, and Caiming Xiong. 2020. Un- supervised paraphrasing with pretrained language models. arXiv preprini arXiv:2010. 12885.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter Liu, et al. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn. Res., 21(140):1-67.
Yasumasa Onoe, Michael J.Q. Zhang, Eunsol Choi, and Greg Durrett. 2021. Creak: A dataset for common- sense reasoning over entity knowledge. OpenRe-
view.
Vaishali Pal, Evangelos Kanoulas, and Maarten Rijke. 2022. Darameter-efficient abstractive question an- swering over tables text. In Proceedings of the Second DialDoc Workshop on Document- Decament-groundee Dialogue and Conversational Question Answering, pages 41-53, Dublin, Ireland. Association for Com- putational Linguistics.
Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don't know: Unanswerahle ques- tions for squad. arXiv preprint
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions machine comprehension of text. arXiv preprint arxiv:1606.0525
Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational linguistics, 31(1):71- 1- 106.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image gen- eration. In International Conference on Machine Learning, pages 8821-8831. PMLR.
Kishore Papineni. Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th annual meeting of the Association for Compu- tational Linguisties, pages 311-318.
Anku Rani, S.M Towhidul Islam Tonmoy, Dwip Dalal, Shreya Gautam, Megha Chakraborty, Aman Chadha, Amit Sheth, and Amitava Das. 2023. FACTIFY- SWQA: 5W aspect-based fact verification through question answering. In Proceedings of the 61st An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10421- 10440, Toronto, Canada, Association for Computa- tional Linguistics.
Jungsoo Park, Sewon Min, Jaewoo Kang, Luke Zettle- moyer, and Hannaneh Hajishirzi. 2021. Faviq: Fact verification from iefoommationsseeiing questions. arXiv preprint
Gaurav Parmar, Richard Zhang, and Jun- Yan Zhu, 2021. On aliased resizing and surprising subtleties in gan evaluation.
Nils Reimers. 2022. Sentencetransformees documenta- tion. https://sbert.net [Online; accessed 2023-01- 02].

---

Nils Reimers and Iryna Gurevych. 2019a. Sentence- bert: Sentence embeddings using siamese bert- networks. arXiv preprini urXiv:1908 10084.
Northwestern Now Story. 2022. Chatgpt writes con- vincing fake scientific abstracts. [Online; accessed 2023-01-02].
Nils Reimers and Iryna Gurevych, 2019b. Sentence- bert: Sentence embeddings using siamese bert- networks. arXiv preprint 10i.:v9908.000.
Hendrik Strobelt, Sehastian Gehrmann, and Alexan- der Rush. 2022. Giant language model room. [Online; accessed
http://gtrrio/dis/indxe..t 2023-01-02).
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. 2021. High- resolution image synthesis with latent diffusion mod- els.
Jing Su, Xiguang Li, and Lianfeng Wang. 2019. The study of a journalism which is almost 99% fake. Lingue Culture RediazionniLanguueee Cultures Me- diation (L.CM Journal), 5(2): 15-137.
Kalhan Rosenblatt. 2023. Chatgpt banned from new york city public schools' devices and networks.
S Suryavardhan. 2023. De-factify 2023. [Online; cessed 2023-01-02].
Arjun Roy and Asif Ekbal. 2021. Mulcob-mulfav: Mul- timodal content based multilingual fact verification. In 2021 International Joint Conference on Neural Networks (ICNN),, pages 1-8.
Raphael Tang, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Jimmy Lin, and Ferhan Ture. 2022. What the daam: Interpreting stable diffusion using cross attention. arXiv preprint 22X.:2220.00485.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. 2022. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.
Yi Tay, Vinh Q Tran, Sebastian Ruder, Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Si- mon Baumgartner, Cong Yu, and Donald Metzler. 2021. Charformer: Fast character transformers via gradient- based subword tokenization. arXiv preprint arXiv:2106. 12672.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. 2022. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.1 14887
James Thorne Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. Fever: large-scale dataset for fact extraction and verification. arXiv preprint arXiv:
Abhilasha Sancheti, Balaji Vasan Srinivasan, and Rachel Rudinger. 2022. Entailment relation aware paraphrase generation. arXiv preprint
Edward Tian. 2022, Gptzero. https://gptzero.mee. (On- line; accessed 2023-01-200
arXiv:2203.1048. 10483.
Christoph
Schuhmann.
2022.
Edward Tian. 2023. Gptzero detec.
Christophschubmaam/impoooo wwwppprtttedd--tettiiieeeeieiice predictor: Clip+mlp aesthetic score predictor.
Hindustan Times. Hindustan times.
Schuster, Adam Fisch, and Regina Barzilay. 202
TOI Times of India 2022. Fact check: Truth behind fake news on times india. [Online; accessed 2023- 01-02].
Get your vitamin c! robust fact verification with con- trastive evidence. arXiv preprint arXiv:2103. 108551..3. 08541.
Craig Silverman. 2020. Verification handbook: Home- page.
Nguyen Vo and Kyumin Lee. 2020. Where are the facts? searching for fact-checked information alle- viate the spread fake news. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020).
Media Smarts. 2017. How to recognize false content online the new 5 ws.
Kathryn T Stofer, James R Schaffer, and Brian A Rosen- thal. 2009. Sports journalism: An introduction to reporting and writing. Rowman & Littlefield Pub- lishers.
Sinong Wang, Han Fang, Madian Khabsa, Hanzi Mao, and Hao Ma. 2021. Entailment as few-shot learner. arXiv preprint arXiv:2 14990.
14690.

---

Wei Wang, Bin Bi, Ming Yan, Chen Wu, Zuyi Bao, Jiangnan Xia, Liwei Peng, and Luo Si. 2019. Struct- bert: Incorporating language structures into pre- training for deep language understanding. arXiv preprint arXiv:1908.0455 (e0...080.555 04577.
Jing Yang. Didier Vega-Oliveros, Taís Seibt, and Ander- son Rocha. 2022. Explainable fact-checking through question answering. In ICASSP 2022 2022 IEEF International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8952-58 8956.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain-of-thouge prompting elicits reasoning in large language models.
Barry Menglong Yao, Aditya Shah, Lichao Sun, Jin- Hee Cho, and Lifu Huang. 2022. End-to-end mul- timodal fact-checking and explanation generation: A challenging dataset and models. arXiv preprini 2:Xx::200512244: 12487.
Kyle Wiggers. 2022a. Gpt-2 output detector demo. [Online; ac-
https//mmeena-eeeeemiddettttttht cessed 2023-01-02).
Jingqing Zhang. Yao Zhao, Mohammad Saleh, and Pe- Liu. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In In- ternational Conference Machine Learning, pages 11328 11339. PMLR.
Kyle Wiggers. 2022b.
Openaï's at- tempts to watermark ai text hit limits.
http://ccchcuunccccoo//222222//2 2/tp////////p.aaiii.....
sitemppssto--watemmark- watermark-ai -i-tttt----------------- ai-text-hit hit-limits. [Online:
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuobui Chen, Christopher Dewan, Mona Diab. Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.00
accessed 2023-01-02].
Wikipedia Wiki_Article. 2023. Five ws.
Tang Xue, Yuran Zhao, Chaoqi Yang, Gongshen Liu, and Xiaoyong Li. 2022. Sect: A successively con- ditional transformer for controllable paraphrase gen- eration. In 2022 International Joint Conference on Neural Networks (LJCNN), pages 1-8. IEEE.
Wenting Zhao, Konstantine Arkoudas, Weiqi Sun, and Claire Cardie. 2022. Compositional task-oriented parsing as abstractive question answering. In Pro- ceedings of the 2022 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4418 4418-4427, Seattle, United States. Association for Computational Linguistics.
Guangxu Xun, Xiaowei Jia, Vishrawas Gopalakrishnan, and Aidong Zhang. 2017. survey on context learn- ing. IEEE Transactions on Knowledge and Data Engineering, 291)538-55.
Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, and Tong Sun. 2021. Lafite: Towards language- free training for text-to-image generation. arXiv preprint arXiv:2111.1
Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, and Yuji Matsumoto. 2020. Luke: deep con- textualized entity representations with entity-aware self-attention. arXiv preprint vrXiv:0010.0105.

---

Frequently Asked Questions FAQs
Does Stable Diffusion offer the adeptness to generalize and scale to different real-world scenarios? In other words, Stable Diffusion is great at generating one-off plausible examples but generalizability to life's combinatorial scenarios a concern?
Ans. Fake news is generally written connecting popular topics and personalities, therefore stable diffusion does decent job. However, there are some limitations, which we discussed in detail in the limitation section 8.2. Moreover, we have generated stable diffusion images for all claims as visual paraphraser mentioned in section 4. In addition, have highlighted range of diverse examples in appendix M. Furthermore, we have presented holistic evaluation through objective (FID) and subjective (MOS) metrics. Please refer to the section 4.3, table 17.
What are the novel assertions in this paper if the multimodal data is generated automatically using SoTA generative models?
Ans. The novelty of this work is three-fold:
Justification of the classification using 5WQA verification
Injecting an adversarial attack in the form of fake news to make the dataset more robust
Adding synthetically generated images using Stable Diffusion to enhance multimodal data
5W SRL is understandable, but how is the quality of the 5W QA pair generation using a language model?
Ans. We have evaluated our QA generation against the SoTA model for QA Tasks T5. Please refer the appendix section table for a detailed description of the process and evaluation. Moreover, please see the discussion in the limitation section 8.4.
What is the overarching idea we're trying to highlight by introducing an adversarial attack?
Ans. The broader point that the introduction of adversarial attack indicates is that fact verifica- tion model needs be more robust in combating synthetically generated fake news, which easily publishable by wrongdoers on the internet. This is of extreme relevance today as Al-assisted writing has become very popular and miscreants spread fake news taking advantage of LLMs.
How does adversarial attack impact the performance?
Ans. As reported in table 2, we see that the performance of the model drops across all categories post adversarial attack using fake claims. This is seen in both instances: text-only and multimodal model.
Despite the controversies surrounding Al-assisted writing, why have we still chosen to use LLMs as our paraphrasers?
Ans. The controversy lies mostly in conversational setting or creative writing. When it comes to para- phrasing news claims, have empirically found that GPT-3.5 (specifically the text-davinci-0301 variant) (Brown al., 2020) performs better in comparison other models such Pegasus (Zhang et al., 2020) and T5 (F (Flan-t5- xxxxx variant) (Chung et al., 2022).
What was the chosen metric of evaluation for text generation using LLMs?
Ans. For now, we have evaluated adversarial claims using GPTZero text detector. Evaluation standard metrics such as control and fluency will be made public along with our dataset.

---

Appendix
This section provides supplementary material in the form of additional examples, implementation details, etc. to bolster the reader's understanding of the concepts presented in this work.
A Data sources and compilation In this section, we provide additional details on data collection and compilation. As mentioned in section 2 we are only interested in the refute category from the available datasets; for support and neutral categories we have collected a significant amount of data from the web. This process semi-automatic. For FEVER and VITC, only the claims belonging in the train split were used for making the dataset. FaVIQ (Park al., 2021) has two sets: Set and Set R. Set A consists of ambiguous questions and their disambiguations. Set R is made of unambiguous guestion-answer pairs. We have used claims from set A in our dataset to make the entailment task more challenging. In the case of Ver (Jiang et al., 2020), we have used all 26171 claims for our dataset.
In the Factify dataset (Mishra al., 2022), the authors have collected date-wise tweets from Twitter handles of Indian and US news sources: Hindustan Times (Times), ANI (International) for India, and (ii) ABC (News), CNN (Network) for the US, based on accessibility, popularity and posts per day. We drew our motivation from (Mishra et al., 2022). Morcover, these Twitter handles are eminent for their objective and disinterested approach. From each tweet, the tweet text and the tweet image(s) have been extracted. Listing A delineates each attribute in the dataset and its respective description while listing B elaborates on the process we followed for collecting data for Support and Neutral categories.
Listing B: Procedure for data collection for Support t and Neutral cate- gories
Listing A:
At-
tributes
For each tweet of account A, authors got similar tweets from ac- count B. Similarity is measured on the basis of text. Text similarity is measured using Sentence BERT (Reimers and Gurevych, 2019b) first, and then the extent of common words is measured as the sec- ond metric.
Claim: Tweet A text
claim_image: Tweet A image
Next, the image similarity for the corresponding images of the tweet pair was calculated. Image similarity is measured using histogram similarity and cosine similarity on pre-trained ResNet50 model.
Claim_ocr:
Tweet A image
OCR
According to the scores for each of these measures, the tweet pair Support_Text,
Document: Tweet B article text
is classified into categories: Support_Multimonn Insufficient MelccinntMuultimmeeeeee Muldimadal, and Insufficient Text. The various thresholds used for classification are listed in Figure 10.
Document Document_image: Tweet B image
From this tweet pair, authors have selected tweet (say tweet B) and obtained the url for the corresponding article published on the source's 's website from the tweet text. Then the tweet text was re- placed with article contents after scraping it (document in dataset), This is done so as mimic real world fact checking process, i... manually comparing claims with documents articles.
Document_oca Tweet B image
OCR
Category
The image OCRs were obtained using Google Cloud Vision API (Google, 2022).

---

B Text and image similarity measures
Table explains the five classes in the dataset. For the appropriate classification of the dataset, two similarity measures were computed.
B.1 Sentence comparison
We adopt two methods to check similarity given set of two sentences:
Sentence BERT:
Sentence BERT (Reimers and Gurevych, 2019b) is modification of the BERT model that uses contrastive loss with a siamese network architecture derive sentence embeddings. These sentence embeddings can be compared with each other to get their corresponding similarity score. Authors use cosine similarity as the textual similarity metric.
We utilize Sentence BERT (SBERT) (Reimers and Gurevych, 2019b) instead of alternatives such as BERT or RoBERTa, owing to its rich sentence embeddings yielding superior performance while being much more time-efficient (in terms of sentences/sec) (Reimers, 2022). We manually decide on a threshold value TI for cosine similarity and classify the text pair accordingly. If the cosine similarity score is greater than 77, then it is classified into the Support category. On the other hand, if the cosine similarity score is lower than TI, the news may may not be the same (the evidence at hand is insufficient to judge whether the news is the same not). Hence it is sent for another check before classifying it into the Insufficient category. NLTK: If the cosine similarity of the sentence pair is below TI, we use the NLTK library (Bird et al., 2009) check for common words between the two sentences. If the score the common word is above different manually decided threshold 72, only then the news pair classified into the Insufficient category. Not sure what this sentence trying to say let's rephrase. Common words are being checked to ensure that the classification task is challenging. To check for common words, both texts in the pair are preprocessed, which included stemming and removing stopwords. The processed texts are then checked for common and similar words, and their corresponding scores are determined. If the common words score is greater than T2, the pair is classified Insufficient else the pair is dropped.
B.2 Image comparison
We adopt two metrics for assessing image similarity:
Histogram Similarity: The images are converted to normalized histogram format and similarity is measured using the correlation metric cite.
Cosine Similarity: The images are converted to feature vectors using pre-trained ViT (Dosovitskiy et al., 2020) model, and these feature vectors are used to calculate the cosine similarity score. Manually decided thresholds, as described in Figure 10, are used to judge whether the text and image pair is similar or not.
The text pairs are first classified into either Support Insufficient cate- gories, and then further sub-classified into Support_eextSuppott_Mie momdmll or Souufficient_Txx/nnsffff
timodal categories based on the similarity of the im- age pairs. If the corresponding images for the texts are similar, then they could be used to judge

---

whether news the same not. The category where both the images and the support_Multimodaa The category where the images are similar but the texts were not is called Aesufficieett_Multie
similar is called
the corresponding images for the texts were not similar, then they could not be used to judge whether news is the same or not. The category where both the images and the texts are not similar is called Insuff insufficient__ ccien__eext. The category where the texts are similar but the images are not is called Support_Ted
HIST
Yes
RE:SMIT 0.75
RESNETT 6
Yee
Yes
SBERY
2028
RESNET
Support Multimodia
Support Text
No
Pair
SBERT sup
HIST am 0:33
Yes
RESNET 0.75
-VOw
RESNET 6
NLTK
Vee
0.3
RESMET
Insufficient
Insufficint
Figure 10: Text and image pair similarity based on classification thresholds pre-trained models.
For the refute category, serape several reliable fact-check websites like Vishwas (News, 2022), Times of India (Times of India, 2022), India Today (India Today, 2022), AFP India (AFP India, 2022), AFP USA (AFP USA, 2022), AltNews (Ahmed et al., 2020), BOOM (Live), Factly (M al.), and NewsChecker (home, 2022). For each article published on these websites, we collect the claim (sentence that states the fake news), document (text that proves claim is false), claim images (fake news image, could be screenshot of the fake post), document image (image that is proof of the fake nature of the claim).
C Paraphrasing textual claims
textual given claim may appear in various different textual forms in real life, owing to variety in the writing styles of different news publishing houses. Incorporating such variations is essential to developing strong benchmark to ensure a holistic evaluation. This forms our motivation behind paraphrasing textual claims. Manual generation of possible paraphrases is undoubtedly ideal, but that process is time-consuming and labour-intensive. On the other hand, automatic paraphrasing has received significant attention in recent times (Niu et al., 2020) (Zhang et al., 2020) (Nighojkar and Licato, 2021). As mentioned in section for given claim, we generate multiple paraphrases using various models and perform entailment using (Wang al., 2019) a SoTA model trained on the on SNLI task (Bowman et al., 2015) - to detect how many of them are entailed in the actual claim.
In the process of choosing the appropriate model based on a list of available models, the primary question we asked is how to make sure the generated paraphrases are rich in diversity while still being linguistically correct. A top level, we delineate the process followed to achieve this as follows (more details later in this section). Let's say we have a claim We generate paraphrases using a paraphrasing model. This yields set of paraphrases, denoted by pf...., Ph Next, make pair-wise comparisons of these paraphrases with c, resulting in c pf...... and c- ph, At this step, we identify the examples which

---

entailed, and only those chosen.
However, there are many other secondary factors, for c.g., a model may only be able to generate limited number of paraphrase variations compared to others but others can be more correct and/or consistent. As such, we considered three major dimensions in our evaluation: (i) coverage, (ii) correctness, and (iii) diversity. To offer transparency around experiment process, we detail the aforementioned evaluation dimensions as follows.
Coverage the number of considerable paraphrase generations that model generates: We intend to generate up to 5 paraphrases per given claim. Given all the generated claims, we perform a minimum edit distance (MED) calculation at the word level instead of a character level. If MED is greater than 2 for any given paraphrase candidate (for e.g., pi in the above example) with the claim then we further consider that paraphrase, otherwise discarded. We evaluated all four models based on this setup to identify the model of choice which is generating the maximum number of considerable paraphrases.
Correctness correctness in paraphrase generations: After the first level of filtration, we performed pairwise entailment and kept only those paraphrase candidates, marked as entailed by the (Liu et al., 2019) (Roberta Large), SoTA trained on SNLI (Bowman al., 2015).
Diversity linguistic diversity in paraphrase generations: We are interested in choosing a model that can produce paraphrases with significant linguistic diversity. This implics that we are interested in checking for dissimilarities between generated paraphrase claims. For e.g., pf P pf PS. pf P4. .... PÍ Pf- this process is repeated for all the other paraphrases and the dissimilarity score is averaged across all paraphrase generations. Since there is standard metric to measure dissimilarity, use the inverse of the BLEU score as a proxy metric. This gives us an understanding of the linguistic diversity of given model.
Based on our experiments centred around the above dimensions, we experimented with three models: (a) Pegasus (Zhang et al., 2020), (b) T5 (Flan-t5-xx1 variant) (Chung et al.. 2022), and (c) GPT-3.5 (gpt-3.5- 501-3.5-turrie 5-turbo--300 variant) (Brown et al., 2020) and found that GPT-3.5 (gpt-3. 5-tuur-o--00000 3 was
ideal. The results of our experiments are reported in table 3 below.
10
GPT 3.5 turbo -0301
PEGASUS
T5 Large
M:3
Model
Coverage
Correciness
Diversity
Diversiy
5.63
Pegasus
32.46
94.38%
3,76
TS
30.26
83.84%
3.17
33
SPP33.-text-aaviciie
35.51
88.16%
7.72
Table 3: Evaluation dimensions textual claim para- phrasers.
3:477
norcasing number paraphrases
Figure 11: Variation of Diversity with Increase in number of paraphrases generated

---

D Visual paraphrasing using Stable Diffusion
Building upon section 4, we highlight the process behind visual paraphrasing in this section. Diffusion models are machine learning models that are trained to denoise random gaussian noise step by step to get sample of interest, such as an image. However one of the major downsides of diffusion models is that the denoising process is both time and memory consumption are very expensive. The main reason for this is that they operate in pixel space which becomes unreasonably expensive, especially when generating high-resolution images. Stable diffusion was introduced solve this problem diffusion. Latent diffusion reduces the memory and computational cost by applying the diffusion process over a lower dimensional latent space instead of on the actual pixel space. It is trained with the objective of "removing successive applications of Gaussian noise to training images", and can be considered as sequence of denoising autoencoders.
depends on Latent
Quality control is a big reason to worry when paraphrasing automatically. There are two aspects we have tested for the available models (i) variations, and (ii) the number of paraphrases generated. Figures 18c, 18d some examples of the advanced capabilities of the model and how it can be used ("misused") to generate fake news. Specifically, these examples highlight events that are fake and solely rely on the uncanny ability of Stable Diffusion to generate realistic art.
D.1 Explainability of generated images
In table 9, we can see that for the word slapped, the driver's cheek and Will Smith's 's hand are getting highlighted. DAAM (Tang et al., 2022), which provides cross-attention interpretation of syntactic textual relations in visual object interactions fosters explainability of our dataset.
E A Assessment of Stable Diffusion generated images
While Stable Diffusion has received great acclaim owing to its stellar performance for variety of use cases, to knowledge, we are the first to adopt it for fake news generation. As such, to assess the quality of generated images in the context of the fake news generation task, utilize two evaluation metrics.
E.1 FID & Relevance Score-based quantitative assessment of Stable Diffusion generated images
While Stable Diffusion has received great acclaim owing to its stellar performance for variety of use cases, to our knowledge, we are the first to adopt it for fake news generation. As such, to assess the quality of generated images in the context of the fake news generation task, we utilize two evaluation metrics i) FID (Heusel et al., 2017) and ii) Relevance Score (Hao et al., 2022) details are discussed in the following paragraphs.
FID Score: To compute the FID scores, we first filter the claims from our dataset that consist of person entities by leveraging the BERT-base-NER model. Following the process adopted (Borji, 2022), we ran the Mediapipe (Lugaresi et al., 2019) face detector twice: first on the entire image to detect faces, and thereafter on the individual detections to prune false positives, to extract faces from the real and Stable Diffusion generated images corresponding to the filtered set of claims. We then compute the FID between the set of faces extracted from the real and Stable Diffusion generated images using the clean-fid package released by (Parmar et al., 2021).
Relevance Score: [wokk in progress] Considering Figure 14.a is the original image and Figure 14.b the SD generated image, we compute the relevance scores the combination of two metrics i) CLIP

---

ET SETHEDEENT ORD WAF CENED
(a) Real Image
(b) Stable Diffusion -v2 generated image
Figure 12: In this example, for the claim: "Former President George W. Bush congratulates President-elect Joe Biden, says the election was ffuddamentlly fair' 'its outcome is clear'. claim image, and on the right where George Bush and Joe Biden are visible is the SD generated one. To assess the quality of the generated image, have calculated the pairwise FID score. First, extract the faces using Mediapipe (Lugaresi al., 2019) Face Detector (Parmar et al., 2021) pairwise. Then for a set of 500 randomly selected samples, we average out the pairwise FID scores. It is 8.67, demonstrating good match overall. The difference between the left image vs. the right is the number of faces. In such case, we take best (lowest) score as the FID score that claim. In this way, we make sure what common minimum between an Al-generated image vs. un actual news image.
image where only Joe Biden visible is original
Stable Diffusion generated image for claim. We then compute the using clean- fid
score (Radford et al., 2021b) (which measures how relevant the generated image is to the user input prompt) this is to measure the semantic similarity between text and image modality using pre-trained vision-language model CLIP, and ii) Aesthetic score (Hao al., 2022), obtained by employing linear estimator on top frozen CLIP model, that is trained by human ratings in the Aesthetic Visual Analysis [MMP12] dataset. This score represents the quality
generated image based on human evaluation pre-scores, representing what human perceives as aesthetically pleasing.
We use the relevance score introduced in (Hao et al., 2022) to measure whether the generated images are relevant to the original input prompt. We compute CLIP (Radford et al., 2021b) similarity scores to measure how relevant the generated images and the original input prompts are. The resulting relevance score is defined
feel (x,y) = Biy [min (20* gCLIP (x,iy) 5.6,0)
(1)
where, ~ (y) means sampling images iy from the text-to-image model & with y input prompt, and &CLIP (--)) stands for the CLIP similarity function.
Second, we employ aesthetic predictor discussed in (Hao al., 2022) to quantify aesthetic prefer- ences. The aesthetic predictor (Schuhmann, 2022) builds a linear estimator on top of frozen CLIP model, which is trained by human ratings in the Aesthetic Visual Analysis [MMP12] dataset. The aesthetic score is defined as:
faes (x,y) E, (x),4 [gaes (ig)
gaes (i+)
(2)
where, 8aes (.) denotes the aesthetic predictor, and iy, the images generated by the prompts y and x. respectively.

---

E.2 MOS-based quality assessment of Stable Diffusion generated images
This section delineates the process followed assess the quality of synthetically generated images, given the prompt used for generation context the human rater. Specifically, asked 10 raters to assign an integral score from (bad quality) to 5 (excellent quality) to the generated images in the context of the given prompt. Specifically, similar (Chambon et al., 2022), the scoring system was verbalized as follows:
Life-like generated image with potentially minor error elements, but practically indistinguishable from an original.
Good generated image with noticeable errors not influencing the claim's veracity assessment.
Moderate errors in the generated image with possible minor negative impacts on the claim's 's veracity assessment.
Errors leading to hallucinated lesions while still preserving the major theme of the claim but influencing the claim's 's veracity assessment.
Severe errors such as the generated image following the prompt's major theme resulting in the claim's 's veracity assessment being impossible.
The raters rated the CLIP re-ranked output for each prompt (so 500 images in total), presented in randomized fashion. As part of a pilot study, we assessed the calibration procedure and the test-retest reliability of 10 raters on a subset 500 generated images by adding a generated image twice to larger test set, similar to (Ledig et al., 2017). We observed good reliability and no significant differences between the ratings of the identical images.
5W SRL
typical SRL system first identifies verbs given sentence and then marks all the related words/phrases haven relational projection with the verb and assigns appropriate roles. Thematic roles are generally marked by standard roles defined by the Proposition Bank (generally referred to as PropBank) (Palmer et al., 2005), such as: ArgO, ArgT, Arg2, and so on. We propose a mapping mechanism to map these PropBank arguments to 5W semantic roles (refer to the conversion table 4).
Not necessarily all the Ws are present in all the sentences. To understand this sparseness, a detailed analysis of the presence of each of the 5W the sentence level has been done and reported in figure 13.

---

PropBunk Role
Why
ARGO
84.48
0.00
3.33
0.00
0.00
ARGI
10.34
53.85
0.00
0.00
0.00
ARGZ
0.00
9.89
0.00
0.00
0.00
ARG3
0.00
0.00
0.00
22 86
0.00
ARG4
?
2184
23.37
25.01
24.44
24.46
11:14
29.6
ARGM/TMA TMP
0.00
1.09
60.00
0.00
0.00
33.77
01.13
10.00
35.25
31.22
ARGM-LOC
10.00
ARGM-CAU
0.00
0.00
0.90
0.00
100,00
0.14
ARGM-ADV
4.39
20.00
0.00
0.00
ARGM MNR
0.00
3.85
0.00
8.57
0,00
20.27
13:37
0.14
0.2
21.23
9.22
10.23
ARGM-MOD
0.00
4.39
0.00
0.00
0.00
ARGM-DIR
11.2
0.24
7.30
ARGM-DIS
0.00
1.65
0.00
0.00
0.00
1088
ARGM-NRG
Figure 13: Sentence level co-occurrence of Ws.
Table 4: A mapping table from PropBank(Palmer et al., 2005)
(Arg0, Argl....) 5W (who, what, when, where, and why).
F.1 Human evaluation of 5W SRL
In this study evaluation for the 5W Aspect, based on semantic role labelling is conducted using mapping accuracy. This involves accuracy on SRL output mapped with 5Ws.
For the purpose of finding how good the mapping of 5W with semantic roles and generation of semantic roles. human annotation of 3500 data points was conducted, 500 random datapoints from the entire dataset, 500 each from FEVER (Thome et al., 2018), FavIQ (Park et al., 2021), HoVer (Jiang et al., 2020), ViTC (Schuster et al., 2021), Factify 1.0 (Mishra et al., 2022) and Factify 2.0 (Mishra et al., 2023), see table 5
G 5W QA pairs generation using language model
For the QG task, shortlisted two pre-trained top perpefforming models for question generation according to the papers with code leaderboard where the model and code have been released. These models were fine-tuned on various SQuAD datasets (Rajpurkar al., 2018) by simply appending the answer to the context. A random sampling on 352k data points was done to get of 15% of the datapoint to find the best -eattiionggn model with respect to 5W. For example, given an answer from "who" based on semantic role labeller and context from the claim, it should generate questions containing "who" and not other Ws. By modelling the claims as context and the outputs from the SRL models as answers, the process of generating 5W questions for the task of fact verification was accomplished. The pre-trained models we utilized for QG are as follows:
BART: BART (Lewis al., 2019) is denoising autoencoder for pretraining eequence-to-sequene models, trained by (i) corrupting text with an arbitrary noising function, and (ii) learning model to reconstruct the original text. BART trained to generate questions in ways: casual generation and context-based generation. For this task, we used the bart bart- squad-qg-hi variant focusing on context- based generation. This variant BART scored 24.15, 25.43, and 52.64 on the BLEU4 (Papineni et al., 2002), METEOR (Banerjee and Lavie, 2005), and ROUGE-L metrics (Lin, 2004), respectively, whereas the current state-of-the-art (SoTA) of the BART model from Textbook 2.0 scores 25.08, 26.73, and 52.55 on the same metrics.
ProphetNet: ProphetNet (Qi al., 2020) is generative model that uses multi-lingual pre-training with masked span generation to create shared latent representations across languages. It generates all the

---

masked spans together, given input sequence, and uses future n-gram loss to prevent overfitting on strong local correlations. ProphetNet is optimized through an n-step look-ahead prediction, which predicts the next n tokens based on previous context tokens at each time step, encouraging the model to explicitly plan for future tokens. It was evaluated on benchmarks for abstractive summarization and question generation tasks such as CNN/DailyMaill. Gigaword, and SQuAD 1.1 (Rajpurkar et al., 2016). ProphetNet has 12-layer encoder and 12-layer decoder with 1024 embedding/ size and 4096 feed-forward filter size. The batch size and training steps were set to 1024 and 500K, respectively, and Adam optimization was used with learning rate of 3 x 10-4 for pre-traue pre-training. The input length was set to 512 and masking done randomly in continuous spans every 64 tokens, with 15% of the total number of tokens masked.
5W QA pair generation is result of two submodules: (i) 5W SRL, and (ii) 5W-based QA pair generations. We have used pretrained models of context-based question generation models, wrapped in automation infrastructure. Contexts are the actual claim, and the answers are the Semantic Role Labeling outputs. As an example, let's consider a claim, "After April 11, 2020, there was a fatality rate of over 1.61 in Malaysia during the coronavirus pandemic". After applying SRL, we obtain the answer to the "When" of the input sentence, yielding "After April 11, 2020", Next, we feed the answer obtained in the prior step (After April 11. 2020) along with the context ("After April 11, 2020, there was fatality rate of over 1.61 in Malaysia during the coronavirus pandemic.") as the input to the model. Finally, this yields question starting with "When", which in this case is "When the COVID-19 pandemic?". We provide variety of examples in section O for readers to additionally look into.
G.1 Human evaluation of 5W SRL and QA generation
For the evaluation purpose, a random sample of 3500 data points was selected for annotation. The questions generated using the Prophetnet model were utilized for this purpose. The annotators were instructed evaluate the question-answer pairs in three dimensions: the question well formed, which means it is syntactically correct, the question is correct which means it is semantically correct with respect to the given claim, and extracted answer from the model is correct. The evaluation results for the datasets are presented in the following analysis, see table
REVER
VitantiCC
Factid 2.0
Quoiiie
FaVIQ
FEVER
HoNer
VitaminC
FactifyL.
Factily2.0
65%
Who
89%
85%
90%
87%
86%
82%
77%
What
56%
68%
78%
81%
93%
563
When
86%
90%
95%
98%
83%
75%
Where 93%
100%
90%
97%
93%
86%
Why
Table 5: Human evaluation of 5W SRI,; It observed that for most the datapoints why missing
Table 6: Human evaluation of QA generation
H 5W -a-ba validation
To design the 5W QA validation system, utilized the claims, evidence documents, and 5W questions generated by the question generation system as input. The answer generated by the 5W QG model is

---

treated the gold standard for comparison between claim and evidence. We experimented with three models, T5-3B (Raffel al., 2020), T5-Large (Raffel et al., 2020), and Bert-Large (Devlin al., 2018). The T5 is an encoder-deccode--asse language model, that treats this task as text-to-text conversion, with multiple input sequences and produces an output as text. The model is pre-trained using the C4 corpus (Raffel et al., 2020) and fine-tuned on a variety of tasks. T5-Large employs the same encoder-decoder architecture as T5-3B (Raffel et al., 2020), but with reduced number of parameters. The final model that we experimented with is the Bert-Large (Devlin et al., 2018) model, which utilizes masked language models for pre- pre-training, enabling it to handle various downstream tasks and represent both single and pairs of sentences in single token sequence. It is trained using MLM and binarized next-sentence prediction task to understand sentence relationships.
1 Selecting the best combination 5W QAG vs. 5W QA validation
We have utilized off tee-the--elf models both for 5W question-anae wnntwer generation and 5W question-answa validation. Given that the datasets using for training the models bear an obvious discrepancy in terms of the distribution characteristics compared to our data (world news) which would probably lead to generalization gap, it was essential to experimentally judge which system offered the best performance for our use-case. Instead of choosing the best system for generation vs. validation, we opted for pair-wise validation to ensure we chose the best combination. Table details our evaluation results the rows denote the QA models while the columns denote QAG models. From the results in the table, we can see that the best combination in terms of QAG and QA validation model was identified as T5-3b and ProphetNet respectively.
Prophct Not
Claim
Parughraae
Claim
Paraphras
BLEU
ROUGHEL
F1
BLEU
ROUGHEL
Recall
ROUGHEL
Recall
ROUGHEL
Recall
F1
T5:36
29.22
48.13
35.66
38.03
28.13
46.18
34.15
36.62
21.78
34.53
28.00
28.07
20.93
33.57
27.65
27.24
a5-Iarge
28.81
48.00 02
35.26
37.81
21.46
46.45
27.19
36.76
21.46
34.00
27.41
27.99
20.88
33.69
20.88
27.31
ВEKT large
46.25
34.55
36.72
27.27
44.10
32.98
26.44
19.74
Table 7: Selecting the hest combination - 5W QAG vs. 5W QA validation
Injecting adversarial assertion for fake news
The extraordinary capabilities of today's large language models to generate realistic text based prompts has had an electrifying impact the scientific community. Per (Story, 2022). "Human reviewers could only detect fake abstracts [of scientific articles] 68% of the time". Given these major advances language models, is even casier today to generate and propagate misinformation in the form of fake news that would be extremely difficult, even for human experts, to detect as false without the proper tools to verify its authenticity.
We have thus included some fake news claims synthetically generated by OPT in our dataset to provide more realistic view of news media in recent times. This adversarial attack would help build more robust fact verification models if they are able detect these fake claims.

---

Figure Representation Human vs Machine
Figure 15: Representation of Right vs Wrong verdicts
K
Additional Figues
Claim Embedding
SBERT Model
Cosine Simllary
Classier
Outputt
Document
Document Embedding
Figure 16: Text-only buseline model which takes only claim and document input.
100
Support Multimedal
80
Support_Text Text-
60
Insuffcienn_Mutttmmddaal
Insufficient Multimodal
40
Insufficient_ Text
20
Refute
5
Figure 17: Heatmap of MOS scores with 500 assessed samples for each category.

---

L
5W SRL and QA generation examples
Claim
Sports star Magic Johnson came to the hospital last month to donate blood to support the COVID-19 crisis.
Who
'Magic Johnson', 'the author with Magic Johnson', 'the author'
Who QA-pair
[( (("Who went to the hospital?", 'Magic Johnson ) ("Who worked with whom?', 'the author with Magic Johnson') ) ('Who took the photo?', 'the author')l 'donated blood", "blood donation" L(" ('Whhat did Magic Johnson do at the hospital?", 'do- nated blood'), ), ('What process Magic Johnson was part of?', 'blood donation") )]
What
What QA-pair
When
'last month, time of the post = Sept - month from August'
When QA-pair
[''When did Magic Johnson visit the hospital?", 'last month, time of the post Sept month from Au- gust')
Where
'hospital.'
Where QA-pair
[( (('Wheee did Magic Johnson pay visit to?", 'hospi- tal.')]
Why
'to donate blood.'
Why QA-pair
[("Why did Magic Johnson visit hospital?', "to do- nate blood.')]
Claim
First volunteer in UK coronavirus vaccine trial has died.
Who
'First volunteer in UK coronavirus vaccine trial'
Who QA-pair
[('who died the coronavirus vaccine trial?', 'First volunteer in UK coronavirus vaccine trial')
What
What QA-pair
(G).
When
When QA-pair
(G)
Where
Where QA-pair
[G]
Why
Why QA-pair
(C) ]

---

Claim
Kamala Harris said that the new and proposed state laws voting mean "if you are going be standing in that line for all those hours, you can't have any food."
Who
Kamala Harris
Who QA-pair
[('who said?', "Kamala Harris')]
What
"if you going to be standing in that line for all those hours, you can't have any water or any food." "
What QA-pair
[( ('What did Kamala Harris say?', "'if you are going to be standing that line for all those hours, you can't 't have any water or any food."")
When
When QA-pair
IG)
Where
'in line'
Where QA-pair
[( ("WWhere are people supposed to stand?', 'in line )]
Why
Why QA-pair
(G)
Claim
Moderna's lawsuits against Pfizer-BioN show COVID-19 vaccines were in the works before the pandemic started.
Who
Moderna lawsuits against Plizer-BioNTi
Who QA-pair
[("Who lawsuits against whom?', "Moderna lawsuits
against Pfizer- BioNTech')]
What
COVID-19 vaccines were the works before the
pandemic started
What QA-pair
[("What does the lawsuit show?', 'COVID-19 vac- cines were in the works before the pandemic started')]
When
before pandemic
When QA-pair
[("When were the COVID-19 vaccines in work?', 'before pandemic')]
Where
Where QA-pair
(G)
Why
Why QA-pair
Claim
Greta Thunberg advised the Chinese to quit Chop-
sticks save trees
Who
Greta Thunberg, the Chinese

---

Who QA-pair
[('who advised the chinese to quit chopsticks?', 'Greta Thunberg ),( ('who did greta thunberg advise to quit chopsticks?", 'the Chinese')) }
What
Chopsticks, trees
What QA-pair
[( [('what did greta thunberg advise the chinese to quit?", 'Chopsticks' (" ('what did greta thunberg want the chinese to quit chopsticks to save?', 'trees") 1
When
When QA-pair
[G. )
Where
Where QA-pair
[G []
Why
Why QA-pair
LG Table 5WQA using SRI. for above examples

---

M
Examples of adversarial fake news stories along with generated images
Stable Diffusion generated images
OPT generated documents
DAAM image explainability
Elon Musk got married to Kamala Harris in California on Thursday. The ccremony took place with 50 guests present, includ- ing Kamala's sister brother-in-lawe Apart from the White House. Kamala's ex-busband played the suxophone
were present
ceremony. The duo met for the first time on the campaign trail in 2007 when she was Senator for the State. Elon proposed to Kamala Harris in August 2019 on a beach. Kamala accepted been dating him ever since.
eto
mask
karrula
GPTZero verdict: human Avg. perplexity: 20
Per-sentence perplexity GPTZero

---

Will Smith slapped taxi driver New York. Actor Will Smith was seen getting into an altercation with laxi driver in New York City yesterday. Witnesses reported that Smith slapped the driver after an argument ensued between two. The driver, who has not been identified, was said have followed Smith's vehicle several blocks hefore the altercation hegan. "Will had just finished filming movic and was on his way to his hotel when the driver started to follow him," said a witness. The driver reportedly pulled up to Smith's car began to argue with him. Witnesses Smith saying something about driver following traffic laws. "The driver was yelling and woukdn't i leave him alone." " said another witness. After few minutes of arguing, Smith reportedll lost his temper and slapped the driver. The driver then reportedly drove away and way. Smith has yet Smith continued incident, hut police are reportedly investigating the incident. It remains what kind of churges, if any, Smith will be facing.
Wi
swith
sloped
TEU
201k
GPTZero verdict: human Avg. perplexity:
Per-sentence perplexity by GPTZero

---

India Wins FIFA World Cup' stunning upset, the Indian national football team has won the FIFA World Cup. defeating defending champions Brazil in the final. This India's everr World Cup victory comes
huge surprise the world. The Indian expected do well in this year's lournament, but they proved everyone wrong with their outstanding performances throughout the competition. They will now be hailed as heroes back home and will go down in history as one the greatest football teams ever.
GPTZero verdict: human Avg. perplexity:
Per-sentence perplexity by GPTZero

---

Vladimir Putin has been elected president of the United States, after a landslide victury over his opponents. The Russian president-elect pledged to restore order and prosperity to America, following years of political unrest and cconomic decline.
GPTZero verdict: human Avg, perplexity: 32
Per-sentence perplexity by GPTZZr

---

Joe Biden Plans to Resign 2022. President Joc Biden plans to resign in 2022. according to report from Politicu. The decision comes after months of speculation about the future of Biden, who has been in office since 2009. Biden reportedly frustrated with the Democratic Party and belioves good outside of government. Il has also expressed interest in running for president again in 2025. If he does resign. it would be the socond time that Biden has left office early he resigned as Vice President after being elected to Senate.
of the do
GPTZero verdict: human Avg. perplexity: 18
Per-sentence perplexity by GPTZero

---

Scientists in the US have announced the discovery of water Pluto. planet that has lung been considered desolate wasteland. The presence of water on Pluto is a major breakthrough for our understanding of Solar System and could lead further discoveries about other plancts. The team. lod Professor Alan Stemn from Brown University, used data from the New Horizons spacecraft to uncover evidence of water ice on the surface of Pluto. The ice appears to be concentrated in certain areas, suggesting may possible water there. questions about how life could exist Pluto and whether not might be possible explore it further. Professor Stern said: "This is exciting finding because we thought.
liquid discovery raises many
GPTZero verdict: human Avg. perplexity: 25
Per-sentence perplxiit GPTZero

---

Windows becoming open-sounce! Microsoft has announced that all future versions of Windows will be released as open-source software, meaning that anyone can access and modify the code hehind it. This move follows Microsoft's ss decision make
cloud platform open source year and likely response growing pressure from competitors such Google and Amazon. While some users may find this change exciting, others worry about the implications for Microsoft's s monopoly un desklup computing. Some fear other companies may able build better-competing products if they have Others question whether this will actually lead to more innovation, many developers are already familiar with developing for Linux and macOS.
Microsoft's codebase.
Windvows
GPTZero verdict: human Avg. perplexity: 22
Per-sentence perpexiit GPTZero

---

Tesla Motors being sold free of charge the public! This unprecedented offer will allow anyone with an interest in electric vehicles to purchuse one of Tesla's latest models, without having to worry about high monthly payments or long-term instalments. The company says this response
growing demand for environmentally friendly vehicles and their posential reduce pollution and save consumers money on fuel costs.
GPTZero verdict: human Avg, perplexity: 37
T
Per-sentence perplexity GPTZero
Table 9: A walkthrough of some synthetically generated fake news examples, with own components: (i) image, (ii) text,
and lastly, (iii) pixel-level attribution heatmap generated using DAAM (Tang et al., 2022).

---

N Examples from the PromptFake 3M dataset
ttxufiicntt
rec ercal led
Text ta not escatled
MONTTA IAME AENICTARE Siger Wire
Taet Neger Krae telked aheat chaatigg
Samary sepriaalteer, seseniing ngg
(a) A claim from Insufficiets Text category on statement made by Roger Stone.
Femile Comerst "Reathers stoters, Farth Cenotine Day. Jomes Clamur
wny varty nnneet to
(b) Another example the Insufficient Text calegory. on Bernie Sanders
BENIE
TAND OGEHERR
(c) Stable Diffusion generated images for the above claim showing Ruger Stone amidsi public.
(d) Stable Diffusion generated example for the above claim Bernie Sanders'.
DAAM heatmap Roger Stone's image generated by Stable Diffusion model.
(i) DAAM healmaps Bernie Sanders' image generated by Stable Diftusion model.
Suppor Saspar__uutrrcca Mubbioocia
Snage
Support Muttmodal
entetled
Image
Tonde and renning serice Vertee
Highs
verice 2916.
moded reaa
teal Bull
(g) An example of the Support SanporrtMultiimnne Multimadal category.
(h) An example of the Support Cupport_Multiin t_Mltiinddal category.
Examples SW QA pairs, paraphrase, and adversarial news stories

---

unteers be injected 5W QA based Explainability
First volunteer in UK coronavirus vaccine trial has died
Who claims
What claims
When claims
Where claims
Why claims
who died in the coronavirus vaccine trial?
Ans: First volunteer UK coronavirus vaccine trial
QI: who died in the coronavirus vaccine
trint?
Ans: First volunteer in UK coronavirus vaccine trial
?
?
?
verified false
not verifiable
not verifiable
not verifiable
not veritiable
Evidence
Elisa Granato of the first two vol-
trial of a po- tential COVID-19 vac- cine Oxford Univer- sity on Thursday April 23, The University Oxford News Office confirmed to Reuters on April 26 that she was "alive and well'.
Figure An illustration SW QA-based explainable fact verification system. Claim: The first volunteer in UK coronavirus vaccine trial has died
Fina wolenteerin in UK cororninis varcine mial has dbed. volmmer
passed away. Prppar 2: The inmal parocipant in the UK's tnai for commavmus vaccitte has died.
Prphr
Kingdam's conousvins vouzzne
Prphar 3: The persoa who was the fist volumeer to paticipatt in the UK coroavinu vacsine cial has died.
Prphr 4: An individual who was the fins to velumtur in the uommivis vaczine misl is the United Kinglom las pooadd away.
Prphi S: A volunter,, who was the fint to pamicipate the conousinis vaceine tntol in the UK, has decressed
Pyphar 6: parsss who was the prinary voluatees the corosavius vaccine sial is United Kingdom, has suceuated.
Prphir 7: individaal who the iniial velunteer the United Kinplom's coconatirus vauzine triil has expined
Figure 20: Claims paraphrased using GPT3.5 (Brown et al., 2020).
Finst
univenity
contimed. Osfeud Univeritty
develoged partuesship
with plameezurical safety Ammzeneea univenity is could diccmetances "independeni process concluled found monitoring recommeeeeed that should coatinue. is cunconly being tested thousands volustees
individual
satery
South Alkka..
Figure 21: An example of OPT (Zhang al., 2022) generated fake news.


Reviews:
[{"review_id": "osWGhLPpdq", "reviewer": "Reviewer_Nv99", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review, please find our detailed response below - \n\nReasons to reject: \n\n-------------------------\n\n“lacks in-depth experimental evaluations or case studies using FACTIFY 3M”\n\nThroughout the paper, we have attempted to justify with concrete, illustrative examples why each aspect of the dataset is important and its potential applications and use cases. Specifically, in section 7 (page nos. 7-8), we have presented baseline models for: multimodal entailment and 5W QA-based validation. Table 2 on pg. 8/35 shows results of our model’s performance before and after adversarial injections using our curated FACTIFY3M dataset. Please clarify if you have any specific suggestion for case studies.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper's strength lies in the introduction of FACTIFY 3M, a comprehensive and large-scale dataset for multimodal fact verification. The dataset's diverse features, including textual claims, paraphrased claims, images, visual paraphrases, and 5W question-answering pairs, provide a rich resource for researchers to explore various aspects of fact verification. The incorporation of pixel-level image heatmaps for image-text explainability enhances the interpretability of fact verification decisions. By addressing the challenge of multimodal disinformation, the paper makes a significant contribution to combating the societal crisis of disinformation and its consequences.", "reasons_to_reject": "While the paper highlights the importance of the proposed dataset, it lacks in-depth experimental evaluations or case studies using FACTIFY 3M. The authors should consider providing concrete examples and use cases to demonstrate the dataset's potential applications and showcase its utility for multimodal fact verification tasks.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper introduces a novel and valuable dataset named FACTIFY 3M for multimodal fact verification. The paper addresses the critical issue of disinformation and its impact on society. It highlights the need for efficient fact verification, especially in the context of multimodal disinformation, given the abundance of images and videos shared on social media platforms. The FACTIFY 3M dataset comprises 3 million samples and includes textual claims, paraphrased claims generated by ChatGPT, associated images, pixel-level image heatmaps, 5W question-answering pairs, and adversarial fake news stories. The dataset aims to facilitate research on various aspects of multimodal fact verification and promote explainability through visual question-answering. The paper emphasizes the dataset's potential to help detect adversarial attacks, verify complex facts, and aid journalists in fact-checking.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4IjYbjvuHE", "reviewer": "Reviewer_c4E1", "summary": "", "questions": "1.\tAdversarial attacks can also be applied to the support class. Why only target the false class?\n2.\tHas there been any manual evaluation for the use of similarity-based classification?\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review, please find our detailed response below - \n\n\nReasons to reject: \n\n---------------------------\n\n“claim to be the first ones to introduce adversarial attacks” - \n\nThank you for bringing these two papers to our notice. We will cite them and acknowledge that those previous research have already discussed adversarial attacks on fact-verification systems. We would like to extend the necessary discussion in the paper utilizing the extra page granted upon acceptance. \n\nHowever, we maintain that this aspect does not diminish the significance of this paper, which emphasized not only adversarial attack, but introducing the overall FACTIFY 3M dataset consists of -  (i) textual claims, (ii) ChatGPT-generated paraphrased claims, (iii) associated images, (iv) stable diffusion generated additional images (i.e., visual paraphrases), (v) pixel-level image heatmap to foster image-text explainability of the claim, (vi) 5W QA pairs, and (vii) adversarial fake news stories.\n\nA nuanced point to be noticed here is the referenced paper [2], which was made public only a week prior to the EMNLP submission deadline. Naturally we missed it, but will add necessary discussion in the final version given one extra page grant.\n\nReferences\n\n-----------------\n\n[1] Du, Y., Bosselut, A., & Manning, C. D. (2022, June). Synthetic disinformation attacks on automated fact verification systems. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 36, No. 10, pp. 10581-10589). https://arxiv.org/pdf/2202.09381.pdf\n\n[2] Abdelnabi, S., & Fritz, M. (2023). {Fact-Saboteurs}: A Taxonomy of Evidence Manipulation Attacks against {Fact-Verification} Systems. In 32nd USENIX Security Symposium (USENIX Security 23) (pp. 6719-6736). https://arxiv.org/pdf/2209.03755.pdf\n\n“did not evaluate the classification accuracy”\n\nWe have conducted an evaluation on a randomly sampled dataset of 1000 samples which was ongoing during submission but has recently concluded. Evaluation results affirm that our claims still sustain the accuracy. We will add this discussion in the final version. \n\n\nQuestions For The Authors:\n\n-----------------------------------------\n\n“Adversarial attacks can also be applied to the support class. Why only target the false class?” - \nThis is a very good point. We have also discussed within our team to keep this exploration for future tasks given the wide gamut of areas this paper touches upon. In this work, “we attempt to confuse the fact verification system by injecting fake examples acting as an adversarial attack.” \n\n“Has there been any manual evaluation for the use of similarity-based classification?” -\nWe have conducted an evaluation on a randomly sampled dataset of 1000 samples which was ongoing during submission but has recently concluded. Evaluation results affirm that our claims still sustain the accuracy. We will add this discussion in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Distinguishing malicious multimodal fake news has indeed become an urgent task. The FACTIFY 3M dataset proposed in this paper provides a large-scale benchmark for this task and facilitates further research.\n\n2. This paper utilizes cutting-edge technologies such as ChatGPT and Stable Diffusion for data augmentation, and generated heatmaps and 5W pairs to achieve interpretability, which is somewhat novel.\n\n3. The paper is well written and structured.\n", "reasons_to_reject": "1. The authors claim to be the first ones to introduce adversarial attacks. However, there have been related works (e.g., https://arxiv.org/abs/2209.03755, https://arxiv.org/abs/2202.09381) in the field of fact-checking that have also introduced adversarial attacks.\n\n2. The authors classify the collected data into support or neutral categories based on similarity, but they did not evaluate the classification accuracy, which I think is not convincing enough.\n", "questions_to_authors": "1.\tAdversarial attacks can also be applied to the support class. Why only target the false class?\n2.\tHas there been any manual evaluation for the use of similarity-based classification?\n", "paper_topic_and_main_contributions": "This paper focuses on multimodal fact verification. The authors have proposed FACTIFY 3M, a large-scale multimodal fact verification dataset. Specifically, they collected data from the web and existing datasets. Each sample in the dataset includes a claim, an associated image, and an evidence document. Additionally, they employed data augmentation techniques, such as using ChatGPT for textual paraphrasing, Stable Diffusion for visual paraphrasing, and generating heatmaps for image explanation. For the interpretability of fact-checking, they also generated 5W pairs (who, what, when, where, and why). The QA process helps determine which part of the claim is inconsistent with the facts. Furthermore, they evaluated the quality of generated text and images to ensure the dataset's effectiveness and trained baseline models for comparison.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The sentences at lines 337 and 350 appear to be repetitive."}, "extra_text": {}}, {"review_id": "QBn5rpNmFk", "reviewer": "Reviewer_woHT", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review, please find our detailed response below -\n\nReasons to reject:\n\n---------------------------\n\n“conduct human evaluation also on the heatmap” - \n\nAs discussed in section 4.3, pg. 5/35 in the main paper and section E.2 pg. 21/35 in the Appendix, we have reported our evaluation of the heatmap generation with Fréchet inception distance (FID) and  Mean Opinion Score (MOS) metrics. MOS is a  numerical measure of the human-judged perceived quality of artificially generated media. 10 human raters were assigned the task of evaluating MOS scores for 500 samples, as reported in fig. 7 on pg. 6/35.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The dataset is thorough and provides inspiration to the community for the following up study. \n\nThe paper is well-written.", "reasons_to_reject": "No particular reason to reject it.\n\nIt would be better to conduct human evaluation also on the heatmap for more accurate and human-centered interpretation.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes a multimodal fake news detection dataset and offers 5W QA to offer explainability. The dataset includes textual claims, chatgpt paraphrased claims, associated images, generated visual paraphrases, heatmap for explainability, and adversarial fake news stories. ", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper "FACTIFY 3M: A Multimodal Dataset for Fact Verification" introduces an intriguing and valuable contribution to the field of fact verification. It addresses the pressing issue of disinformation and its pervasive impact on society, particularly in the context of multimodal disinformation disseminated through images and videos on social media platforms. The dataset, FACTIFY 3M, is the central focus of the paper, consisting of 3 million samples encompassing textual claims, paraphrased claims, associated images, pixel-level image heatmaps, 5W question-answering pairs, and adversarial fake news stories. The primary objective of FACTIFY 3M is to support research endeavors related to multimodal fact verification and enhance interpretability through visual question-answering. The paper also emphasizes the dataset's potential applications, including detecting adversarial attacks, verifying complex facts, and assisting journalists in fact-checking.

In summary, the paper introduces a promising and relevant dataset in the field of multimodal fact verification. Its strengths lie in its comprehensiveness, the critical societal issue it addresses, and its focus on interpretability. However, to strengthen its case for acceptance, the paper should consider augmenting its content with concrete examples and experimental results demonstrating the practical value of FACTIFY 3M for researchers and practitioners engaged in fact verification tasks.
Decision: Accept-Main