Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models
Authors: Jiaying Wu
Fanxiao Li
Zihang Fu
Min-Yen Kan
Bryan Hooi
Venue: ICLR 2026 Poster (2026)
Keywords: multimodal misinformation detection
vision-language models
creator intent
Primary area: datasets and benchmarks
TL;DR: We reveal that state-of-the-art VLMs remain blind to misleading creator intent, establishing the need for intent-aware benchmarks and models as the next frontier in multimodal misinformation detection.

Abstract: The impact of multimodal misinformation arises not only from factual inaccuracies but also from the misleading narratives that creators deliberately embed. Interpreting such creator intent is therefore essential for multimodal misinformation detection (MMD) and effective information governance. To this end, we introduce DeceptionDecoded, a large-scale benchmark of 12,000 image-caption pairs grounded in trustworthy reference articles, created using an intent-guided simulation framework that models both the desired influence and the execution plan of news creators. The dataset captures both misleading and non-misleading cases, spanning manipulations across visual and textual modalities, and supports three intent-centric tasks: (1) misleading intent detection, (2) misleading source attribution, and (3) creator desire inference. We evaluate 14 state-of-the-art vision-language models (VLMs) and find that they struggle with intent reasoning, often relying on shallow cues such as surface-level alignment, stylistic polish, or heuristic authenticity signals. To bridge this, our framework systematically synthesizes data that enables models to learn implication-level intent reasoning. Models trained on DeceptionDecoded demonstrate strong transferability to real-world MMD, validating our framework as both a benchmark to diagnose VLM fragility and a data synthesis engine that provides high-quality, intent-focused resources for enhancing robustness in real-world multimodal misinformation governance.

Published as a conference paper at ICLR 2026
SEEING THROUGH DECEPTION: UNCOVERING MIS- LEADING CREATOR INTENT IN MULTIMODAL NEWS WITH VISION-LANGUAGE MODELS
Jiaying Wu Fanxiao Li Zihang Fu'
Min-Yen Kan' Bryan Hooi!
National University of Singapore Yunnan University jiayingwueu. edu (kanmy, (ecomp.p@cc....o nus. edu.
ABSTRACT
The impact of multimodal misinformation arises not only from factual inaccuracies but also from the misleading narratives that creators deliberately embed. Inter- preting such creator intent is therefore essential for multimodal misinformation detection (MMD) and effective information governance. To this end, we intro- duce DECEPTIONDECODER pairs grounded in trustworthy reference articles, created using an intent-guided simulation framework that models both the desired influence and the execution plan of news creators. The dataset captures both misleading and non-misleading cases, spanning manipulations across visual and textual modalities, and supports three ienerc-centric tasks: (1) misleading intent detection, (2) misleading source attribution, and (3) creator desire inference. We evaluate 14 state-of-ther of-the-art vision- language models (VLMss and find that they struggle with intent reasoning. often relying on shallow cues such as surface-level alignment, stylistic polish, or heuristic authenticity signals. To bridge this, our framework systematically synthesizes data that enables models to learn implication-leve intent reasoning. Models trained DECePTIONDECODED demonstrate strong transferability to real-world MMD, validating our framework as both a benchmark to diagnose VLM fragility and data synthesis engine that provides high-quality, intent-focused resources for enhancing robustness in real-world multimodal misinformation governance."
large-scale benchmark of 12,000 image-caption
Content Warning: this paper contains potentially harmful text and images.
INTRODUCTION
Multimodal misinformation, which combines persuasive text with compelling visuals, poses signifi- cant threats public understanding and can lead to serious societal harm (Zhou & Zafarani, 2020; Alam et al., 2022; Do Nascimento et al., 2022). Research in multimodal misinformation detection (MMD) has primarily focused cross-modal misalignment, typically in two forms: (1) out-of- context (OOC) misinformation (Yuan et al., 2023; Qi et al., 2024), where images and captions from unrelated events or time periods are falsely paired; and (2) multimodal media manipulation (Shao et al., 2023; Liu et al., 2025), involving subtle changes to visuals or captions that alter interpretation. Yet, current benchmarks rely on heuristic strategies such baase mismatching (Luo al., 2021) or sentiment substitutions (Sudhakar et al., 2019; Shao et al., 2023), which fail to capture the complexity of real-world multimodal misinformation.
Beyond surface-level cross-modal misalignment, the key to effective real-world misinformation governance lies in detecting and understanding misleading creator intent (Appelman et al., 2022; Jaidka et al., 2025). Many misinformation campaigns are deliberately crafted to advance specific agendas, often without the audience's awareness, yet still manage to substantially influence public opinion (Ecker et al., 2022; Broda & Strömbäck, 2024). As illustrated in Figure 1, even slight caption manipulations can significantly distort trustworthy news context (e.g., by misattributing iceberg collapse to covert military actions) which ultimately erodes institutional trust. Although preliminary efforts have explored intent interpretation (Da et al., 2021; Gabriel et al., 2022; Wang et al., 2025), they often limited to unimodal settings and fail to capture the synergy between visual and textual
Data and code are available https: //gi thub.uu..cc ctw/ http:///////iiia.o/o//////eooooo http:////eeeepepecceeeeeee

---

Published as a conference paper at ICLR 2026
Trustworthy News Context: pelieereeenn
The vitues Haley Bay as base for scientlic operations stem from its high latitude ... During the Royal Society expedition was used to make weather observation ons, studies of the formation Multimodal News Grounded in Context:
Non-Misleading Reporting: rephrase the ception es "Flat-topped icebergs in Antarctica'.
Intent: to evoke fear and suspicion about covert military activities in Antarctica, suggesting that these activities are causing significant environmental damage.
Misleading Reporting: manipulate the caption as "Massive icebergs breaking apart to secret underwater nuclear tests in Antarctica".
X
Caption: Tabular icabergs Antarctica.
Why creator intent detection matters in multimodal news. Creators can distort trustworthy news contexts by crafting intent-loaded misinformation that appears semantically aligned (e-g.. portraying broken icebergs) yet deliberately conveys false narratives (e.g.. attributing them to secret underwater nuclear tests).
Figure I:
cues. Moreover, these approaches rely on reader inference of creator intent, which are prone to inaccuracy due to the deceptive framing of such content. These limitations highlight pressing need for transparent modeling of ground-truth creator intents in multimodal content, along with systematic evaluation frameworks for detecting misleading communicative objectives.
To address this gap, we introduce an intent-guided framework for simulating intent-aware multi- modal news reporting. In this framework, intent is defined as a combination of desired influence and execution plan, two components grounded in communication strategy theory (Hallahan et al., 2007: Paul, 2011). Leveraging this formulation, we construct DECEPTIoNDECODER a large-scale benchmark of 12,000 image- caption article triplets (I, A). where 7 is the news image. T is the ac- companying caption, and A is a trustworthy reference article (i.e., "trustworthy news context") drawn from the VisualNews dataset of verified reporting (Liu al., 2021). By anchoring manipulations to verifiable news contexts, this design enables explicit labeling of misleading versus non-misleading reporting, while also ensuring coverage of ten public-interest criteria. The benchmark further models realistic malicious influence scenarios such as political polarization and public health misinformation ($3. ()3,1), all while preserving a professional news reporting tone. Data quality and realism are validated through systematic human evaluation ($3.4), and the benchmark supports three intent-centric tasks central to effective MMD: (1) Mislead- ing Intent Detection, which determines whether multimodal news piece was deliberately crafted to mislead readers; (2) Misleading Source Attribution, which identifies whether the misleading signal originates from the image or the text; and (3) Creator Desire Inference, which infers the targeted area of societal impact (e.g.. political polarization or public health disruption).
domains selected using journalistic and
Using DECEPTIONDECODED we benchmark 14 representative VLMs and find they still struggle to recognize misleading creator intent. Models often rely on superficial cues such as image- text alignment, stylistic polish, or heuristic authenticity signals, leaving them vulnerable simple adversarial manipulations including stylistic reframing (Chen & Shu, 2024; Wu et al., 2024) and persuasive prompting attacks (Zeng et al., 2024). This fragility highlights a core problem: the lack of intent-focused data severely limits VLM robustness to deception. To address this, our core technical contribution, the intent- green-- multimodal news simulation framework ($3.2 and $3.3), serves as both diagnostic benchmark and constructive data synthesis engine. By systematically synthesizing intent-loaded data, it enables models to learn implication-lewe intent reasoning, a capability central to effective MMD, Our analysis in $5.1 confirms this utility; fine-tuning on DECEPTION DECODED transfers improvements directly to three general MMD benchmarks. As evolving generative models heighten the risk of persuasive misinformation, our framework provides the high-quail intent- focused resources necessary for enhancing real-world governance and developing robust detectors that move beyond surface-level consistency.
2 RELATED WORK
Multimodal Misinformation Detection (MMD) With the increasing prevalence of persuasive visual-textual misinformation online platforms, research has expanded from detecting purely
2

---

Published as a conference paper at ICLR 2026
Content Modulity
Benchmark
Task Setup
Creator Intent
Trustworthy Context
Textual
Visual
EMU (Da at., 2021)
visual manipulation understanding
v (viewai-parccive)
Fakeddir (Nakamura al. 2020)
mixed-source MMD
MRF (Gabriel et al., 2022)
reader pereeption reasoning
Sradec-penccivved preader-penceivee
NewsINT (Wang et 2025)
news intent interpretation
PoliiFaatt (Shu al.
MMD on social media
GossipCop (Shu et al., 2020)
MMD on social media OOC misinformtioon detection mulimedia manipulation detection
NewsCLIPpiags (Lao et 2021)
DGM4 al., 2023)
MMFakebench (Liu et., 2025)
mixed mixed-ssurce source MMD
DecEPTIONDECODED (Ours)
misleading intent detection
(crentor-producee))
Table 1: Comparison between DECEPTIONDECODEDD and prior benchmarks on misinforrmation-relaeed tasks.
textual misinformation (Rashkin al., 2017; Chen & Shu, 2024; Wu al., 2024) to tackling
multimodal misinformation detection (MMD). A growing body of work emphasizes misinformation arising from subtle cross-modal misalignments, such out-of-context (OOC) misinformation (Qi et al., 2024) and multimedia manipulation (Shao et al., 2023; Liu et al., 2025). Recent advances leverage VLMs like LLaVA (Liu et al., 2023) and GPT-4o (Hurst et al., 2024) within retrieval- augmented frameworks, where the model reasons over multimodal inputs using retrieved trustworthy evidence (Khaliq et al., 2024; Xuan et al., 2024; Zhou et al., 2024; Braun al., 2024). While effective in grounding responses, these approaches largely overlook the role of news creation intent, which has been identified as a key driver of misinformation (Sharma et al., 2019). Our work builds upon this wvidence-based paradigm by evaluating VLMs on their ability to detect misleading communicative intent in multimodal news. We show that even the most advaneed models still fall short in this setting, highlighting the need for intent-awan frameworks in MMD.
Misinformation Benchmarks A variety of benchmarks have been developed to support misinformatioon-elateed research (Hansclowski et al., 2019; Yao al., 2023; Li et al., 2025; Wu et al., 2025b; Zeng et al., 2025). Representative datasets such as PolitiFact and GossipCop (Shu et al., 2020) collect naturally occurring misinformation from social media, while others approach multimodal misinformation detection (MMD) through multimedia manipulation; for example, NewsCLIPpings (Luo et al., 2021) and DGM4 (Shao et al., 2023) introduce cross-modal mismatches by substituting images or captions based on visual or textual similarity. Mixed-source datasets such Fakeddit (Nakamura et al., 2020) and MMFakeBench (Liu et al., 2025) combine naturally occurring content with synthetically manipulated samples. Preliminary efforts such as EMU (Da et al., 2021), NewsINT (Wang et al., 2025), and MRF (Gabriel et al., 2022) have begun to consider news creator intent, but these remain limited unimodal settings and typically define intent from readers' subjective perspectives, which may diverge from the actual communicative purpose of the creator. In contrast, we introduce DECEPTIONDECODED a benchmark of creator- prouured multimodal misinformation where intent is explicitly defined during content generation. A comparison of DECEPTIONDECODEE with prior datasets shown in Table 1. Beyond this, DECEPTIONDECODED is guided by established communication theories, grounded in trustworthy news contexts, and validated through human evalu- ation, providing both a principled framework for modeling creator intent and a resource with broader utility for advancing general intent-aware MMD.
3 DECEPTIONDECODED
To systematically investigate the role of misleading creator intent in multimodal news, is necessary to construct a large-scale benchmark that (1) explicitly defines diverse communicative intents across multiple domains and (2) uses these intents to guide multimodal content generation. Achieving this with real-world news data is difficult, since published articles already reflect finalized narratives and annotators can only infer intent retrospectively from a reader's perspective. To address this challenge, we introduce
(Figure 2), a benchmark of 12,000 multimodal news instances created through an intent- guiddd synthesis pipeline. Each instance is anchored in a trustworthy news context drawn from verified articles and paired with predefined creator intent configuration. which enables controlled manipulation while maintaining content realism. The quality and plausibility of the generated data further validated through rigorous human evaluation (see $3.4).
3

---

Published as a conference paper at ICLR 2026
Misleading Image (Subtie):
Misleading Text (Subtle):
Create chantic scene with emergency vehiclee and pensononal appeasing
Create caption
overwhelmed to incite fear
ruggests
sabotage
suspicion.
Trustworthy News Context:
Holiday getaway drivers face being caught up in long delays after fierce lorry fire closed busy motonway. Police heading south along remain shut for repairs for much of Saturday from junction 13 for Stafford south, after fire melted the road. Millions roads weekend broke up for the summer. Images of the lorry fire emerged on social media, while motorists and passeng gers vented their frustrations...
Caption: lorry suspected to be an act sabotage, means part of the M6 will stay closed for much of Saturday.
planning that might
expected hit many schools
Misleading Image (Significant)
Misleading Text (Significant):
Create a frantic scenc with injured people lying on the road to heighten Sear and ansiety smong readerss
caption suggests instill among public.
Multimodal Grounded Context:
Caption: Terrorist attack causes massive fire on M6, authorities urge residents to stay indoors.
Non-Misleading Image:
Create szane of a lonry on fire night to report this intident in an objective manner.
Non MinMeediedddnggg Text:
Caption:
The lorry fire means part of the M6 will stay closed for much of Saturday.
Create fire night
report this incident
objective
Due to larry fire, a section of the M6 will remain closed for most Saturday.
Caption:
Figure DECEPTIONDECODED: Overview of multimodal news curation guided by diverse simulated creator intents, covering both misleading and non-misleading cases.
3.1 SOURCE NEWS COLLECTION
We draw source data from VisualNews (Liu et al., 2021). large-scale repository of trustworthy multimodal news, ensure factual grounding. From this corpus, we select ten topic categories that provide broad societal relevance and sufficient sample richness, spanning domains such as politics, disasters, and public health. The full list of topics is provided in Appendix B.I.
To support meaningful evaluation of misinformation detection, we apply rigorous filtering criteria informed by both misinformation governance and professional journalism. From governance per- spective, we prioritize content that serves the public interest (Kruger et al., 2024), focusing on events with high societal relevance and potential for broad impact. Consistent with journalistic standards, require that selected content demonstrates professionalism, neutrality, and clarity (Maras, 2013; Farley et al., 2014), ensuring that language remains objective, unbiased, and unambiguous. To further mitigate harm, exclude instances referencing specific individuals identifiable public figures (see Appendix B.2).
This process yields 2,000 high-quality news samples, evenly distributed across the ten categories. Each instance is represented as N {I, T, A}, where is the news image, T is the accompanying caption, and A is the corresponding trusted reference article (i.e., "trustworthy news context").
3.2 CREATOR INTENT ESTABLISHMENT
Given the context provided by each original news instance N (I,T,A) A), we simulate both malicious and trustworthy content creators using GPT-4o (Hurst et al., 2024), leveraging its demonstrated capacity for social simulation (Anthis et al., 2025).
Inspired by strategic communication theories (Hallahan et al., 2007: Paul, 2011) and prior work on textual intent inference (Wang al., 2025), conceptualize creator intent (Cim) along two
4

---

Published as a conference paper at ICLR 2026
core dimensions: (1) desired influence and (2) execution plan. The desired influence specifies the societal sector(s) the creator seeks to affect (e.g., "public health and safety"), selected from a predefined taxonomy (up to three per instance; Appendix B.3). The execution plan is expressed as open-ended text describing how the creator aims to achieve the intended influence.
3.3 INTENT-GUIDED MULTIMODAL NEWS CREATION
Guided by Cint. we generate both misleading and masmmiiladding variants of each news piece obtained in $3.1 by modifying either the image or the text. Misleading variants are categorized into two levels: (1) subtle, involving minor distortions to framing or background details that nudge reader interpretation, and (2) significant, involving major alterations that substantially change the perceived message. For non-mislcading cases, the creator is prompted to faithfully paraphrase the caption or reconstruct the image while retaining consistency with the trustworthy news context A.
For textual modification, we use GPT-4o (Hurst et al., 2024) to generate captions aligned with the specified intent, as defined in $3.2, while keeping the original image unchanged. This produces samples of the form Nixt = {I, A... where I is the original image, T" is the modified caption, and A is the corresponding trustworthy news context.
For visual modification, GPT-4o is prompted to generate textual descriptions of the intended manipulation, which are then used to synthesize images with the open-source FLUX.1 [dev] model (Black Forest Labs, 2024). The resulting sample takes the form Ninage = {I', TT, A}, where is the modified image, T is the original caption, and A is the trustworthy news context.
Dataset Statistics. For each of the 2,000 filtered news samples from $3.1, we generate both misleading and non-mislcading variants for each of the six creator intent categories outlined in Figure 2, yielding total of 12,000 instances. All samples are crafted to reflect the professional tone of real-world news reporting. Illustrative data examples are provided in Appendix B.4.
3.4 HUMAN VERIFICATION DATA QUALITY
To assess the quality of news samples created in DECEPTIONDECODED through intent-guided creation, we conduct human evaluation by randomly sampling 2% of the dataset, comprising 120 text-modified and 120 image-modified instances that span all six fine-graing intent categories. The primary goal is verify whether human annotators perceive the misleading or non-misleading nature of each sample in alignment with the intent simulated by the news creators.
We recruit three graduate students annotators. Each annotator is presented with 240 label-shuffled pairs of (original, DECEPTIONDECODED -generrttd)) news pieces, where the generated version may be either misleading (M) or non-misleading (NM). Annotators are instructed to provide a binary label indicating whether the generated version deviates from the original in a way that could plausibly mislead an average reader. Details of the annotation protocol are provided in Appendix C.
We evaluate data quality using two metries: (1) accuracy, computed as agreement between the dataset labels and aggregated human annotations, and (2) inter-annotator agreement, measured by Fleiss' x (Fleiss, 1971) I). Results show high reliability: 99.2% accuracy with = 0.877 for text samples and 89.2% accuracy with ς 0.703 for image samples.
Beyond verifying the correctness of these binary labels, we also assess data realism and intent aligmment. As described in Appendix C.2, annotators judged whether each synthesized piece could plausibly appear in today's news or social media. Accuracy exceeded 98% for text and 91 % for images in standalone plausibility (judging each piece in isolation), and remained above 87% in pairwise plausibility (judging original + manipulated pairs), confirming the contextual realism of the data. Furthermore, Appendix C.3 reports evaluations whether generated manipulations remain faithful to their simulated desired influence and execution plan. Accuracy exceeded 92% for text and 86% for images across both dimensions, with substantial inter-annotator agreement.
Together, these evaluations demonstrate that DeceptionDecoder produces synthetic news samples that are not only correctly labeled as misleading or non-minn mon-miila,,ing, but also realistic in presentation and meaningfully aligned with the intended creator objectives.
5

---

Published as a conference paper at ICLR 2026
Text
Image
Model
M-Sub
M-Sig
NM
Avg.
M-Sub
M-Sig
NM
Arg.
(63.9)
(87.9)
95.4 (95.4)
83,8 (82.4)
(20.))
41.8 (33.8)
91.6 (91.6)
53.0 (48.5)
04-mini
C
72.4 (70.3)
94.0 (92.4)
88.7 (88.7)
85.0 (83.8)
46.3 (33.7)
66.5 (50.3)
78.7 (78.7)
63.8 (54.2)
Claude-377SSomeet
64.5 (55.0)
90.0
92.8 (92.8)
82.4
46.9 (43.8)
67.2 (7.2.
84.5 (84.5)
66.2 (64.1)
67.4 (66.1)
90.2 (89.7)
90.3 (90.3)
82.6 (82.0)
50.3 (31.3)
71.1 (50.6)
81.9 (81.9)
67.8 (54.6)
Semini-2.5-PPp
71.0 (70.7)
92.9 (92.5)
93.5 (93.5)
85.8 (85
43.9 (31.1)
66.4 (50.2)
86.4 (86.4)
65.6 (55.9)
72.0 (71.7)
(93.0)
(92.2)
85.7 (85.6)
53.3 (35.5)
73.9
(80.3)
GPT-4o
67.2 (60.9)
93.1 (85.8)
91.6 (91.6)
84.0 (79,4)
42.6 (39.8)
61.9 (59.9)
85.8 (85
63.4 (61.8)
70.2 (63.6)
(85.1)
83,4 (78.4)
51.7 (469)
69.3 (66.2)
78.0 (78.0)
(63.7)
Slaxde-3.5-Somme
58.0 (53.2)
88.5 (84.0)
94.8 (94.8)
80.4 (77.3)
25.3 (18.8)
42,7 (34.0)
93.8 (93 8)
53.9 (48.9)
66.8 (66.0)
92.1 (91.6)
90.3 (90.3)
83.1 (82.6)
41.9 (23.7)
63.5 (37.4)
87.0 (87.0)
64.1 (49.4)
1e5ini-1.5-Pra
84.7 (83.6)
97.0 (96.3)
76.5 (76.5)
86.1 (55.5)
54.6 (36.1)
72.4 (51.6)
74.8 (74.8)
67.3 (54.2)
C
80.5 (79.0)
94.6 (94.0)
78.0 (78.0)
84.4 (63.7)
56.9 (34.6)
74.5 (50.1)
72.2 (72.2)
67.9 (52.3)
GPT-to-mini
C
18.8 (17.2)
7,3 (5.1)
25.3 (19.8)
45.9 (43.2)
95.7 (95.7)
99.3 (99.3)
53.5 (52.0)
44.0 (41.4)
21.3 (18.1)
5.2 (4.6)
35.8 (33.2)
9.7 (9.1)
94.6 (94.6)
99.2 (99.2)
50.6 (48.6)
38.0 (37.6)
Claude-3.--Hiik
3.9 (2.7)
(12.6)
40.7 (38.4)
(99.9)
C
4.2 (3.5)
15.7 (14.1)
99.7 (99.7)
39.9 (39.1)
3.0 (2.7)
5.6 (5:2)
99.7 (99.7)
36.1 (35.9)
Qwen2.5-VL-72B
47.8 (44.1)
81.0 (75.0)
97.0 (97.0)
75.3 (72.0)
19.5 (17.0)
30.2 (26.9)
92.6 (92.6)
47.4 (45.5)
49.6 (47.4)
82.5 (79.9)
94.4 (94.4)
75.5 (73.9)
28.8 (25.3)
42.2 (38.8)
88.6 (88.0) 6)
53.2 (50.9)
Qwen2.5-VL-32BB
18.3 (17.3)
45.4 (44.0)
98.4 (98.4)
54.0 (53.2)
16.0 (15.3)
25,9 (25.2)
96.7 (96.7)
46.2 (45.7)
27.5 (27.1)
61.1 (60.8)
21.8 (17.9)
33.6 (28.3)
94.6 (94.6)
L.ama-3.2-Vision 11B
8.5 (7.9)
18.1 (13.7)
97.0 (92.7)
41.2 (38.1)
7.0 (4.7)
8.4 (7.1)
96.6 (93.9)
37.3 (35:2)
9.1 (5.7)
19.4 (9.2)
94.7 (88.5)
41.1 (34.5)
11.9 (12.2)
15.5 (13.9)
94.2 (88.0) ())
40.5 (38.0)
IntemVL-88
3.9
(89.4)
35,0 (34,4)
(8.88
(88.1)
(34.9)
C
5,5 (2.2)
6,2 (2.4)
96.3 (89.5)
36.0 (31.4)
6.0 (11.4)
(12.1
96.3 (89.4)
36.5 (37.6)
LLaVA-v1.6-7B
(.01
0.0 (0.1)
100,0 (100.0)
33,3 (33.4)
0.0
100.0 (100.0)
33.3 (33.3)
(0.0)
0.0 (0.0)
100.0 (100.0)
33,3 (33,3)
(0.0)
(0.0)
100.0 (100.0)
(33.3)
Qwen2.5-VL-7B
I 0,0 (0.0)
0.0 (0.0)
100.0 (100.0)
33,3 (33.3)
0.0 (0.0)
0.0 (0.0)
100.0 (100.0)
32.3 (33.3)
0.0
0.0 (0.0)
(100.)
33.3 (33.3)
(..00..)
100.0 (100.0)
33.3 (33.3)
Table 2: VLM accuracy (%) on misleading intent detection (black) and source attribution (colored). T
denotes the implication-oriented approach, and C the consistency-oriente approach (see $4. Green cells mark the top two models. Red cells highlight hallucinations in smaller VLMs, where misleading source (either image or text) is attributed despite the instance being predicted as non-misleading (NM).
3.5 EVALUATION TASKS AND METRICS
We design three intent-centric tasks in DECEPTIONDECODED to evaluate complementary dimensions of creator intent reasoning.
Task 1: Misleading Intent Detection. Given an input triplet of either (T, A} or T,T',AA the
objective is to predict whether the instance is misleading (M) or non-misleading (NM).
Task 2: Misleading Source Attribution. A three-way classification task in which the model must attribute the source of misleading intent to either the image or the text modality, or output "NA" the instance is non-misleading.
Task 3: Creator Desire Inference. A multi-label classification task where the model identifies the creator's intended societal influence, selecting up to three options from a predefined list (see details in Appendix B.3).
Evaluation for Tasks and 2 is based on classification accuracy, and for Task 3 on Fl score.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Using DECEPTIONDECODED we evaluate 14 representative vision-language models (VLMs) span- ning a range of sizes, model families, and access levels. These include (1) multimodal Large Reasoning Models (MLRMs), such as Claude-3.7 3.7--.nne.ttttt (Anthropic, 2025b); (2) proprietary multimodal Large Language Models (MLLMs), such as GPT-40 (Hurst et al., 2024); and (3) open-source MILMMs, such as the Qwen-2.5-VI. series (Bai et al., 2025). The complete list of models used in our experiments is provided in Table 11.
We evaluate two paradigms in misinformation detection, distinguished by their reasoning focus: (1) The iiniicatioooorientte (I) approach, which focuses the inferred implications of the news sample; and (2) The consistency-orient modalities and between the image caption pair and the trustworthy news context. Unless otherwise specifiee, all experiments on DECEPTION: DeriooDDDDDDDDD ECODED default to the consistency-oriente (C) approach. Details of the experimental setup and evaluation prompts are provided in Appendix D.1.
(C) approach, which assesses consistency both across

---

Published as a conference paper at ICLR 2026
Text
Image
M-Sub (Text)
M-Sig (Text)
Model
Model
M-Sub
M-Sig
Avg.
M-Sub
M-Sig
Avg.
1+T
T+A
Full
I+T
T+A
Full
04-mini
60.9
79.7
70.3
57.7
56.3
47.0
GPT-4o
21.8
63.4
70.2
51.6
88.4
Clamme-..-Soomer
61.1
82.6
21.2
44.7
65.4
55.1
GPT-4o-mini
3.6
36.8
18.8
10.2
64.4
45.9
GPT-4g
$7.0
75.9
66.5
39.9
55.4
47.7
Owen2.5-72BB
10.6
53.6
49.6
24.6
83.8
82.5
Gomini-
Qwenn2.5-326
6.2
35.8
18.4
67.2
GPI-4o-mini
0.8
3.2
2,0
1.7
2.6
2.2
Clande H.5-Haiku
0.0
0.0
0.0
0.0
0.0
0.0
Qwen22.5 V-L-255-VL-72B VL-
44,1
74.0
59.1
24.6
37.6
31.1
Table 4: VLM performance (Acc. on misleading intent prediction under partial modality settings, Results suggest that smaller VLMs tend to over-rely image-text consistency when detecting misleading intent. (I+T: image & caption only; T+A: caption & reference article only; Full: modalities.)
Owen2.5 VL-32B
0.0
0.2
Table 3: Performance (F1 () on creator desire inference. Green cells indicate the best-performing models, and red cells zero performance.
4.2
MAIN RESULTS
Consistency-Orieet Reasoning Outperforms Implication- Orreetted Reasoning, As shown in Table 2, the consistency-oriented approach generally yields stronger performance than the implication- oriented approach. A likely explanation is that misleading news is often created through deliberate content manipulation. Detecting unsubstantiated inconsistencies either between the image and its caption or between the a more reliable signal for identifying misleading intent.
pair and the trustworthy news context provides models with
VLMs Struggle to Reason About Misleading Creator Intent. As shown in Table 2, even state- of-the-art MLRMs such as Claude-3.7 exhibit limited performance in detecting and attributing misleading creator intent. Performance drops further on the more challenging task of creator desire inference (Table 3), indicating a deeper difficulty in reasoning about underlying communicative goals.
These findings raise concerning implications. Our evaluation reflects realistic deployment scenarios in which readers encounter subtle yet misleading representations of otherwise trustworthy multimodal news and must rely on automated systems for support. The consistent underperformance of VLMs under such conditions highlights the urgent need to examine the features and heuristics these models rely on when assessing authenticity, motivating our in-depth analyses presented in $4.3.
4.3 VLMS STRUGGLE BEYOND SURFACE-LEVER SIGNALS
VLMS Can Be Misled by Image-Text Semantic Consistency. Although misleading intent is often unsupported when compared with the trustworthy news context, the manipulated image and caption typically appear internally consistent (Figure 1). Ideally, models should integrate information across all three modalitics: image, text, and article, to identify such deception.
To investigate this, we evaluate VLMs under two partial input settings: (1) Image + Text, and (2) Text + Article. As shown in Table 4, the stronger GPT-4o model follows the expected trend, achieving its best performance when all modalities are available. In contrast, smaller VLMs deviate from this ideal. For example, GPT-4o-mini performs better when detecting misleading text by comparing it directly with the article, yet its performance drops notably once the misleading image is added. This suggests that internal image-text consistency can mask deception, exposing a weakness in models that lack deeper reasoning about the credibility of information across modalities.
N S.Sub (loe)) Professional Siyte N-Sub (-sabh Mixleading Sty
MiSig (lent) Professional Sty M-dig (heat) Madeeding Style
725
Acc. (K))
20
GPT-40
Cloude-3.5
GPT-4o
Claude 3.5 Halku
Owen2.5 VL
Owen2 V..
-Sonnet
umini
.788
-328
Figure 3: Misleading intent detection performance of VLMMs misleading text presented either in profes- sional tone (as used the original DECEPTIONDE- CODED setting) versus an explicitly misleading style.
VLMs Fail to Detect Misleading Text Framed in Credible-Sounding Styles. Prior work on textual misinformation detection (Chen & Shu, 2024; Wu et al., 2024) shows that large language models (LLMs) often rely on stylistic cues rather than factual correctness when making veracity judgments. To test whether this bias extends to the
7

---

Published as a conference paper at ICLR 2026
All Misleading (Acc. %)
All Non- Mon-MMsleaainn (Ace. %)
Model
Original
w/ Helpful Hint
w/ Adversarial Hint
Original
w/ Helpful Hint
w/ Adversarial Hint
04-mini
69.8
(+11.2)
46,8 (-23.0)
83.7
90.2 (46.5)
68.4
Claude:3.7
69.7
(+18.8)
49.3 (-20.4)
86.1
(+7.8)
58.5 (-27.6)
Gemini-2.5-Pro
73.0
84.6 (+11.6)
43.2 (-29.8)
86.3
99.8 (+13.5)
72.8 (-13.5)
GPT-4o
71.1
87.6 (+16.5)
29.3 (-4188
82.3
97.4 (+11.1)
61.1 (-21.2)
Claude-3.5- S.nnet
66.1
74.6 (+8.5)
47.5 (-18.6)
88.6
95.6 (+7.0)
82.4 (-6.2)
Gemini-1.5-PP
76.6
96.3 (+19.7)
54,3 (-22.3)
75.1
97.3 (+:22.2)
51.0 (-24.1)
GPT-4o-mini
30.4
84,9 (+54.5)
6.4 (-24.0)
95.1
99.8 (+4.7)
50.4 (-44.7)
Claude-3.5-
7.1
34.5 (427.4)
0.8 (-6.3)
99.7
100.0 (40.3)
95.7 (-4.0)
Qwen2.5-VI. -2.
50.7
81.9 (+31.2)
30.2 (:20.5)
91.5
98.2 (+6.7)
61.6 (-29.9) 3.4 (-92.5)
Qwen2.5-VL-32B
35.3
99.5 (+64.2)
(-34.1)
95.9
99.6 (+3.7)
Table VLM performance on misleading intent detection with spurious authenticity hints, averaged over misleading and non-misleading cases. Helpful/ / Adversarial hint construction details are provided in $4.3.
multimodal setting, we reframe misleading captions in DECEPTIONDECODER from professional, authoritative tones to more explicitly deceptive or sensational ones (see details in Appendix D.2). As illustrated in Figure 3, we observe the same trend: VLMs are more likely to misclassify mis- leading content when it is presented in a polished, credible style. This suggests that current models may be disproportionately influenced by surface-level linguistic cues, posing serious risks when misinformation is deliberately crafted to appear authoritative.
VLMs React Strongly to Spurious Authenticity Cues in Prompts. We further test whether models are influenced by high-level framing cues that are not grounded in the input content. To do so, we inject two types of authenticity-related hints into the prompts (see Appendix D.3): (1) Skeptical, which presumes the news piece may contain intentional distortions (helpful for misleading samples but adversarial for non-misleading ones), and (2) Trusting, which assumes the content is reliable (reversing the helpful/ad shifts in performance: accuracy improves when the hint aligns with the ground truth but degrades sharply when it conflicts, even though the prompt explicitly instructs the model to rely on visual and textual evidence (Figure 15). This behavior suggests that VLMs often treat the hint as an overriding authoritative signal rather than as contextual information to be weighed, likely reflecting combination of (1) primacy-driven positional bias (Wang et al., 2023), where information provided in the earlier positions of the prompt is overweighted, and (2) instruction authority bias (Sharma et al., 2024), where the hint is interpreted as a directive about the expected outcome. This reveals a core behavioral limitation: VLMs fail to follow the intended reasoning sequence and instead shortcut to the easiest and most salient cue, resulting in brittle performance and indicating that current models not yet capable of stable, evidence- evidence-grounded intent detection.
roles). As shown in Table 5, these cues produce large and systematic
5 DISCUSSION
5.1 BROADER UTILITY OF INTENT-GUIDED MISINFORMATION SIMULATION
central contribution of DECEPTIONDECCDEED its focus misleading creator intent, aspect largely overlooked in existing multimodal misin- formation benchmarks yet essential for developing robust MMD systems. This focus is instantiated in our intent-guided multimodal news simulation framework ($3.2 and $3.3), which we position not only as diagnostic tool but as a construc- tive method for synthesizing high-fidelity, intent- labeled data needed to enhance VLM robustness. To assess its effectiveness a method for enhance- ment, we fine- tune LLaVA-v1.66 1..-7- and Qwen2.5-VL-7B on 6,000 DECEPTION DECODED samples (labels: binary misleading/ non-mishe non-misleading) and evaluate transfer performance three representative multimodal misinformation benchmarks: MMFakeBench (Liu et al., 2025), Fakeddit (Nakamura al., 2020), and NekeNewsNet (Shu al., 2020), with dataset and training details in Appendix D.5.
Model (MacroFF %)
MMFakeleench
Fakoddit
FakeNewsNe
Liova-vl.6-7 1.678
44.41
34.63
43,73
EFT (6k samples)
52.37 (47.56)
39.43 (44.80)
65.22 (421.4)
Qwes2,5.VI.3
27.96
32.69
44,87
(wl simples)
58.66
(-300,70)
41.96 (41.27)
68.59 (22372)
Table 6: Utility of DECEPTIONDECODER for enhancing multimodal misinformation detection. Fine-tuning VLMs DeceptionDecodeR leads to consistent performance improvements across three popular benchmarks for general MMD.
8

---

Published as a conference paper at ICLR 2026
Trustworthy News Context:
Misleading Image (FLUX (FLUX-generaaee generetteddd
Misleading Image (GPT-mmge-1 genereted
Firulightters battle
Powerf
Calforria, Sunday, grown nearly 20,000 scres
Create
group of gallered ed anound Lake
Eloabeth in the early morning
hours Sunday near Lake
Hughes, California, visblee
background. people appear sparkklers
colebrating
Caption (NMI: Fire surrounds Lake Elizbett in the early momning hours Sunday near Lake Hugher, California.
Caption INM): Fire surrounds Lake Elzzbeth in the early moming hour Sunday near Lake Hughhe, Califomii.
small
creating hemosphere
VLM Prediction:
Non-Misleading (sourcw: N(A)
VLM Prodiction: Misleading (Sounce: Image!
despite surrouncing
VLM Reasoning:
The and caption consistent reforence article regarding the Poweroouse fire Lake Hughos.
VIM Remsnning:
The news piece has been intentionally distored the image shows
holding sparklers
near a fire which is inconsistent with the serious nature of the fre described the reference artiche.
Figure 4: Case Study: a VLM (GPT-40-mini) fails detect misleading creator intent in a PLUX--eenerai image. The model overlooks the presence crowd with fireworks, an unsuhstantiated addition given the powerhouse fire described in both the news context and the caption. In contrast, the state-of of-the-artt GPT GPT-image-1 model produces more vivid depiction of fireworks, enabling the correct prediction.
As shown in Table 6, fine-tuning yields consistent and substantial macro-F1 gains across all three benchmarks, indicating that training creator-intent focused data compels models to learn more generalizable principles of implication-level intent reasoning and leads to stronger MMD.
5.2 IMAGE REALISM OUTPACES VLMS' MMD CAPABILITIES
DECEPTIONDECODED incorporates reasonably high-quality images generated with FLUX.1 I [dev] (Black Forest Labs. 2024), as confirmed by human validation in $3.4 and Appendix C. To examine the implications of recent advances in image generation, we conducted controlled evaluation using subset of 200 sampled misleading instances, replacing FLUX outputs with higher-fidelipy versions generated by GPT-image-1 (OpenAI, 2025b).
As shown in Table 8 and Figure 4, several VLMs exhibit modest, consistent gains when evaluated on GPT-generated images, likely because higher visual fidelity clarifies intent-related cues (for ex- ample, more legible, vivid depictions of fireworks and crowds than the FLUX-genentee These observations indicate that stronger visu- als make certain manipulations easier to de- tect. However, despite these improvements, mod- cling the underlying communicative intent in mul- timodal news remains difficult. Surface-level fi- delity insufficient; effective detection requires reasoning explicitly about creator objectives.
Model
FLIX
M-Sub (Image) GiP--magee
FLUUX
M-Sig (Image)
GiTT-image
GPT-oo
$1.0
66.0
69.0
$4.0
(lande-3.5.Some
42.5
64.0
65.0
80.0
GPI-4o-mini
47.5
Claude-3.5-
3.0
9.0
6.0
18.0
variant).
Qwen2.5-728 5-728
22.5
53.5
33.0
$7.0
Qwea2.5-32B
30.0
37.0
46.5
46.0
Table 7: Misleading intent detection performance of VLMs images generated by different models. Evaluation is conducted on 200-sample pilot
At the same time, the growing realism of GPT- images introduces substantial risks. When high-fidelitt visuals align convincingly with misleading textual narratives, the resulting deception becomes more subtle and more difficult to detect. In our experiments, prompting GPT-image-1 to simulate malicious creators, without explicitly requesting misinformation, is able bypass the established safety guardrail and generate convincing manipulations for all 200 cases. The combination of scalability and realism heightens risk, particularly since VLMs remain weak at intent detection even under these enhanced conditions.
5.3 EXTENSION OF INTENT-GUIDED MISINFORMATION SIMULATION TO IMAGE EDITING
As powerful image editing models emerge (Google, 2025b; Wu et al., 2025a), fine fine-grained modifica- tions such as localized object changes or subtle expression shifts introduce a growing risk: creators convey misleading intent through minimal yet scalable visual distortion.

---

Published as a conference paper at ICLR 2026
Trustworthy News Context:
Misleading Image (Subtle):
Bentain's gr growwng economy and stronger pound have triggeeed new wave of lebour migration from Roland other wsst Europear countiee official figures suggees lateed Office market
Using the prodiced image the checking a parspor- long queue utiilly background
National stics number east Europeans working Britain rose to B95, 020 December...
UKK Border
appcor diverzo torms age athnicity
eegressione impattennee
and body language discomorrt. Keep offeer the inmmediste scene and him escctly
Multimodal News Grounded in Context:
Image (Significant):
UK Border
Using the provided image of the off checking paraport, please modify the passport to display a large. prominent "REJECTED stamp across in Frourne the stamp is clearly vishie and looks offical Keep evcrything elsc ececily the same.
UK Border
MEI
Caption: Parsports airportin
Extension of our cntent-guided multimodal misinformation simulation framework ($3.2 and $3.3) to image editing, where misleading creator intent is introduced through high-fidelity visual modifications by Google's ' Nano Banana model (gemini-2. 2...ii.......lf.t 5-flash-image)
Figure 5:
To examine whether intent-guided multimodal misinformation simulation framework ($3.2 and $3.3) can extend to this realistic threat model, we conduct an exploratory study using Google's 's recently released Nano Banana model - 5-flass-image) (Google, 2025b). We first specify the creator's communicative intent (desired influence and execution plan), then generate image editing instructions aligned with that intent. An illustrative example is shown in Figure 5. Following the exploratory setup in $5.2, we produce 200 edited samples for each of the M-Sub (Image) and M- Sig (Image) classes, derived from 200 source news items spanning 10 domains. Using the same consistency-oriente detection prompt in our main experiments (Figure 15), we compare five VLMS on Nano Bananaedited images and GPT-ggenraatd images (Table 8).
From these results, we draw three observations. First, VLMs remain weak at detecting visually conveyed deception, and edited images are consis- tently more challenging than high-fidelity, GPT- generated images. Second, the difficulty posed by edited samples demonstrates that our intent- guided simulation framework naturally extends diverse and realistic manipulation types. Third, consistent with our observation in $5.2, image editing can bypass established safety guardrails of Nano Banana and produce intent-aligned manipu- lations at near pear-perfect success rates. Together, these findings highlight the practical relevance of our intent-guided simulation framework and highlight the heightened risks of intent-driven, implication- level visual manipulation.
M-Sub (Image) GPT-gen
M-Sig (Image) GPT
Model
Edited
Edited
GPT
66.0
48.0
84.0
61.0
GPT--to-mini
47.5
22.5
41.5
Claude 3.5 Haiku
9.0
6.0
18.0
11.5
Qwen2.5-728
53.5
32.0
52.5
Qwen2 5.3BB
37.0
21.5
46.0
41.0
Table 8: Mislcading intent detection performance of VLMs generated and edited images. Evalua- tion is conducted on 200-sample pilot set.
CONCLUSION
We establish creator intent as a central perspective for multimodal misinformation and introduce DECEPTIONDECODED. a large-scale benchmark grounded in trustworthy real-world news contexts. Through extensive experiments on 14 VLMs, we reveal that current -fte-offthhe-rt models are weak at intent reasoning, often relying on spurious cues such as cross- moosl consistency, stylistic polish, and heuristic authenticity signals. To overcome this fragility, our framework systematically synthesizes intent- data that compels models to learn implication-level intent reasoning. Demonstrating strong transferability to real-world general MMD, DECEPTIONDECODED serves as both a diagnostic benchmark and constructive data synthesis engine, laying the foundation for intent- modeling and robust governance of multimodal misinformation at scale.2
2I Limitations and future work are discussed in Appendix A.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We express our gratitude to Jianyang Gu for valuable discussions that helped refine the framing of this work. We also thank the anonymous reviewers for their constructive feedback. This research is supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier (TI 251RES2507, TI 251RES2508) and MOE AcRF TIER Grant (MOE-MOET3202-000)
ETHICS STATEMENT
The intent-guided simulation multimodal misinformation in DECEPTIONDECODER necessarily involves strategies that could be misused to generate misleading content with VLMs and open-source image generation models (e.g., FLUX). Nonetheless, given the pressing challenge of detecting misleading creator intent in multimodal news, it is important to transparently acknowledge these risks while reporting empirical findings on the strengths and limitations of current ofate-of-the-art VLMs.
Our benchmark is developed solely to highlight the important yet underexplored aspect of modeling malicious creator intent MMD, and advance the understanding of current system limitations. To minimize harm, we explicitly avoid manipulating news content involving specific, identifiable individuals; all simulated content based on anonymized or synthetic entities. To further mitigate misuse, will open-source the dataset and evaluation scripts but refrain from releasing the specific generation prompts that could be repurposed for deception. Access will be restricted to verified researchers under binding usage agreement, and all data collection complies with the terms of service of the underlying models and platforms. Overall, we prioritize transparency while enforcing safeguards against misuse.
Importantly, our work does not seek to promote or enforce a single interpretation of news events. Instead, it focuses detecting creator intent when multimodal content distorts the implications of a trustworthy news context by embedding unsubstantiated claims or implications (see example in Figure and Figure 2). This framing distinguishes between responsible reporting, which preserves neutrality, and misleading reporting, which introduces unsupported interpretations. Ultimately, our goal is to support the development of intent-aware systems that safeguard trustworthy reporting while respecting the rights of information consumers to hold diverse viewpoints.
LLM USAGE STATEMENT
LLMs were used solely for minor grammatical editing of the manuscript. All methodological design, analysis, and writing were conducted under full human oversight and responsibility.
REPRODUCIBILITY STATEMENT
We provide concrete examples DECEPTIONDRCODED data Figure and Figure with addi- tional samples included in the GitHub repository for more comprehensive overview. For safety considerations (see Ethics Statement), we do not publicly release the specific prompts used in the final step of intent-guided multimodal news creation ($3.3), Nevertheless, the -giidt framework underlying dataset construction is described detail in $3, ensuring that the creation process can be reproduced.
The full experimental setup for VLMs, covering all reported experiments, is provided in $4 and Appendix D. including model specifications, training objectives, evaluation prompts, and hyperpa- rameters. Collectively, these resources enable reproducibility of both dataset design and experimental evaluation while maintaining safeguards against potential misuse.
11

---

Published as a conference paper at ICLR 2026
REFERENCES
Firoj Alam, Stefano Cresci, Tanmoy Chakraborty, Fabrizio Silvestri, Dimiter Dimitrov, Giovanni Da San Martino, Shaden Shaar, Hamed Firooz, and Preslav Nakov. A survey on multimodal disinformation detection. In Proceedings of the 29th International Conference on Computational Linguistics, pp. 6625-6643, 2022.
Jacy Reese Anthis, Ryan Liu, Sean M Richardson, Austin C Kozlowski, Bernard Koch, James Evans, Erik Brynjolfsson, and Michael Bernstein. Lim social simulations are promising research method. arXiv preprint 2r..ix:2504.022::4 2025.
Anthropic. Claude 3.5 sonnet. httpps///wwt http:://wwwwwwwa http:///ww..atttooooocccoo//eees sonnet, 2025a.
Anthropic. Claude 3.7 sonnet and claude code.
httts://www.attropicc.ccm///
claude-3-7-sonne 2025b.
Anthropic. Claude 3.5 haiku.
comp.///ccccmmlaaeeeaaiaaau 2025c
Naomi Appelman, Stephan Dreyer, Pranav Manjesh Bidare, and Keno C Potthast. Truth, intention and harm: Conceptual challenges for disinnorrmation-targeeed governance. Internet Policy Review, 2022.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
Black Forest Labs. Flux.
2024.
Tobias Braun, Mark Rothermel, Marcus Rohrbach, and Anna Rohrbach. Defame: Dynamic evidence- based fact-checking with multimodal experts. arXiv preprint arXiv:2412.10510, 2024.
Elena Broda and Jesper Strömbäck. Misinformation, disinformation, and fake news: lessons from an interdisciplinary, systematic literature review. Annals of the International Communication Association, 48(2):139- 166, 2024.
Canyu Chen and Kai Shu. Can LLM-generated misinformation be detected? In The Twelfth International Conference on Learning Representations, 2024.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et Expanding performance boundaries open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.055771 2024.
Jeff Da, Maxwell Forbes, Rowan Zellers, Anthony Zheng, Jena D. Hwang, Antoine Bosselut, and Yejin Choi. Edited media understanding frames: Reasoning about the intent and implications of visual misinformation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference Natural Language Processing (Volume 1: Long Papers), pp. 2006200200022200000 2039, 2021.
Israel Junior Borges Do Nascimento, Ana Beatriz Pizarro, Jussara M Almeida, Natasha Azzopardi- Muscat, Marcos André Gonçalves, Maria Björklund, and David Novillo-Ortiz. Infodemies and health misinformation: systematic review of reviews. Bulletin of the World Health Organization, 100(9):544, 2022.
Ullrich KH Ecker, Stephan Lewandowsky, John Cook, Philipp Schmid, Lisa K Fazio, Nadia Brashier, Panayiota Kendeou, Emily Vraga, and Michelle A Amazeen. The psychological drivers of misinformation belief and its resistance to correction. Nature Reviews Psychology, 1(1):13-29, 2022.
Elizabeth Farley, Fiona Grady, Dean S Miller, Rory O'Connor, Howard Schneider, Michael Spikes, Constantia Constantinou, et al. Spj code of ethics. The Power of Images, 2014.
Joseph L Fleiss. Measuring nominal scale agreement among many raters. Psychological bulletin, 76 (5):378, 1971.
12

---

Published as a conference paper at ICLR 2026
Saadia Gabriel, Skyler Hallinan, Maarten Sap, Pemi Nguyen, Franziska Roesner, Eunsol Choi, and Yejin Choi. Misinfo reaction frames: Reasoning about readers' reactions to news headlines. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume
I: Long Papers), pp. 3108-3127, 2022.
Petko Georgiev, Ving lan Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprini arXiv:2403.055300 2024.
Google. Gemini 2.5 pro. https:/ htlp:://deeemnndd /tgpg///gg//eecooogge/////////// 2025a.
Google. Introducing gemini 2.5 flash image (nano banana). https::/ddeellpprs. com//en/introducing http://trrrucciing-einn--------- gemini-2--- flash-lshh-hh flash-imgee/ 2025..
gog.geeel....rs.gggoll.e..oo googleblog.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama herd of models. arXiv preprini urXiv:2407.2178. 2024.
Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as--judge. arXiv preprint 2024.
Kirk Hallahan, Derina Holtzhausen, Betteke Van Ruler, Dejan Verčič, and Krishnamurthy Sriramesh. Defining strategic communication. International journal of strategic communication, 1(1):3-35, 2007.
Andreas Hanselowski, Christian Stab, Claudia Schulz, Zile Li, and Iryna Gurevych. A richly annotated corpus for different tasks in automated fact-checking. arXiv preprint arXiv:1911.01214, 2019.
Renee Hobbs. Mind over media: Propaganda education for digital age. WW Norton & Company, 2020.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Kokil Jaidka, Tsuhan Chen, Simon Chesterman, Wynne Hsu, Min- Kan, Mohan Kankanhalli, Mong Li Lee, Gyula Seres, Terence Sim, Araz Taeihagh, et al. Misinformation, disinformation, and generative ai: Implications for perception and policy. Digital Government: Research and Practice, 6(1):1-15, 2025.
Mohammed Abdul Khaliq, Paul Yu-Chun Chang, Mingyang Ma. Bernhard Pflugfelder, and Filip Miletić. RAGAR, your falsehood radar: RAG-augmented reasoning for political fact-checking using multimodal large language models. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pp. 280-296, 2024.
Ariel Kruger, Morgan Saletta, Atif Ahmad, and Piers Howe. Structured expert elicitation on disinformation, misinformation, and malign influence: Barriers, strategies, and opportunities. Harvard Kennedy School Misinformation Review, 2024.
Fanxiao Li, Jiaying Wu, Tingchao Fu, Yunyun Dong, Binghing Song, and Wei Zhou. Drifting away from truth: Genai-driven news diversity challenges Iylm-based misinformation detection. arXiv 2025.
preprint arXiv:2508.1271
Fuxiao Liu, Yinghan Wang, Tianlu Wang, and Vicente Ordonez. Visual news: Benchmark and challenges in news image captioning. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.). Proceedings the 2021 Conference Empirical Methods in Natural Language Processing, pp. 6766-667777 Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.18663/v122021.emm www.p-main.nn.... URL https://aclanthol.oo
2021 http222111..mmmmaaamm. emnlp-main. 542/.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
13

---

Published as a conference paper at ICLR 2026
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava- next: Improved reasoning, oct, and world knowledge, January 2024. URL htttps://laavss github. wwt.io///ooo/eeeeeeee 2024-01 30 1lava-next/.
Xuannan Liu, Zekun Li, Pei Pei Li, Huaibo Huang, Shuhan Xia, Xing Cui, Linzhi Huang, Weihong Deng, and Zhaofeng He. MMFakebench: A mixed-source multimodal misinformation detection benchmark for LVLMs. In The Thirteenth International Conference on Learning Representations, 2025. URL https: httpss///ppenrewee httppp/www...e.eeeeeeoooonodddda net.:oorummid-D22nnooeee
Grace Luo, Trevor Darrell, and Anna Rohrbach. NewsCLIPpings: Automatic Generation of Out- of-Context Multimodal Media. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6801-6817, 2021.
Steven Maras. Objectivity in journalism. John Wiley & Sons. 2013.
Sadiq Muhammed T and Saji K Mathew. The disaster of misinformation: review of research in social media. International journal of data science and analytics, 13(4):271-285, 2022.
Kai Nakamura, Sharon Levy, and William Yang Wang. Fakeddit: A new multimodal benchmark dataset for fine-grained fake news detection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 6149-6157, 2020.
OpenAI. GPT-4o mini: Advancing Cost-Efficient Intelligence, July 2024.
OpenAl. Introducing openai o3 and 04-mini. introducing--3-and------ 2025a.
OpenAI. Introducing 4o image generation. introducingg44--iage- www.aaaggggggeeeeooonn.... 2025b.
https://opemai.ccm/
Christopher Paul. Strategic communication: Origins, concepts, and current debates. Bloomsbury Publishing USA, 2011.
Peng Qi, Zehong Yan, Wynne Hsu, and Mong Li Lee. Sniffer: Multimodal large language model for explainable out-of-context misinformation detection. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 13052-13062 2024.
Qwen Team. Qwen2.5-vl, January 2025. URL htttt::t///phnili Baalll...itte github. ho/blog//wee2.55v1/
Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2931-2937, 2017.
Rui Shao, Tianxing Wu, and Ziwei Liu. Detecting and grounding multi-modal media manipulation.
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Karishma Sharma, Feng Qian, He Jiang, Natali Ruchansky, Ming Zhang, and Yan Liu. Combating fake news: survey on identification and mitigation techniques. ACM transactions on intelligent systems and technology (TIST), 10(3):1444 42, 2019.
Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel Bowman, Esin DURMUS, Zac HatfieldDoode Scott R Johnston, Shauna M Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. Towards understanding sycophancy in language models. In The Twelfth International Conference on Learning Representations, 2024.
Kai Shu, Deepak Mahudeswaran. Suhang Wang, Dongwon Lee, and Huan Liu. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news social media. Big data, 8(3): 2020.
Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Maheswaran. Transforming delete, retrieve, generate approach for controlled text style transfer. arXiv preprint arXiv: 1908. 09368, 2019.
14

---

Published as a conference paper at ICLR 2026
Yiwei Wang, Yujun Cai, Muhao Chen, Yuxuan Liang, and Bryan Hooi. Primacy effect of ChatGPT. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. pp. 108- 115, 2023.
Zhengjia Wang, Danding Wang, Qiang Sheng, Juan Cao, Siyuan Ma, and Haonan Cheng. Exploring news intent and its application: A theory-driven approach. Information Processing Management, 62(6))104229, 2025.
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan. Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report, 2025a.
Jiaying Wu, Jiafeng Guo, and Bryan Hooi. Fake news in sheep's clothing: Robust fake news detection against Immempowered style attacks. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3367-3378, 2024.
Jiaying Wu, Zihang Fu, Haonan Wang, Fanxiao Li, Jiafeng Guo, Preslav Nakov, and Min-Yes Kan. Beyond the crowd: Lim-augmented community notes for governing health misinformation. arXiv preprint arXiv:2510. 11423, 2025b.
Keyang Xuan, Li Yi, Fan Yang. Ruochen Wu, Yi R Fung, and Heng Ji. Lemma: towards Ivlm- enhanced multimodal misinformation detection with external knowledge augmentation. arXiv preprint 0rXiv:2402.11943, 2024.
Barry Menglong Yao, Aditya Shah, Lichao Sun, Jin-Hee Cho, and Lifu Huang. End-to-end multi- modal fact-checking and explanation generation: A challenging dataset and models. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2733-2743, 2023.
Xin Yuan, Jie Guo, Weidong Qiu, Zheng Huang, and Shujun Li. Support or refute: Analyzing the stance of evidence to detect out-of--cottce mis- and disinformation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 4268-44 0288-480, 2023.
Yi Zeng, Hongpeng Lin, Jingwen Zhang. Diyi Yang. Ruoxi Jia, and Weiyan Shi. How johnny can persuade LLMS to jailbreak them: Rethinking persuasion challenge Al safety by humanizing LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. -4322- 2024.
Zhi Zeng, Jiaying Wu, Minnan Luo, Xiangzheng Kong, Zihan Ma, Guang Dai, and Qinghua Zheng. Understand, refine and summarize: Multi-view knowledge progressive enhancement learning for fake news video detection. In Proceedings of the 33rd ACM International Conference Multimedia, pp. 9216-9222 9065,225,22 2025.
Xinyi Zhou and Reza Zafarani. A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys (CSUR), 53(5):1 2020.
Xinyi Zhou, Ashish Sharma, Amy X Zhang, and Tim Althoff. Correcting misinformation on social media with large language model. arXiv preprint arXiv:2403. 2024.
A LIMITATIONS AND FUTURE WORK
Through DECEPTION we establish structured and scalable framework for understanding misleading creator intent multimodal news, representing meaningful first step toward intent-aware multimodal misinformation governance. While our benchmark makes a substantial contribution, it also opens several promising directions for extending its value and impact.
First, although DECEPTIONDECODED supports intent interpretation through tasks such mis- leading intent detection and creator desire inference, fully open-ended intent articulation and evaluation remains open challenge. Future work could explore adaptive prompting strategies or leverage emerging paradigms such as LLLM-as-a-Judee (Gu et al., 2024) to assess generated intent representations in more flexible, reliable, and scalable ways.
15

---

Published as a conference paper at ICLR 2026
Second, while current focus is intent detection and interpretation, the downstream societal influence of misinformation is not directly measured. In real-world settings, the effectiveness of misleading narratives depends not only creator intent but also how audiences interpret and respond. Incorporating social grounding (such as user feedback signals, comment chains, or engage- ment patterns) would provide richer context for evaluating the reach and impact of misinformation, and for calibrating model responses accordingly.
Overall, DECEPTIONDECODE] provides principled foundation for systematic, intent-aware eval- uation of VLMs, while also charting path toward more socially grounded, resilient, and scalable approaches for governing multimodal misinformation.
B DECEPTIONDECODED DETAILS
B.1 NEWS TOPICS
VisualNews (Liu et al., 2021) provides fine-grained topic annotations for each news article. For DECEPTIONDECODED, we select 10 major topic categories based on sample abundance, topical diversity, and real-world relevance.
The selected categories are:
'world',
'science_re teannnceetchhoooyy....
'politics_ ceciccss_eecttosss.
'environment',
pessiness_econony
"technology's 'disaster accident',
'conflittattack"
'us-news",
Crealth_medicine aoadicine_cnvironmentt
These topics form the foundation for generating diverse and realistic multimodal misinformation instances in DECEPTIONDECODED.
B.2 SOURCE DATA ACQUISITION
In $3.1, we obtain high-quality source data for DECEPTIONDECODED with the prompt outlined in Figure 6.
Data filtering in DECEPTIONDECODED
You are senior journalist tasked with evaluating news caption based on five key criteria:
Public Interest: Does the caption describe significant event likely to capture public attention? Professionalism: Does maintain professional, objective tone?
Neutrality: Is it free from political, ethnic, gender, or religious bias?
Anonymity: Does it avoid mentioning any specific person's 's name or national/world leaders?
Clarity: Is the message clearly conveyed without ambiguity, misunderstanding, or controversy?
Conclude with single word "Yes" the caption meets five criteria or single word "No" fails any.
Caption: (caption) Response:
Figure 6: Prompt for selecting VisualNews (Liu et al., 2021) samples as source data for DECEPTION- DECODED.
16

---

Published as a conference paper at ICLR 2026
B.3 ASPECTS OF DESIRED INFLUENCE
We define set of major desired aspects of societal influence to facilitate creator intent establishment $3.2. The aspects are listed as follows:
Political Polarization,
Social Polarization,
Cultural and Religious Polarization, Economic Misinformation, Public Health and Safety, Environmental and Scientific Polarization, Geopolitical and International Relations, Psychological and Emotional Manipulation
During creator intent formulation ($3.2), we explicitly prompt the model to construct an intent using up three desired influence aspects selected from the aforementioned list of eight. This combinatorial design encourages diverse intent configurations and enables DECEPTIONDECODER to generate nuanced, high-fidelity misleading content across ten widely covered news domains ($B.1).
Table presents the distribution of desired influence aspects among misleading samples in DE- CEPTIONDECODED. The distribution closely aligns with patterns reported in prior misinformation studies (Hobbs, 2020; Muhammed T & Mathew, 2022), with "Psychological and Emotional Manip- ulation" and "Public Hcalth and Safety" emerging as the most prevalent categories. This suggests that the LLM reflects real-world salience rather than defaulting to uniform or mode-collapsed out- puts. Moreover, the appearance of every intent type across all four manipulation settings (subtle vs. significant; text vs. visual) indicates that these intents manifest through varied modalities and transformation strategies, rather than being confined single style of manipulation.
Intent Type
M-Sub (Text)
M-Sig (Text)
M-Sub (Image)
M-Sig (Image)
Political Polarization
581
732
462
566
Social Polarization
467
517
490
519
Cultural and Religious Polarization
29
27
19
9
Economic Misinformation
309
268
272
191
Public Health and Safety
913
942
1,011
1,096
Environmental and Scientific Polarization
467
430
531
509
Geopolitical and International Relations Psychological and Emotional Manipulation
319
501
305
390
1,744
1,911
1,886
1,944
Table 9: Distribution of desired influence aspects across modalities and manipulation types. M-Sub: Misleading (Subtle), M-Sig: Misleading (Significant). Class definitions are detailed in $3.3.
B.4 DECEPTIONDECODED DATA EXAMPLES
To illustrate how intent-guided modifications manifest in practice, we present two representative DECEPTIONDECODED examples: a textual modification in Figure where the caption manipu- lated to convey misleading intent while the image remains unchanged, and visual modification in Figure 8, where the image is altered to mislead while the caption is preserved. These examples highlight how creator intent can be embedded through either modality, reinforcing the multimodal nature of the benchmark.
C HUMAN EVALUATION DETAILS
DECEPTIONDECODED benefits from our proposed synthetic framework for intent-guided multimodal news creation, which grounds manipulations trustworthy news contexts while explicitly modeling creator intent. To validate that the generated data are realistic, relevant, and aligned with intended objectives, conduct series of human evaluation studies. These evaluations provide assurance that the benchmark both reliable for systematic model testing and meaningful for real-world misinformation scenarios.
17

---

Published as a conference paper at ICLR 2026
Original Caption (from trustworthy media]: Locals place flowers on the edge the crash site cordon outside the village of Grabovo.
Misleading (Subtlel:
Locals gather near the wreckage, seemingly indifferent the crash site outside the village of Grabovo.
Misleading (Significantl:
Locals celebrate near the remains of a downed aircraft, showing no regard the tragedy that occurred.
Trustworthy News Context:
Armed rebel soldiers black access to main crash site Malaysia Airlines flight MH17, which crashed 17 July near Grabovo ... Ukrainian state emergency service employees search for bodies amongst the wreckage...
Minleliindding:
Residents lay flowers at the perimeter of the crash site near the village Grabovo.
Figure 7: An example from DECEPTIONDECODED illustrating intentional caption-based misrepre- sentation. Given an image of locals placing flowers at the perimeter of the MH17 crash site near Grabovo (left), the original trustworthy caption accurately describes a memorial act. The creator then introduces two distorted variants: (1) a subtle misleading caption that reframes the gathering as "seemingly indifferent" to the tragedy, injecting unsupported negative interpretation; and (2) a significant misleading caption that falsely claims locals are "celebrating" near the wreckage, fabricating intent and moral disregard. Both distortions alter the perceived social meaning of the scene without changing the image itself, thereby misleading readers about the news context and the subjects' intentions.
Misleading (Subtle)
CDMX
CDM
MEXICO
Misleading (Significant)
Caption:
The letters of the new acronym of Mexico City are seen a square of the city.
Trustworthy News Context:
Mexico has rechristened its capital city, embracing the name by which it is known worldwide, but causing conundrum for residents who for decades have referred to the sprawling megalopolis the Federal District or "DF"
GDMX
Non- Misleading
An example from DECEPTIONDECODED illustrating intentional visual misrepresentation. Starting from a neutral scene of the "CDMX" landmark in Mexico City (left), the creator introduces two distorted variants: (top right) subtly manipulated version that alters the visual tone and context while preserving plausibility, and (bottom right) significantly distorted image that fabricates chaotic protest scene with "MEXICO" sign and national flag. Both edits imply social unrest and a turbulent atmosphere not supported by the original scene, thereby misleading readers about the underlying context.
Figure 8:
Beyond the central evaluation of misleading non-misleading perception in $3.4 (detailed in Appendix C.I), we further assess two additional aspects: data realism (Appendix C.2) and intent alignment (Appendix C.3). Across all studies, three independent graduate student annotators
18

---

Published as a conference paper at ICLR 2026
Original News
Modified News
Caption: The lorry fire means part the M6 stay closed for much of Saturday.
Caption: The lorry fire means part the M6 will stay closed for much Saturday.
Instruction:
You are provided with two multimodal news pieces reporting the event, consisting image and corresponding caption (I eLeft: original, Right: modified). Does the modified news piece distort the meaning implication that could mislead readers? Annotate with "0" if the modified image Not
of the original news piece Mislending, or "I" if Misleading.
Important Note: Focus only on the multimodal news content and implications. The presence of Al- Al-gencrated artifacts
should not be considered misleading unless they significantly change the implication or meaning conveyed by the image.
Human evaluation instructions for assessing the data quality in DECEPTIONDECODED (detailed in $3.4).
Figure 9:
followed the same setup described in $3.4. Accuracy is reported based on majority-vote labels, and inter-annotator agreement is measured using Fleiss' x. Together, these evaluations provide comprehensive view of data validity from multiple perspectives.
C.1 HUMAN EVALUATION OF DATA QUALITY
This evaluation (results detailed in $3.4) establishes whether human annotators perceive the generated manipulations misleading or non-misleading in line with the intended labels. Validating this perception essential because ensures that the benchmark captures manipulative strategies in ways that are interpretable to human readers, not just detectable by models.
We illustrate the annotation process Figure 9. As reported in $3.4, human evaluation results show high reliability of DECEPTIONDECODER data and substantial agreement between the three independent annotators: 99.2% accuracy with x 0.877 for text samples and 89.2% accuracy with 0.703 for image samples.
Discussion on Edge Cases of Annotator Disagreement To better understand the subtleties of intent interpretation, we examine DECEPTIONDECODED cases where annotators disagree on whether modified multimodal news item conveys misleading intent. These edge cases surface the inherent subjectivity of intent judgment and offer valuable guidance for future intent-oriented misinformation research.
Across the dataset, annotators disagreed on 26 out of 120 image-modified samples and 10 out 120 poption-modified samples. Notably, 32 of these 36 disagreements occurred within the Misleading (Subtle) class, underscoring that ambiguity is concentrated in fine-grained manipulations rather than overtly deceptive ones.
qualitative review reveals three primary sources of disagreement:
Intent Ambiguity (Figures 10, 11): The modification is deliberately mild, blurring the line between stylistic emphasis and intentional exaggeration, mirroring real-world ambiguity in creator intent.
19

---

Published as a conference paper at ICLR 2026
Original News Image:
Trustworthy Context:
HHigh winds and rain cause floods in California]
Original News Caption:
creates of weter while passing through flooded Thursday in Mill Valley, California.
The modified image non-misleading.
Justification: The inclusion of people carrying belongings walking on flooded roads is common and plausible occurrence during a flood, interpreting the adcitions merely illustrative realism.
Modified News Image: Misleading (Subtle)
The modified image misleading.
Justification: The combined additions of smoke (implying gas leakage/wwddee wideepreaddddd destruction) and displaced individuals constitutes intentional exaggeration of the severity and scope localized flood event.
Figure 10: An example of intent ambiguity in DecepTioNDeCOUE human evaluation (Ap- pendix C), where annotators diverge whether the modified image is misleading due to differing interpretations of consequences of the reported flood in California.
Annotator Oversight (Figure 12): A small number of cases reflect clear labeling mistakes rather than genuine interpretive differences.
Data Generation Issues (Figure 13):
Rare artifacts introduced during content generation (e.g.. typographical anomalies in images) lead to confusion unrelated to intent.
As summarized in Table 10, the dominant source of disagreement arises from intentional subtlety rather than annotation noise or data flaws. These edge cases highlight the nuanced and often implicit ways misleading implications are conveyed, suggesting promising directions for future work on graded intent modeling and fine-grained governance mechanisms.
Category
Intent Ambiguity
Annotator Oversight
Data Generation Issues
Image Modifications (n = 26)
23
2
Text Modifications (n 10)
10
Table 10: Distribution of reasons behind annotator disagreement in DEcEPTIONDECODED human evaluation (Appendix C) on the misleading intent of multimodal news items.
C.2 HUMAN EVALUATION OF DATA REALISM
While label accuracy validates whether manipulations are recognized as misleading, it is equally important to whether the content feels plausible humans. generated samples are unrealistic, then models trained on them would not generalize to real-world misinformation scenarios.
Standalone Plausibility. Setup: Annotators judged whether synthesized news piece could plausibly appear in today's news social media.
Question: "After reading the synthesized news piece, could this plausibly appear on today's 's news or social media?"
120 text-synthesized samples (80 misleading, 40 non-misleading): 98.3% accuracy, ς: 0.854. 120 image-synthesized samples (80 misleading, 40 non-misleading): 91.7% accuracy, 0.815.
20

---

Published as a conference paper at ICLR 2026
Original News Image:
Trustworthy News Context:
The number wolves Yellowstone National Park continued to grow amid a push to remove the gray wolves from federal protection. New figures from park show there ware least 104 walves in packs in Yellowstone in December last year, inclucligg nine breeding pairs, with 40 pups surviving until the end the year...
modified caption misleading.
The origina news caption merely reported wolf pack of grey timber wolves at Yellowstone National Park. Their presence alone does imply they were sighted near popular hiking trails, nor that visitors were at increased risk.
Justification:
Original Caption:
A wolf pack of gray timber wolves on snow at Yellowstone National Park.
The modified caption non-misleading
Justification: Yell Yellowstone National Park widely known for many popular hiking trails. Observing an entire pack of wolves could reasonably suggest that wolves are prevalent in the park, which may imply danger visitors.
Modified News Caption: Misleading (Subtie)
Dangerous wolf pack spotted near popular hiking trails, pasing threat to visitors ot Yellowstone National Park.
Figure 11:
An example of intent ambiguity in DECEPTIONDECODEL human evaluation (Ap- pendix C), where annotators diverge on whether the modified caption is misleading due to differing interpretations of the reported wolf occurrence in Yellowstone National Park.
Original news image:
Trustworthy Context:
first snow survey of Sierra Nevade snowpack more snow than year this time, but not enough impact the California drought. The Department of Water Resources conducted the survey Tuesdaay about 90 miles cast Sacramento. Frank Gohrke, chicf of the Califoriia Cooperative Snow Surveys Program, said there were 21 21.3in
found
ICY A
the ground. Following recent storms, he said, the survey found more snow in the mountains than last year at this time
Original News Caption:
Manipulated image: Misleading (Significant)
sign warns drivers of the road conditions on Donner Pass Road near Soda Springs Califomia December 2014.
Annotatien Error:
One annotator mistakenly rated this as Non-Misleading.
EXTREME HEAT
When consider the image-caption pair, both original and significantly manipulated images are factually ly eligned with the ception (ile., "a sign warns drivers of road conditions", However, the "Extreme Heat" sign image significantly distorts the implications of trustworthy news context, which centers snow survey and should therefore imply an icy road condition.
Figure 12: An example of annotator oversight in DECEPTIONDECODED human evaluation (Appendix C), where disagreement arises from a clear mislabeling by one annotator rather than genuine interpretive ambiguity.
High standalone plausibility confirms that samples resemble authentic news-style reporting and could realistically challenge automated systems.
Pairwise Plausibility. Setup:
Annotators compared original manipulated pairs and judged whether the manipulated version could plausibly appear as deceptive reporting.
Question: "Given the original manipulated pair, could the manipulated version plausibly appear on today's ' news or social media, created by an untrustworthy publisher to mislead the audience?"
80 misleading text-synthesized samples: 97.5% accuracy, λ = 0.853.
80 misleading image-synthesized samples: 87.5% accuracy, k 0.663.
21

---

Published as a conference paper at ICLR 2026
Original News Image:
Modified News Image: Non-Misleading
Original News Caption:
Commuters wait for buses outside the Foggy Bottom station in Washington to take them to the Rosslyn station in Virginia.
Trustworthy News Context:
...There earlier incident with arcing insulator the Foggy Bottorn station that caused major delays, Metro warned there are residual delays both
Generation Issue:
The unusual spelling on the sign caused two annotators to label modifedd image as Misleading.
the
An example of a data generation issue in DECEPTIONDECODED where a typographical artifact the generated sign led two annotators judge the instance as misleading.
Figure 13:
Strong pairwise plausibility shows that manipulations are subtle enough to deceive when contrasted with authentic reporting, reinforcing their utility for evaluating fine- grained detection capabilities.
C.3 HUMAN EVALUATION OF INTENT ALIGNMENT
Finally, beyond realism and label validity, we test whether manipulations remain faithful to the intended creator objectives ($3.2). Verifying intent alignment is necessary, since without alignment to the desired influence or execution plan, the dataset would fail to capture the communicative strategies central to misinformation governance.
Annotators judged whether manipulations matched the creator's specified societal target (e.g,, political polarization, public health).
Desire Alignment. Setup:
Question: "After reviewing the original + manipulated pair, is the manipulation consistent with the creator's established desire to exert negative influence on the specified societal aspect(s): (list of desire aspects)
80 misleading text-synthesized samples: 92.5% accuracy, K = 0.732.
80 misleading image-synthesized samples: 86.3% accuracy, k = 0.636.
Execution Plan Alignment. Setup: Annotators judged whether manipulations followed the specified execution plan of the creator.
Question:
"After reviewing the original + manipulated pair, is the manipulation consistent with the creator's s established execution plan: {plan)?"
80 misleading text-synthesized samples: 93.8% accuracy, λ 0.772.
80 misleading image-synthesized samples: 86.3% accuracy, K 0.643.
These evaluations confirm that manipulations are not only plausible but also systematically aligned with simulated intent. This alignment ensures that DECEPTIONDECODED captures the communicative mechanisms underlying misinformation, making a robust benchmark for intent-aware multimodal misinformation detection.
22

---

Published as a conference paper at ICLR 2026
Model
Model Card
04-mini (OpenAI, 2025a)
04-mini-2025-04
Claude-3.7-Somee (Anthropic, 2025b)
claude--277-00mee--22222211
Gemini-2.5-Por (Google, 2025a)
gemmin--2.--prrrpreeiieee--
GPT-4o (Hurst et al., 2024)
gpt-4o-2902-008-6
Claude-3.5- 3.5-onnnettt (Anthropic, 2025a)
clauee335--ommett222412222
Gemini-1.5-PP (Georgiev et al., 2024)
gemini-pro-1.5
GPT-4o-mini (OpenAI, 2024)
Bpt=44omiini-2224-77--1
Claude-3.5-Haiku (Anthropic, 2025c)
claude-3-5-taikk--22441222
Qwen2.5 ------------------- (Qwen Team, 2025)
(wen2.5-vv172b-instte
Qwen2.5- ----L--2B--nntttteee (Qwen Team, 2025)
qwen2. 1wen2..-v1132biinsstue
LL 1oma--3.--iis L Vissiii----nsttcct-- 11B IB (Grattafiori et al., 2024)
1lama-3. 1.ma--32-111 2emmmm..---bbiiiii--isttrruu
Intern VL22-B (Chen et al., 2024)
InternVL2-88
LLaVA-1.6-7B (Liu al., 2024)
1lava-v1.6-mistraal-77 Qwen2 (wen2-5-vL-7-Innttruce
Qwen2.5-VLL-B-Instmee (Qwen Team, 2025)
Table 11: Model cards for all VLMs evaluated throughout our work.
D VLM EVALUATION DETAILS
D.1 EXPERIMENTAL SETUP
With DECEPTIONDECODED systematically evaluate 14 representative VLMs spanning different families, sizes, and access levels. Detailed model names and versions are listed in Table 11. To ensure reproducibility and stability, all evaluations removes sampling variance and allows fair comparisons across models. Evaluation prompts for the implication-orientedd and consistency-oriente approaches provided in Appendix D.4.
once with temperature set to O. This
D.2 CAPTION STYLE REFRAMING FORMULATION
Stylistic cues can strongly influence model predictions when the underlying factual content is un- changed. To probe this vulnerability, we reframe original captions in DECEPTIONDECODED-WW which adopt professional, neutral news tone-into manipulative styles that sound sensational, biased, or emotional. This allows us to test whether models mistake stylistic credibility for factual reliability.
The experiments in $4.3 and Table examine whether VLMs over-rely on linguistic polish or credi- bility markers instead of reasoning about creator intent and cross cross-modal modal consistency at implication level. The exact prompt used for stylistic reframing shown below:
You are given factual news caption. Your task is to rewrite it in an unreliable style: that is, make it sound sensational, biased, emotional, or manipulative, while preserving the original content and factual meaning. Do not add or remove any factual information. Only change the tone and style reflect how misinformation might manipulate readers through language. Original Caption: (caption) Unreliable-style Caption:
Table 12 shows side-by-side comparisons of professional (default in DECEPTIONDECCDEE) vs. unreliable styles, highlighting how superficial stylistic changes can shift perceived reliability.
D.3 AUTHENTICITYRRELTEE HINT FORMULATION
Prompt-level framing can introduce subtle biases that shape model predictions even before any multimodal evidence is processed. To probe this vulnerability, we inject authenticity-related hints that explicitly bias models toward or against assuming reliability. We design two types of hints, described in $4.3, and insert them directly into the evaluation prompts simulate prior assumptions about content trustworthiness, This setup allows us to test whether VLMs rely on ungrounded framing cues rather than reasoning from the actual news content.
23

---

Published as a conference paper at ICLR 2026
Professional Style
Unreliable Style
A firefighter struggles alone against massive wild- fire due to severe budget cuts, highlighting gov- ernment's failure to support emergency services.
lone firefighter heroically battles a monstrous wildfire, abandoned by negligent government that has slashed emergency service budgets, leaving our communities to burn!
Massive explosion at secret military facility near Santa Clarita raises questions about government transparency.
Shocking blast at hidden military base near Santa Clarita sparks outrage and demands for govemn- ment accountability!
Terroiist attack causes massive fire M6, author- ities urge residents stay indoors.
Catastrophic terrrist attack ignites inferno M6, authorities desperately plead with terrified resi- dents to hide indoors!
Table 12: Examples contrasting professional and unreliable reporting styles.
The two hint variants are defined as follows:
Hint (Skeptical):
"Hint: The news piece likely to contain intentional distortions. Carefully analyze the image and caption to identify any misleading or manipulative elements."
Hint (Trusting):
"Hint: The news piece is sourced from reputable outlets and is therefore unlikely to contain intentional distortions."
D.4 EVALUATION PROMPTS
In $4.2, evaluate 14 VLMs the task of detecting misleading creator intent. The full evaluation prompts are illustrated in Figure 14 (implication-orientee reasoning) and Figure 15 (consistency-orientee reasoning).
D.5 EVALUATING BROADER UTILITY OF INTENT-GUIDED MISINFORMATION SIMULATION
In $, evaluate the broader utility of our proposed intent-guided multimodal misinformation simulation framework ($3.2 and $3.3). We show that by fine-tuning VLMs on DECEPTION the models acquire generalizable capability detect misleading creator intent, which, reported in Table 6, translates into significant performance gains on multiple multimodal misinformation benchmarks and thus improved robustness to general multimodal deception.
In this section, we describe the data construction and training setup used for these fine-tuning and transfer experiments.
D.5.1 TRAINING SETUP
As described in $5.1, we evaluate the broader utility of creator intent identification for general multimodal misinformation detection by fully fine-tuning two open- source VLMs on DECEPTION- DECODED: LLa VA-v1.6-Misttrrl-77 and Qwen-2.5-VL-777 Each model is trained for one epoch on 6,000 DeceptionnDecoded samples (six intent classes, 1,000 per class) and then tested on MM- FakeBench (Liu et al., 2025), general MMD benchmark. For evaluation, we use MMPakeBBechhs validation split of 1,000 instances (700 fake, 300 real). To prevent label-order bias, the training samples are randomly shuffled, since the raw files list all non-misleading cases first and all misleading cases last.
Each training instance consists of an image paired with its caption, inserted into a fixed prompt (shown verbatim in Figure 16, with (caption] substituted). The model produces binary "Yes/No" prediction, where "Yes" indicates misleading and "No" indicates non-misleading. Supervision is answer-only: all prompt tokens are masked, and the loss is computed solely the terminal answer
24

---

Published as a conference paper at ICLR 2026
Implication-Oriente Approach (I)
Your task determine whether news piece has been intentionally distorted mislead readers. The news piece consists an image its caption: [caption]. You are also provided trustworhy reference article related to the same event: (article).
Follow the structured steps below and strictly adhere to the output format. Important Note: Focus only the content of the news piece. The use of Al tools to assist image creation does not, by itself, indicate distortion.
Answer the following questions:
QI. Has the news piece been intentionally distorted by its creator to mislead readers, such that it conveys biased, controversial, manipulated message?
Respond with one
following: [Yes. Noj. Then provide brief explanation.
Q2. What is the source of the manipulation text or image?
If answer to Q1 is "Yes", answer with one of the following: [Text, Image].
If answer Q1 is "No", answer: "NA".
Q3. What does the distortion suggest about the news publisher's intent?
answer to Q1 "Yes":
Primary Influence (Desire): Select up to three from the following list, ranked by relevance: [Political Polarization, Social Polarization. Cultural and Religious Polarization, Economic Misinformation, Public Health and Safety, Envirommental and Scientific Polarization, Geopolitical and International Relations, Psychological and Emotional Manipulation). If answer to QI is "No", respond with "NA" for Q3.
Output Format: Respond in the following exact JSON format, preserving the structure, punctuation, and delimiters. Do not add any extra line breaks formatting outside this structure: "ppred": "Answer Q1, followed by brief explanation", "source": "Answer to Q2".
"desire": "Answer to Q3"
Figure 14: Prompt for the implication-oriented (I) approach to detecting misleading creator intent, utilized in $4.2.
span, located using a robust matcher tolerant to whitespace and punctuation variations. Inputs are left-padded and encoded without truncation preserve full context.
Optimization employs an effective batch size of 32 (per-device batch size with gradient accumulation 32), a learning rate of 1 10-5, and 6 mixed precision. This configuration designed to isolate the decision boundary, align the training objective with the evaluation setting, and maintain consistent training capacity across both model families.
D.5.2 TRANSFER DATASETS
We fine-tune VL.Ms on 6,000 samples from DECEPTIONDRCODED (evenly distributed across six intent classes), through which the models acquire the capability to detect misleading creator intent. We then evaluate how this capability contributes to enhancing general MMD performance by testing the fine-tuned models on three widely used multimodal misinformation datasets:
MMFakeBench (Liu et al., 2025) is a comprehensive benchmark for mixed-source MMD that encompasses 12 subtypes of forgery. We use its official validation set, which contains 700 Fake samples and 300 Real samples, to evaluate the VLMs fine-tuned on DECEPTIONDECODED
Fakeddit (Nakamura et al., 2020) is real-world MMD dataset sourced from Reddit. Using its binary Real/Fake labels, we randomly sample 1,000 instances (500 Fake and 500 Real) to evaluate the VLMs fine-tuned on DECEPTIONDECODED
FakeNewsNet (Shu et al., 2020) is a real-world MMD benchmark that contains multimodal from sources: PolitiFact, which covers political news, and GossipCop. which covers entertainment news. We randomly sample 1,000 instances (500 Fake and 500 Real) to evaluate the
VLMs fine-tuned on DECEPTIONDECODED
25

---

Published as a conference paper at ICLR 2026
Consistency-Orientee Approach (C)
Your task determine whether news piece has been intentionally distorted mislead readers.
The news piece consists an image its caption: [caption]. You are also provided trustworthy reference article related to the same event: (article).
Follow the structured steps below and strictly adhere to the output format. Important Note: Focus only the content of the news piece. The use of Al tools to assist image creation does not, by itself, indicate distortion.
Answer the following questions:
Q1. Has the news piece been intentionally distorted by its creator to mislead readers, such that there exists inconsistencies between the image and caption, between the news piece and the reference article?
Respond with one the following: [Yss, No). Then provide a brief explanation.
Q2. What is the source of the manipulation text or image?
If answer Q1 is "Yes", answer with one of the following: [Text, Image].
If answer Q1 "No", answer: "NA".
Q3. What does distortion suggest about the news publisher's intent?
If answer to Q1 is "Yes":
Primary Influence (Desire): Select up to three from the following list, ranked by relevance: [Political Polarization, Social Polarization, Cultural and Religious Polarization, Economic Misinformation, Public Health and Safety, Environmental and Scientific Polarization, Geopolitical and International Relations, Psychological and Emotional Manipulation]. If answer to Q1 is "No", respond with "NA" for Q3.
Output Format: Respond in the following exact JSON format, preserving the structure, punctuation, and delimiters. not any extra line breaks or formatting outside this structure: "prod"; "Answer to Q1, followed by brief explanation", "source"; "Answer to Q2",
"desire": "Answer to Q3")
Prompt for the consistency-oriented (C) approach detecting misleading creator intent, utilized in $4.2.
Figure 15:
Prompt for VLM Fine-Tuning
You given piece of multimodal news content consisting of an image and its caption: (caption). ).
Determine whether this image-caption pair contains misinformation. Specifically:
1. Identify any factual inaccuracies in the caption.
2. Check for inconsistencies or mismatches between the image and the caption. Answer "Yes" if the content contains misinformation, "No" if it does not. Important: Focus only on the news content itself. The presence Al-generated artifacts stylistic cues alone does not indicate misinformation.
The answer is:
Figure 16: Prompt for the VLM fine-tuning experiments in $5.1 and Appendix D.5.
26


Reviews:
[{"review_id": "8qt7sRcLNz", "reviewer": "Reviewer_cLyQ", "summary": "This paper introduces DECEPTIONDECODED, a large-scale benchmark for understanding and detecting misleading creator intent in multimodal news. This work centers on modeling the combination of desired influence and execution plan behind deceptive news creation. The benchmark comprises 12,000 image–caption–article triplets, each grounded in trustworthy news contexts from VisualNews and simulated through intent-guided generation using GPT-4o and FLUX.1. It supports three intent-centric tasks: (1) misleading intent detection, (2) misleading source attribution, and (3) creator desire inference. Comprehensive evaluations of 14 VLMs reveal that even leading models struggle to reason about creator intent. Fine-tuning on DECEPTIONDECODED improves performance on external MMD benchmarks (e.g., MMFakeBench), underscoring its transferability.", "questions": "1.Have the authors compared DECEPTIONDECODED against existing unimodal intent detection models? Can the dataset still challenge models when only one modality is provided? This would clarify the necessity and distinct value of multimodal intent reasoning.\n2.Could the framework be extended to include dual-modality misleading constructions. For instance, simultaneously altering both caption and image? Would such jointly deceptive instances better reflect real misinformation and raise the benchmark’s difficulty?\n3.Have the authors quantified the number of samples for each misleading-intent category or verified that the dataset is balanced across intent types and news domains? If certain intents (e.g., political or health-related deception) dominate, how might this imbalance affect model evaluation and the generality of conclusions about “intent reasoning” performance?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The shift from content-level to intent-level misinformation detection is a underexplored direction that extends beyond factual correctness to communicative objectives.\n2.The benchmark construction pipeline is well thought out: grounded in verified contexts, controlled through creator-intent configurations, and validated via human evaluation with high inter-annotator agreement\n3.Evaluating 14 diverse VLMs across multiple reasoning paradigms (implication vs. consistency-oriented) offers valuable cross-model insights into weaknesses in current multimodal reasoning.", "weaknesses": "1.Absence of joint multimodal deception synthesis. Misleading instances are generated by modifying either text or image, but never both simultaneously. This simplification limits the ecological realism and potential difficulty of the dataset, as real-world misinformation often involves coordinated visual–textual deception.\n2.Conceptual ambiguity of AI synthesis traces in the news context. The authors explicitly exclude AI-generated artifacts from being considered evidence of deception. However, this assumption may be problematic in the journalism setting, since the very use of AI-generated visuals in news reporting could itself indicate misleading or inauthentic intent. \n3.Over-reliance on generative synthesis instead of precise editing. The misleading visuals are produced entirely through image generation (e.g., FLUX.1), without exploring fine-grained editing operations such as subtle facial expression shifts, contextual object replacements, or localized semantic changes. This may reduce the dataset’s ability to capture nuanced real-world manipulations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3M7P7dMXHb", "reviewer": "Reviewer_cUUE", "summary": "This paper introduces DECEPTIONDECODED, a novel benchmark designed to evaluate Vision-Language Models (VLMs) in detecting creator intent behind misleading multimodal news content. The dataset is constructed using a synthetic, intent-guided framework that generates manipulations grounded in real news, ensuring relevance and control over deception intent. The study evaluates state-of-the-art VLMs under various input conditions (e.g., image+text, text+article) and with authenticity cues (helpful or adversarial hints).", "questions": "1 The paper is purely diagnostic. How can the insights from this benchmark be used to build more resilient models? What concrete methods do the authors suggest for improving model robustness using these findings?\n\n2 While inter-annotator agreement is reported, what types of manipulations led to disagreement on “misleading intent”? Could you discuss a few ambiguous cases to better illustrate the subjectivity in labeling?\n\n3 The study focuses on caption manipulation with real images. How might the findings change if the images themselves were manipulated or synthetic, as is common in real-world disinformation?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1 The paper shifts the focus from simple fact-checking to intent detection in multimodal misinformation, which is a more nuanced and realistic challenge. This addresses a critical gap in current VLM evaluation.\n\n2 DECEPTIONDECODED is well-constructed, grounded in real news, and systematically manipulates intent. The use of human evaluation to validate realism, intent alignment, and label correctness strengthens the dataset’s credibility.\n\n3 The experimental design is thorough, testing models under multiple input modalities and with adversarial authenticity cues. This provides deep insights into model behavior and failure modes.\n\n4 The findings reveal specific weaknesses in current VLMs—such as susceptibility to misleading images, reliance on style over substance, and vulnerability to spurious hints—offering clear directions for future research.", "weaknesses": "1 The paper is purely diagnostic. It identifies problems but does not propose or evaluate any methods to improve VLMs’ robustness against the identified deception strategies, limiting its impact on advancing model capabilities. \n\n2 Although inter-annotator agreement is measured, judgments about “misleading intent” can be subjective. The paper could better discuss edge cases or ambiguous examples where annotators disagreed.\n\n3 The study focuses primarily on caption manipulation within a fixed image-news context. Real-world misinformation often involves deeper fabrications, manipulated images, or entirely synthetic content, which are not addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9WlB8Dphn2", "reviewer": "Reviewer_f8Fr", "summary": "This paper introduces DECEPTIONDECODED, a benchmark dataset for analyzing misleading creator intent in multimodal news. The dataset contains 12,000 image–caption–article triplets, each grounded in verified VisualNews articles, with both misleading and non-misleading variants generated under predefined “creator intents.”  They evaluate 14 vision–language models, including GPT-4o, Claude-3.7, Gemini-2.5-Pro, and Qwen2.5-VL. The results indicate that even state-of-the-art models perform poorly on intent reasoning, tending to rely on surface-level cues such as image-text consistency or stylistic polish.", "questions": "5. Ambiguity between the “implication-oriented (I)” and “consistency-oriented (C)” paradigms.\nWhile both evaluation paradigms are central to the analysis, their conceptual distinction and implications for model reasoning are not clearly defined.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper moves beyond conventional multimodal misinformation benchmarks that emphasize factual misalignment, by explicitly modeling creator intention—a rarely addressed but crucial dimension in understanding real-world deception.\n2. The cross-model comparison offers valuable insights into the current limitations of multimodal LLMs, especially regarding consistency-based reasoning versus implication-based reasoning.", "weaknesses": "1. Diversity of generated intents is insufficiently substantiated. The paper relies heavily on GPT-4o to simulate creator intents but does not provide a systematic method to ensure semantic diversity beyond domain coverage. Merely prompting GPT-4o may not guarantee varied “intent expressions,” leading to possible homogeneity in narrative framing.\n2. Exclusion of deepfake/manipulated-identity content limits ecological validity. Identity-based manipulations (e.g., political figures’ face swaps) represent a significant real-world threat vector. The absence of such cases restricts the benchmark’s relevance to the broader multimodal deception landscape.\n3. Lack of evaluation for misleading vs. non-misleading cases. Since the distribution is potentially skewed, reporting only accuracy can be misleading. The paper should provide F1 or macro-F1 scores to account for imbalance, and compare with related datasets such as NewsCLIPpings and MMFakeBench to show whether DECEPTIONDECODED introduces greater realism or challenge.\n4. Ambiguity in controlling the boundary between subtle and significant manipulations. The paper distinguishes between “subtle” and “significant” misleading cases in both image and text manipulations, but it is unclear how the generation process enforces or verifies this distinction quantitatively. Moreover, the reported 2% human evaluation is used only to assess label consistency and plausibility, which is insufficient to guarantee that the subtle–significant boundary is consistently maintained across 12,000 samples.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fn4fwYc83Q", "reviewer": "Reviewer_fbiN", "summary": "This paper introduces DECEPTIONDECODED, a multimodal news benchmark with explicitly defined creator intent to support misleading intent detection, source attribution, and desire inference. It reveals that current VLMs fail to reason about intent beyond surface alignment and stylistic cues.", "questions": "1. In Table 2, GPT-4o-mini and several open-source models achieve almost 100 percent accuracy on non-misleading (NM) cases simply by predicting “NM”, so could the authors explain why such trivial behavior is not prevented by the dataset design and how this may inflate average accuracy.\n\n2. Figure 3 shows models fail much more often when misleading text is written in a professional tone, so could the authors clarify how they ensured that “professional misleading style” does not accidentally include implicit emotional or sensational phrasing, which might cause style-based bias.\n\n3. Since the article context is information-rich, and Table 4 shows that adding the article dramatically boosts performance for most of the models, could the authors analyze whether article–caption lexical overlap is causing shortcut exploitation instead of true intent reasoning.\n\n4. In human evaluation, image-modified instances show noticeably lower agreement than text-modified ones (89.2 percent vs 99.2 percent), so could the authors show concrete examples where annotators disagreed to illustrate what types of visual manipulations fail to convey misleading intent clearly.\n\n5. Table 5 shows that adding only one line of external framing hint leads to accuracy changes up to +54.5 or −41.8, so could the authors analyze which components of their prompt template are causing models to overly rely on such hints rather than visual–textual evidence.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The definition of creator intent as “desired influence + execution plan” is theoretically motivated by communication strategy literature and operationalized explicitly during content generation, which avoids the usual post-hoc inference ambiguity\n\n2. Experiments across 14 diverse VLMs and multiple reasoning paradigms systematically show that even leading models like Claude-3.7 and GPT-4o struggle\n\n3. Transfer experiments demonstrate that adding this intent-centric data improves Macro-F1 by up to +30.7 for Qwen-VL-7B, suggesting this dataset captures knowledge that generalizes beyond its own controlled setting and is beneficial for real-world scenarios", "weaknesses": "1. The dataset’s misleading intent is fundamentally pre-scripted by prompts and therefore risks oversimplifying real-world deception in which intent is shaped by dynamic sociopolitical incentives, audience feedback, and long-term agenda setting, none of which are captured by one-shot synthetic manipulations\n\n2. The manipulations rely mainly on GPT-4o (text) and FLUX or GPT-image-1 (image) generation, so the dataset may inherit generator-specific patterns or stylistic artifacts\n\n3. Evaluation tasks are closed-set classifications, whereas real intent is continuous, multi-layered, and often only partially present, so current task design may underestimate the ambiguity in real world cases", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)