THEMIS: Towards Holistic Evaluation of MLLMs for Scientific Paper Fraud Forensics
Authors: Tzu-Yen Ma
Bo Zhang
Zichen Tang
Junpeng Ding
Haolin Tian
Yuanze Li
Zhuodi Hao
Zixin Ding
Zirui Wang
Xinyu Yu
Shiyao Peng
Yizhuo Zhao
Ruomeng Jiang
Yiling Huang
Peizhi Zhao
Jiayuan Chen
Weisheng Tan
Haocheng Gao
Yang Liu
Jiacheng Liu
Zhongjun Yang
Jiayu Huang
Haihong E
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Large Language Model
Vision Fraud Reasoning
Scientific Paper Fraud Detection
Benchmark
Primary area: datasets and benchmarks
TL;DR: We present THEMIS, a holistic multi-task benchmark of over 4K questions derived from authentic retracted-paper cases and realistically simulated synthetic data, to systematically evaluate the fine-grained visual fraud reasoning abilities of MLLMs.

Abstract: We present **THEMIS**, a novel multi-task benchmark designed to comprehensively evaluate multimodal large language models (MLLMs) on visual fraud reasoning within real-world academic scenarios. Compared to existing benchmarks, THEMIS introduces three major advances. (1) **Real-World Scenarios and Complexity**: Our benchmark comprises over 4,000 questions spanning seven scenarios, derived from authentic retracted-paper cases and carefully curated multimodal synthetic data. With 60.47\% complex-texture images, THEMIS bridges the critical gap between existing benchmarks and the complexity of real-world academic fraud. (2) **Fraud-Type Diversity and Granularity**: THEMIS systematically covers five challenging fraud types and introduces 16 fine-grained manipulation operations. On average, each sample undergoes multiple stacked manipulation operations, with the diversity and difficulty of these manipulations demanding a high level of visual fraud reasoning from the models. (3) **Multi-Dimensional Capability Evaluation**: We establish a mapping from fraud types to five core visual fraud reasoning capabilities, thereby enabling an evaluation that reveals the distinct strengths and specific weaknesses of different models across these core capabilities. Experiments on 16 leading MLLMs show that even the best-performing model, GPT-5, achieves an overall performance of only 56.15\%, demonstrating that our benchmark presents a stringent test. We expect THEMIS to advance the development of MLLMs for complex, real-world fraud reasoning tasks.

Published as a conference paper at ICLR 2026
THEMIS: TOWARDS HOLISTIC EVALUATION OF MLLMS FOR SCIENTIFIC PAPER FRAUD FORENSICS
Tzu-Yer Ma', Bo Zhang", Zichen Tang, Junpeng Ding, Haolin Tian, Yuanze Li, Zhuodi Hao, Zixin Ding, Zirui Wang, Xinyu Yu, Shiyao Peng, Yizhuo Zhao, Ruomeng Jiang, Yiling Huang, Peizhi Zhao, Jiayuan Chen, Weisheng Tan, Haocheng Gao, Yang Liu, Jiacheng Liu, Zhongjun Yang, Jiayu Huang, Haihong Et Beijing University of Posts and Telecommunications
httes://buut-rraaooong-........ htt.://tttuu..........ee. TO/THEMIS
httpe://ggiccub..com//Bte CBw.ccccc/BBBPP.c. ht httts::/hugginggfeee co http:///gw.aaaa.cccttaateeeeeeee B
Lab
Real-World Scenarios & Complexity
Fraud-Type Diversity
Multi Dutt-Dimensional Capability Evaluation
Gramularity
Dap
Expplicigg
Copy-
EIA Invositeecyy
Leoding MILMs
Dipplcationn
Tran Smange
KAA
Stained Micrograph
Diagram
Phyytcal
0
I
Micrograph
THIME
Comparacive
Visual
Exressee
ACC
Medical Imaging
Spottal
EM
Localsation
Figure 1: Overview of THEMIS. (1) Real-World Scenarios and Complexity, comprising over
4,000 questions across representative scenarios; (2) Fraud-Type Diversity and Granularity. covering 5 challenging fraud methods with 16 fine-grained manipulation operations; (3) Multi- Dimensional Capability Evaluation, mapping fraud types to core reasoning capabilities.
ABSTRACT
We present THEMIS, a novel multi-task benchmark designed to comprehensively evaluate multimodal large language models (MLLMs) visual fraud reason- ing within real-world academic scenarios. Compared to existing benchmarks, THEMIS introduces three major advances. (1) Korld Scenarios and Com- plexity: Our benchmark comprises over 4,000 questions spanning seven scenar- ios, derived from authentic retracted-paper cases and carefully curated multimodal synthetic data. With 60.47% complex-texture images, THEMIS bridges the criti- cal gap between existing benchmarks and the complexity of real-world academic fraud. (2) Fraud-Type Diversity and Granularity: THEMIS systematically cov- ers five challenging fraud types and introduces 16 fine- ann-ggriieed manipulation op- erations. On average, each sample undergoes multiple stacked manipulation oper- ations, with the diversity and difficulty these manipulations demanding high level of visual fraud reasoning from the models. (3) Multi-Dimensionll Capa- bility Evaluation: We establish a mapping from fraud types to five core visual fraud reasoning capabilities, thereby enabling evaluation that reveals the dis- tinct strengths and specific weaknesses of different models across these core capa- bilities. Experiments 16 leading MILLMs show that even the best-performing model, GPT-5, achieves an overall performance of only 15.15, demonstrating that our benchmark presents stringent test. We expect THEMIS advance the development of MILLMs for complex, real-world fraud reasoning tasks.
"qqual contribution. 'Corresponding author.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Multimodal large language models (MLLMs) (Singh ct al., 2025 OpenAl 2025: Bai et al. 2023
2025 Wang al., 2025e; Meta 2025 have emerged as powerful paradigm for unifying vision and language, driving rapid progress in multimodal understanding and reasoning. Early studies have primarily validated MLLMS on simple visual capabilities, including fundamental perceptual skills such as object recognition, grounding, and caption generation (Plummer et al.) [015 Chen al. 2013). Recent research has further demonstrated that MLLMs also exhibit advanced abilities, extending beyond basic perception to more complex benchmarks such as comprehensive multimodal evaluation and fine-grained capability assessment (Liu al. 2025b
Zhang et al. 2025).
Despite the demonstrated progress on simple and advanced visual capabilities, expert-level visual capabilities, the ability perform robust and comprehensive reasoning realistic, complex scenar- ios, remain insufficiently explored and rigorously validated. To probe whether MLLMs truly possess such competence deep visual understanding and reasoning, we adopt scientific paper fraud as the boundary scenario for evaluation, as such forensics demand not only pixel-level anomaly percep- tion but also an understanding of the underlying scientific context and logical consistency within the image. This setting introduces unprecedented challenges, raising the demands intrinsic visual reasoning depth, precision, and robustness to a level unseen in existing benchmarks.
A major bottleneck in pursuing this agenda is the absence of appropriate benchmarks. Existing benchmarks (Li et al. 2025b] Zhou & Hong. 2024, Shi et al.] 2025 Wang et al.) 2025b] Liu et al, 2025a Wang et al.) (025a) generally fall short of meeting the demands of expert-level visual rea- soning in terms of real-world scenario complexity, fine-grained fraud-type diversity, and multi- dimensional capability evaluation.
To fill this gap, as shown in Figure we present THEMIS, a holistic multi-task benchmark of 4,054 questions derived from authentic retracted-paper cases and synthetic data, to systematically evaluate the fine-grained visual fraud reasoning abilities of MLLMs. By grounding evaluation in realistic contexts, THEMIS provides the foundation for probing expert-level capability. Specifically, THEMIS achieves this through three core design principles:
Real-World Scenarios and Complexity. THEMIS spans seven representative academic scenar- ios (e.g.. Micrograph and Medical Imaging), derived from authentic retracted cases and care- fully constructed synthetic data, ensuring realism and controllability. Importantly, 60.47% of the images-primarily micrographs, physical objects, and medical imaging sanples-contain com- plex textures, which substantially increases the difficulty of manipulation detection.
Fraud-Type Diversity and Granularity. THEMIS systematically covers five challenging fraud methods (c.g., Al-Generated and Duplication) and introduces 16 fine-graines manipulation oper- ations (e.g., scaling and color temperature modification). On average, each sample involves 2.08 stacked operations, producing highly diverse manipulations that require robust reasoning over composite alterations.
Multi--Dieeiional Capability Evaluation. To dissect model performance on these expert-level tasks, THEMIS establishes principled mapping from fraud methods to five core reasoning capa- bilities that characterize expert-level visual forensics. Expert Knowledge Utilization evaluates whether models can incorporate prior domain knowledge to contextualize manipulations. Visual Recognition tests their ability to accurately perceive and distinguish complex visual elements. Spatial Reasoning requires understanding positional and structural relationships among manip- ulated components. Region Localization focuses on precisely identifying tampered areas at the sub-figure level. Finally, Comparative Reasoning assesses the ability to contrast multimodal evidence, such as cross-image or text-image consistency.
We evaluated 16 leading MILMS on THEMIS, revealing three key findings: (1) a universal limita- tion in expert-level reasoning, with the state-of-the-art (SOTA) GPT-5 achieving only 56.15% over- all performance; (2) pronounced vulnerability to compound transformations, where GPT-5's Fl score on the Duplication Identification task plummets from 38% to 17% as the number of stacked manipulations increases from one to three; (3) imbalanced capability profiles across all models, exemplified by GPT-5's disparate performance Visual Recognition (53.50%) versus Region Lo- calization (24.14%). These findings collectively underscore the challenge of our benchmark and the critical reasoning gap in current MILMss
2

---

Published as a conference paper at ICLR 2026
Table I: Comparison of THEMIS and other related benchmarks. Mod.: Modality; Manip.:
Manipulation Operations; Real: Contains real fraud cases; I: Image: T-I: Cross Text-Image Task: BC: Binary Classification; IP-L: Image Partial Localization. SMF, CMO, and CMI denote Single-
Mode Forgery, Composite Manipulation Operations, and Cross-Modal Inconsistency, respectively, with suffixes -I and -I indicating Identification and Localization (-II, for both). Fraud Type: 1
Splicing, 2
Covy-Move, 3 Al-Generated,
4 Duplication, and 5 Texe-Image Inconsistency.
Benchmark
Mod.
Fraud Type
# Manip.
Real
# QA Pairs
Task
FakeBench
3
x
54K
BC BC
DiffuSyn Bench
3
7
x
848
SHIELD
12
v
1,5K
BC/CMO-1
MFC-Bench
T-I
1 5
7
35K
BC/CMI-I
MMFakeBench
T-I
1 5
12
11K
BC/CMI-I
Forensics-Bench
I/T-1
1
2
3
5
15
x
63K
BC/IP-//CMI-II
THEMIS (ours)
I/T-I
3 5
16
4K
SMF-L/CMMO-I/MI-II
2
RELATED WORK
2.1 VISUAL REASONING OF MLLMS
The advent of MILMs has catalyzed paradigm shift from basic visual perception to complex vi- sual reasoning. Pioneering models like BI.I--2 (Li et al.
2023) bridged modalities via efficient
alignment. Building this, the LLaVA series (Liu et al.
2023 2024) significantly advanced vi-
sual instruction tuning, while GPT-4o (OpenAl ct al. (024)) enabled unified end-to-end multimodal reasoning. Recent developments such as the ol family (OpenAl 2025) demonstrate that integrating inference-time Chain-of- of-Thhught (CoT) (Wei et al.) 2022) further enhances reasoning depth.
However, eritical blind spot persists in current evaluations. While MLLMs excel on bench- marks requiring domain-speciee knowledge (e.g., mathematics or chart analysis) (Yue et al.) 2024 Lu 2024 Wang al. 2024b), their performance often hinges the powerful textual reasoning of the underlying large language model (L..)) rather than on genuine visual acuity. In these tasks, visual content can frequently be abstracted into descriptions, which the LLM then solves. Con- sequently, existing evaluations fail to decouple visual reasoning from this textual dependence, This limitation effectively obscures the true state of MILMs' intrinsic visual capabilities, particularly in tasks like forensics where visual evidence cannot be losslessly converted into text.
2.2 VISUAL FRAUD REASONING BENCHMARK
Although benchmarks designed for traditional image forensics, such as CIMD (hhng et al. 2024) and GIM (Chen et al. 2025). have established rigorous standards for specific detection tasks, they ill-suited for assessing the holistic reasoning capabilities of MILMs. However, as shown in Table current benchmarks targeting MLLMs still fall short of expert-level standards. The limitations of existing works are threefold: (1) Restricted Scope: FakeBench (Li et al.] 2025b), DiffuSSn Bench (Zhou & Hong 2024), and SHIELD (Shi al., (025) cover only narrow range of fraud types; (2) Coarse Granularity: MFC-Bench (Wang et al. 20255) and MMFakeBench (Liu et al., 2025a) primarily focus on binary classification and identification tasks, lacking fine- graiinad multi- dimensional evaluation metries; (3) Lack of Realism: Forensics-Bent (Wang et al. (225aa) relies heavily on synthetic data while ignoring authentic cases, thereby failing to capture the complexity of real-world scenarios.
To bridge these gaps, we propose THEMIS. Addressing the lack realism, THEMIS integrates 152 real-world forensic cases distilled from a rigorous screening of 1,432 retracted papers, alongside high-fidelity synthetic data to ensure scenario authenticity. Unlike prior works limited to coarse metrics, THEMIS establishes a principled mapping from diverse fraud types to five reasoning capabilities. This enables a precise, fine-grad grained diagnosis MLLMs' intrinsic visual forensic competencies, aligning task difficulty with the rigorous demands of expert forensics.
3

---

Published as a conference paper at ICLR 2026
Numevical Incotaiitinnyy
Insinaiitancy 3.50%
Copy More 3.0g%
Infreence Forgery 1.0l
600
Naturaaionn
Spiicing 1s 45%
500
Local Inplication 33-99%
Region Bdittiig $5-5
300
dveling
Ginbal Duplication
Nusperkoal
100
Treed
Splicing
Copy-Move
Deplication
Inconstatency
Distribution of fraud methods.
(b) Distribution of manipulation operations (synthetic data).
Figure 2: Statistics of THEMIS. (a) Distribution of fraud methods. (b) Distribution of manipulation operations (synthetic data). See forr real cases. IIF: Image Inference Forgery; TRE: Targeted Region Editing; CT: Color Temperature; DR: Direct Reuse; HF: Horizontal Flip; VF: Vertical Flip.
THEMIS BENCHMARK
3.1 OVERVIEW OF THEMIS
As illustrated in Figure 2 THEMIS constructs a comprehensive benchmark with a hierarchical forensic structure. Sourced from academic publications and retracted papers. the dataset undergoes detailed manual annotation and human-AI collaborative augmentation, covering five major fraud methods. Certain categories are further subdivided into secondary sub-types to support fine-grained visual fraud reasoning evaluation.
Building upon this taxonomy, we design three distinct tasks: Single-Mode Forgery Identifica- tion and Localization, Composite Manipulation Operations Identification, and Cross-Modal Inconsistency Identification and Localization. These questions fully encompass the aforemen- tioned fraud methods. In total, 4,054 high-quality QA pairs constructed, including 152 manually annotated samples derived from real-world academic retraction cases, which significantly enhances the reliability and annotation accuracy of the benchmark.
3.2 DATASET CONSTRUCTION
the dataset construction is divided into two stages: the first stage involves the extraction and parsing of source data, while the second stage focuses on fraud data generation and quality control for the five major fraud types.
As shown in Figure 3
Stage 1: Extraction and Parsing. We utilized primary data sources for our benchmark, with summary statistics in Appendix A. (1) Real Data: We curated a gold-standard set of real fraud cases by collecting retracted papers from Retraction Watch and PubPee From this, experts ex- tracted 194 authentic panels (defined as subgraph units with independent semantics within a larger figure). The collection and annotation process is described Appendix A.2 (2) Synthetic Data: We gathered 5,253 research papers from PubMed Centra After rigorous selection, 879 high- quality papers were identified as source material. From this, we extracted 41,422 high-resolution panels. For cross-modal tasks, we curated 150 (figure, caption, related sentence) triplets, selecting one from each of 150 chosen papers. The six-step extraction and parsing pipeline, including YOLOv7-based (Wang et al. 2023) panel segmentation, is detailed in Appendix A.3.
Stage Fraud Data Generation. To ensure a comprehensive evaluation of forensic capabili- ties, simulate realistic academic paper fraud through generation pipeline covering five major fraud data types. The detailed methodology and quality control for each type are described in Ap- pendix A.3 The key strategies summarized follows:
https::/zetractiiowws
https://blcag com https://pmc.ncet mmm.mmmmmmm....
4

---

Published as a conference paper at ICLR 2026
Stage 1: Extraction & Parsing
Stage 2: Fraud Data Generation
Pages
Figures
Panels
Splicing
Forgery
Splicing
SAM
Copy-Move
Real Panel
CT Mask
Porgery Fane)
Copy-Move
41 Captions
SAM
Al-Generated
Real Panel
Imsaee
Forgery Panel
Thrgeted Borgery Pandl Battiig
Indereboe
Related Sentences
Jas
denotel by inteobeeeer
Duplication
Real
Reused Pancl
Text-Image Inconsistency
Numerical Inconslitency
Inconalstency
"Trnndorttaioon
Geometric
Phcumatee Modification
Original Caption: reteograda,
Origital Related Setiince:
stationary, 40.4% 4,00;; roverning, (b) Distribution of gradd....d.......
demonptrated rapamycin tnatimet naluoed
Local Duplication
lenels of Senescence reattdd
Real Panel
18,3fs 2 2.1%, stortoeary, $o.ate * 4.00% reverning, 9.10 1.3%. Distribution anterogrode...
demonstrated that rapamyein trastmmntt redooce decressing related...
Paramater Modification
Figure 3: Dataset construction pipeline of THEMIS. The dataset is built through stages: Stage 1: Extraction and Parsing, where figures, captions, and related sentences are parsed from scientific PDFs and segmented into panels; Stage 2: Fraud Data Generation, where 5 major fraud types (Splicing, Copy-Move, Al-Generated, Duplication, and Text-Image Inconsistency) are applied to construct challenging tasks.
Splicing. We collected sigg--imilartty panel pairs the basis and created them by recombining the foreground and background of the panels, while employing bidirectional splicing and bound- ary fusion to ensure visual consistency and coherence.
Copy-Move. We generated Copy-Move samples by replicating and translating specific objects within the same panel. During this process, we utilized the Segment Anything Model (SAM) (Kir- illov et al., (023) to perform automatic object extraction and mask optimization, and employed an adaptive gnd positioning strategy to determine the placement of copied regions.
Al-Generated. We constructed two types of Al-Generated panels: (1) Image Inference Forgery, which used the Flux (Labs et al. 2025) model to generate full-image forgeries; (2) Targeted
Region Editing, which based text prompts mask information and employs models such as Stable Diffusion (Hsser et al., 2024). SenseNova V6 Miaohua (Sense'Time 2025) and GPT-
Image-1 (OpenAl 2025) for local editing.
Duplication. We implemented this operation to simulate the reuse of panel content under vari- manipulations. We designed two distinct strategies capture different scales of duplication: (1) Global Duplication, which targeted reuse the entire image by performing direct reuse or applying geometric transformations and parameter modifications to the original panel to generate a manipulated duplicate; (2) Local Duplication, which simulated the subtler reuse of specific regions. This process began by extracting two panels with overlapping regions from the original panel. Then, employing soop-then-transhie
strategy, we applied geometric transformations parameter modifications to a specific area of one panel. Finally, both panels were reassembled onto standardized canvas to form visually coherent duplication pair with consistent dimensions.
Texte Inconsistency. We constructed 150 inconsistent text-image pairs by altering cap- tions or related sentences so that they no longer matched the image content. We divided the modifications into two categories: (1) Numerical Inconsistency, where modified numerical values such as data and proportions; (2) Trend Inconsistency, where we replaced rend-related keywords with their antonyms.
5

---

Published as a conference paper at ICLR 2026
Al-Generated
Splicing
Copy-Move
Task Definition
Task Definition
Task Definition
Panel wholly or partially Al-generated; cues: synthetic textures, implausible details, inconsistent semantics.
Different sources of content combined in panel; cues: unnatural seams, boundary mismatches, inconsistent textures.
region copied within the panel; cues: duplicated patterss/objetts with same
moise/ texture but shifted/rotated/ sccaled
Question
Question
Question
Detect image forgery type and localisa forged regions.
Detect image forgery type and localiza forged regions.
Detect image fargery type and localiza forged regions.
AjSplictng DMA-Geserated DJNo Forgery
#XCopy- Move
AsSaliiig Gene-renesttdd DJNo Burgery
B)Copy Move
AlSplicing ByCagy-Mow Mooe Generated DJNo Porgory
KiNot
E/Not
EjNot
comma seperated
<MASKo comma- separated region numbers (uMsers-/BMAAK>>
region numbers</MMBBR
region numbera<
Duplication
Text- Image Inconsistency
Task Definition
Task Definition
Repeated content reused across panels; cues: identical pear-identical regions, consistent textures/ patterns, matching structures passibly under geometric parametric transformations.
Determine whether the accompanytnng text consistent the figure by identifying contradiction arise numerical vallaes in described trends.
Question
Detect whether caption and related sentences problematic parts.
Question
corresponding figure and localize
Identify every doplication type that appears.
AlNumecical Inconelsteet.
AlDirect B15caline
CiRetated DiPtioped
Aigal
B) 3iTrend Inconaistent
Cooonsistent Sure
Modiiidd
GyNot Sure
PiNoo Duplication
<SENTINCES .ingnaa (P
0.85)...4
Figure 4: Evaluation task design of THEMIS. A principled mapping from 5 fraud types 5 core reasoning capabilities (Expert Knowledge Utilization, Visual Recognition, Spatial Reasoning, Region Localization, and Comparative Reasoning). The capability distribution bars on the right of each box illustrate the reasoning skills involved and their relative emphasis, with the darkest color highlighting the primary capability being evaluated.
Quality Control. The operation involves over 16 academic review experts, supported by GPT-4o mini The entire quality control process took approximately 200 hours, and about 20% of low- quality or unreasonable samples were removed from the original synthetic data. After this rigorous filtering, 4.635 positive samples were retained, alongside 1,540 negative samples without fraud.
3.3 EVALUATION TASK DESIGN
As shown in Figure to evaluate the comprehensive perception and reasoning capabilities of MLLMS across five fraud methods, we designed three core evaluation tasks. For each, the detailed definition, strict input/output format, and prompting instructions are provided in Appendix A.4
Task 1: Single-Mode Forgery Identification and Localization (SMF-IL). This task corresponds to three fraud methods: Splicing, Copy-Move, and Al-Generated. The model must determine whether the input panel is No Forgery. Not Sure, or one of the three specific forgery types. If forged, the model must further provide block-based localization of the forged region. We evaluate identification with Accuracy (ACC) and localization with Intersection over Union (IoU).
Task 2: Composite Manipulation Operations Identification (CMO-I). This task corresponds to Duplication. The model is required to analyze a pair of panels and discriminate the types of manipulation operations involved, selecting from seven potential categories: Direct Reuse, Scaling, Rotation, Flip, Parameter Modification, No Duplication, and Not Sure. Since multiple selections are permitted for a single pair, we employ the Set-based F1 Score for evaluation.
Task 3: Cross-Modal Inconsistency Identification and Localization (CMI-IL.. This task corre- sponds to Tmxee Inconsistency. It requires the model to judge whether Numerical or Trend inconsistency exists based on the input image and its corresponding text. If detected, the model must further localize the minimum sentence unit with inconsistency in the text and provide corrected con- tent. We evaluate identification with ACC and localization with Text-based F1 Score
Real Cases. THEMIS includes 152 real-world forensic issues derived from a rigorous screening of 1,432 retracted papers sourced from Retraction Watch and PubPeer. Following strict inclusion criteria, 41 nigh-quality manuscripts were retained as the source corpus. These issues predominantly involye three major fraud types: Splicing, Copy-Move, and Duplication. To ensure data reliability,

---

Published as a conference paper at ICLR 2026
collaborative protocol where each paper was independently reviewed by three experts academic image forensics. Consequently, this subset represents a curated gold standard of real academic fraud cases to support fine-grained reasoning.
employed
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Benchmarked MLLMs. We evaluate nine proprietary models (Singh et al. 20025 OpenAl 2025
Comanici et al., 2025 Bai al. 2023 Team [025 Anthropic 2025 Team al.
2026) and
open-source models Meta 2025
Bai et al. 2025 Team et al. [2025] Li et al. [2025a]] Liu et al. 2024
Wang et al. 2025c) covering mainstream industrial progress and diverse open- architectures.
Input Processing. We use high-resolution PNG images whenever possible. If an input exceeds a model's s limit, a two-stage adaptive compression applied: (1) scale the long edge each model's maximum input size while keeping aspect ratio: (2) further reduce total pixels to satisfy model constraints, with iterative 10% reduction necessary.
Evaluation Metrics. We adopt two primary metric categories: Identification Score (Id Score) and Localization Score (Loc Score), applied according to the diverse visual fraud reasoning tasks.
Single-Mode Forgery Identification and Localization. We use ACC as the Id Score and IoU as the Loc Score.
Composite Manipulation Operations Identification. We employ the Set-based F1 Score as the Score, since multiple selections are permitted.
Cross-Modal Inconsistency Identification and Localization. We use ACC as the Id Score and Text-based F1 Score as the Loc Score, reflecting precise localization of inconsistent textual units.
We further introduce Balanced Robustness Index (BRI), which adjusts mean performance by nalizing large variance across tasks. The formal mathematical formulations for all metrics, alongside the detailed derivation and sensitivity analysis of the BRI, provided in Appendix B.3
Table 2: Model performance across tasks THEMIS synthetic data. We report the Id Score (Identification Score) and Loe Score (Localization Score) for fraud methods: SPL (Splicing), CM (Copy-Move), AIG (AI-Generated), DUP (Duplication), and TII (Text- Image Inconsistency), with BRI (Balanced Robustness Index). The best and second-best scores are highlighted.
Composite Manipulation Operations Ideutification (Id Score)
Cross-Modal Incomsistency Identification & Localization
Single-Mode Forgery Identification Score)
Single-Mode Forgery Localization (Loc Score)
Model
CM
AIG
SPL (807)
AIG (897)
THl Id Score Loe Score
(807)
(242)
(897)
Avg.
(242)
Avg.
(2,079)
Proprictary MLLMs
GPT-5
43.51
72.73
44.26
53.50
16.67
36.41
19.33
60.67
27.44
56.15
OpenAl o4-mini-high
41.49
77.89
35.67
51.68
10.44
29.78
19.79
20.00
30.34
66.33
32.22
52.34
Qwea-VL-Max
87.40
35.43
51.07
40.07
48
35.85
41.42
56.00
15.36
49.83
Gemini 2.5 Tlash
63.39
67.56
35.45
55.47
56.72
46.98
38.87
47.52
24.96
36.33
28.24
44.70
Doubao Seed. thinking
35.67
74.17
36.57
48.55 8D
12.09
13.58
15.19
13.62
20.22
60.00
31.71
37.14
Doubao-Sed--1 v.vvvion
20.84
61.78
27.54
36.72
31.42
26.97
29.90
29.43
45.13
37.00
30.43
33.47
Gemini 2.5 Pro
30.60
47.46
14.90
30.99
46.65
46.61
47.20
46.82
21.49
44.67
37.31
31.97
GLM-4.5V
58.43
47.39
8.54
13.80
31.57
Claude Sonnet 4.5
29.71
75.66
30.43
45.27
14.95
44.12
20.98
26.68
2..86
35.67
27.4 .2
29.96
Open-Source
Qwea2.5-VL-7228
36.40
77.27
51.06
54.91
$1.66
35.57
47.46
16.75
61.33
12.32
47.16
Int5maVL335-8B
30.97
58.42
67.00
52.13
43.38
40.24
33.19
38.94
33.28
55.00
4.78
38.73
Liama 4 Maverick
23.37
51.24
58.19
44.27
17.97
28.50
19.71
22.06
19.01
$4.00
16.08
34.78
41.80
47.55
46.93
45.43
35.97
25.22
32.06
31.08
8.01
50.00
12.57
23.59
32.00
84.00
34.00
50.00
50.70
45.25
34.01
43.32
10.73
41.33
4.28
18.40
Qwen2,5-VL-332
30.31
57.48
39.24
42.35
5.91
18.93
7.45
10.76
15.26
58.33
17.94
18. 22
Gemma 27B
25.39
28.87
34.23
29.50
27.78
32.80
26.44
29.01
12.20
31.67
21.41
9.59

---

Published as a conference paper at ICLR 2026
4.2
MAIN RESULTS
Table presents the evaluation results across all models on synthetic data. Our main findings are:
Overall performance remains limited. Across all evaluated tasks, MLLMs exhibit significant room for improvement, with even the SOTA GPT-5 reaching peak BRI of only 56.15%. Notably, the open-source Qwen2.5-VVL-725 achieved 47. 47.16%, performing on par with several proprietary models and highlighting the competitive potential of open-weight architectures.
Models exhibit pronouneed specialization. Most MLLMs excel in one or two subtasks but fall short in others, revealing substantial imbalance. This specialization highlights the lack of integrated visual fraud reasoning capabilities current models.
Localization is markedly harder than identification. Performance consistently declined across all MLLMs when shifting from Single-Mode Forgery Identification task Localization task, with GPT-5 dropping by 55% and OpenAI o4-mini-high by 61%. By contrast, Gemini 2.5 Flash exhibited only minor declines (14%), indicating relatively stronger spatial perceptual capacity.
Limitations in fine-grained cross-modal alignment. Although they demonstrate relatively high judgment accuracy on Text-Image Inconsistency subtask, they struggle to ground these judgments in specific textual spans. This limitation may stem from overreliance global semantic associ- ations while lacking sufficient modeling of local cross-modal mappings.
None
Gaussian Blur
GPT-5
Qwen-VI-Max
JPBG Compression
Scaling
OpenAl o4-mini-high
Qwen2.5-Ve -VL-7aBB
Localization Score (96
50
49
Identiftcation Score
70
40
30-
30
50-
Identification
20
30-
20
SPL
CM AIG DUP
SPL
CM
AIG
(a) Impact input perturbations
performance.
on Duplication identification performance.
Impact manipulations
Identification Score (%6
Localization (96)
GPT-5
Score (96)
50-
.30
Llama Maverick
40-
(CoT)
50
20
30-
Llama (CoT)
GPT (Few-Shot)
30
Llama (Few-Shot)
SPL
CM
AIG
SPL
CM
AIG
(c) Impact CuT prompting on everall performance.
(d) Impact few-shot prompting on idfntification performance.
(e) Impact of few-shot prompting on localization performance.
Figure 5: Impact of different factors on model performance using synthetic data. SPL: Splicing; CM: Movy-Move; AIG: AI- AI-Generated; DUP: Duplication; DUP-Id: ld Score for Duplication; THI- Id: Id Score for Text-Image Inconsistency; TII-Loc: Loc Score for Text-Image Inconsistency: SMF-Id: Average Id Score of Single-Mode Forgery Identification; SMF-Loc: Average Loc Score of Single-Mode Forgery Localization.
4.3 FINE-GRAINED ANALYSIS
Tables 2 and Figure 5 present the fine-grained results from our further analysis. For detailed quantitative analysis, provide two sets of supplementary results in the appendix: breakdown of performance on real versus synthetic data and under input perturbations in Appendix C and per-fraud-type performance analysis for all models in Appendix (D) The key findings are as follows:
Lack of transformation sensitivity. Models perform reasonably on direct reuse but degrade markedly under geometric transformations or appearance adjustments. They often fail de- tect duplication and cannot determine the transformation type, underscoring the limited spa- reasoning ability of current MLLMs and their insufficient robustness to transformations.
8

---

Published as a conference paper at ICLR 2026
This reveals that current models have basic similarity matching ubility but re- main weak in edge sensitivity.
Table 3: Performance comparison on real cases. We report the results for Single-Mode Forgery Identification and Localization (SMF-IL) and Composite Manipulation Operations Identification (CMO-I) tasks.
Current models lack robustness input perturbations. To test MLLM sensitivity on synthetic data, we ap- plied Gaussian blur, JPEG compres- sion, and scaling. Gaussian blur caused the steepest drops across tasks, while JPEG compression and scaling also de- graded performance.
SMF-IL
CMO-1
Model
Id Score
Loc Score
Id Score
Dwen2.5-VL-72B
50.00
45.41
22.39
Gemini 2.5 Flash
44.55
43.07
49.58
GPT-5
43.64
28.68
23.22
OpenAl o4-mini-high
34.55
21.67
36.04
Insufficient edge perception. Mod- els perform better on Copy-Move than Splicing, as the former can exploit both edge anomalies and region simi- larity, while the latter depends mainly on boundary cues.
Gemma 27B
20.00
16.15
42.18
Qwen-VL-Me VL-Max
6.67
9.43
20.95
Llama 4 Maverick
3.34
5.44
39.75
Synthetic fraud data exhibit a level of deceptiveness comparable to real fraud data. In the Composite Manipulation Operations Identification task, the synthetic data impose even greater identification pressure than real data. In the Single-Mode Forgery Identification and Localization task, their difficulty is similarly on par with real data. The main exception lies in Splicing subtask, where real data remain more challenging due to the sophistication and subtlety of splicing patterns in real-world manipulations.
4.4 ERROR ANALYSIS
We conducted an error analysis on the best-performing model, GPT-5, Specifically, we sampled 20 failure cases for each of the five core fraud types, resulting in total of 100 erroneous instances. These errors are then mapped to the five core reasoning capabilities. Notably, single case may involve multiple deficiencies. leading to overlapping classifications. For a detailed qualitative anal- ysis. Appendix B visualizes 22 representative cases covering the full spectrum of seven academic scenarios and five fraud types.
Expert Knowledge Utilization (43/100): The model often fails to leverage prior domain knowl- edge recognize specific manipulation patterns, resulting in lack of contextualized reasoning.
Visual Recognition (37/100): When confronted with high texture complexity or fine-grained ma- nipulations, the model exhibits insufficient perceptual capacity, leading to perception- errors.
Spatial Reasoning (19/100): The model struggles cases requiring spatial reasoning, failing to correctly infer positional and geometric relationships among components.
Region Localization (25/100): Predicted fraud regions frequently appear blurred or spatially misaligned, reflecting unreliable fraud localization performance.
Comparative Reasoning (21/100): The model often overlooks subtle differences or fails to es- tablish effective cross- modal comparisons, thereby producing erroneous conclusions.
4.5
PROMPTING STRATEGIES
In our benchmark, we evaluate two prompting strategies synthetic data. Few-shot (Brown et al., 2020) provides only limited and inconsistent benefits, whereas CoT (Wei et al. 2022) consistently enhances reasoning performance. Detailed performance breakdowns across specific fraud types, along with the full CoT and few-shot prompt templates, are provided in Appendix C.2
Few-shot prompting yields mixed effects on forensic reasoning. While it brings moderate gains in certain subtasks, such as Copy-Move and AI-Generated, performance often declines in other inconsistent. We hypothesize that few-shot examples may
subtasks, and overall improvements constrain models to imitate local patterns in the demonstrations rather than generalize, limiting their ability to adapt to diverse fraud types.

---

Published as a conference paper at ICLR 2026
CoT prompting proves effective for deeper forensic reasoning. In benchmark, GPT-5 (the strongest model) and Llama Maverick (a weaker baseline) both benefit from CoT. showing consis- tent improvements. By enforcing stepwise analysis of visual cues and decision options, CoT enables models to move beyond surface-level correlations in scientífic figures. Notably, Llama Maverick benefits more from CoT than GPT-5, suggesting that models with weaker native reasoning abilities benefit disproportionatel from stepwise prompting.
5 CONCLUSION
In this paper, introduce THEMIS, novel benchmark designed to evaluate the expert-level visual reasoning capabilities of MLLMs. By integrating real-world forgery scenarios, diverse, fine- grained fraud methods, and multi-dimensional capability evaluation within a unified frame- work, THEMIS systematically probes the performance of MLLMs on complex academic multi- modal fraud detection. Our extensive evaluation of leading MILMs reveals significant limita- tions in their current abilities, with even SOTA models performing poorly on compound manipu- lations and exhibiting highly imbalanced capability profiles across different reasoning dimensions. We hope this work will inspire future research into more robust and diagnostic evaluation paradigms for MLLMs.
BROADER IMPACT
The rapid evolution of MLLMs presents dual-edged sword to the scientific community. While these technologies accelerate research, they simultaneously lower the barrier for fabricating sophis- ticated scientific data, enabling new forms of misconduct- as Al-generated imagery and text that are increasingly difficult to distinguish from authentic work.
Current defense mechanisms, which rely heavily manual inspection by editorial boards or spe- cialized models with limited scope, are becoming inadequate against these emerging, scalable AI- generated content (AIGC) threats. The relatively low number of publicly exposed retraction cases likely reflects a detection gap rather than a rarity of misconduct, as the community currently lacks the capacity for large-scale, automated, and precise verification.
Our work addresses this critical asymmetry by advocating for an AI for AI Governance approach. By introducing THEMIS, we provide the first comprehensive benchmark that simulates the com- plexity of real-world academic fraud, ranging from subtle pixel-level tampering to logical incon- sistencies in generated content. This benchmark serves foundational testhed for developing trustworthy multimodal systems capable of automated forensic reasoning. We hope THEMIS will catalyze the development of high-throughput forensic agents that can assist human experts, thereby safeguarding scientific integrity and restoring trust in the era of generative AI.
ETHICS STATEMENT
This work adheres the ICLR Code of Ethics. In this study, human subjects animal experi- ments were involved. All datasets used were sourced in compliance with relevant usage guidelines, ensuring violation of privacy. We have taken care to avoid any biases discriminatory out- comes in our research process. No personally identifiable information was used and no experiments conducted that could raise privacy security concerns. We committed maintaining transparency and integrity throughout the research process.
REPRODUCIBILITY STATEMENT
We are committed to making our research fully reproducible. All code, datasets, and evaluation scripts have been publicly released on our Project Page GitHub repository] and Hugging Face The experimental setup, including detailed model configurations and hardware specifications. is pro- vided in Section 4.1 and Appendix B We also provide a comprehensive description of THEMIS to facilitate future research and benchmarking in academic fraud forensics. We believe these resources will enable other researchers to replicate our findings and further advance the field of multimodal reasoning.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported by the National Natural Science Foundation of China (Grant Nos. 62473271, 62176026), the Beijing Natural Science Foundation (Grant No. QY25338), the Fundamental Re- search Funds for the Beijing University of Posts and Telecommunications (Grant No. 2025A14503), and the BUPT Innovation and Entrepreneurship Support Program (Grant No. 2025-YC-A042), This work is also supported by the Engincering Research Center of Information Networks, Ministry of Education, China. We would also like to thank the anonymous reviewers and area chairs for con- structive discussions and feedback.
REFERENCES
Anthropic. System Card: Claude Sonnet 4.5. Technical report, Anthropic, May 2025. URL httpst //
com wttpaacc.cocccaaaaaaao..aaaaaaaa aude sonnet 4-5 --5-----temmmmmm---- card Accessed: 2025-09-
29.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report, 2023. URL
2309. 1099.16600 16609
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report, 2025. URL https:///ssxxiii..tt g aaaabb/22220 13923
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processsig Systems, volume 33, pp. 1877-1901. Curran Associates, wwwess/appeeee///22
Inc., 2020. URL httpppss/pppoocedngg..eeeeeeeeee 0/41://550o66r0000111111aa222222
Xinlei Chen, Hao Fang, Tsung-Yi Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollar, and C. Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server, 2015. URL
https://arxiv. wttp:/xiw..gg/a//////aaa//
(trg/abs/1550 00325
Yirui Chen, Xudong Huang, Quan Zhang, Wei Li, Mingjian Zhu, Qiangyu Yan, Simiao Li, Hanting Chen, Hailin Hu, Jie Yang. Wei Liu, and Jie Hu. Gim: A million-scale benchmark for generative image manipulation detection and localization. Proceedings of the AAAI Conference on Artificial
Intelligence, 3902)2311-2319, Apr. 2025. doi: 10. 160%/aaai. 3912.32231. URL https: [/ojs.aaai. www.aa...rriinne index hhtpp/AA//atiicceiieeeeee 32231
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities, 2025, ItLLthtts:///aatt
org htrrg/abb/25000 06261
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scar- lett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 12606-12633, PMLR, 21-27 Jul 2024. RRLhttts: htttt:::://pocceddinng http://ss.....eeeee//ee//eee//e.
ml
html
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo. Piotr Dollar, and Ross Girshick. Seg- ment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2011-40266 October 2023. Segment Anything
http::/:s///ppeaccessss wtweeesssshheeeee.eee theewf com/ cont
ent Dnt/IICcv2023ppapezzz L htg.:aaoooooeeeeeeeaaaaaaaaaoono ICCV 2e__c___2223_ppee_____2_ pc
f
11

---

Published as a conference paper at ICLR 2026
Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English. Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context image generation and editing in latent space, 2025. URL htt.httss://arattt
org aba /250 1
Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun MA, and Chunyuan Li. Llava-interleave: Tackling multi-image, video, and 3d in large multimodal models. In Y. Yue, A. Garg, N. Peng. F. Sha, and R. Yu (eds.), International Conference Learning Representa- tions, volume 2025, pp. 81182-8 199, 2025a. URL ht httpppppppppppocoggp..coc. wcw.iinnns..i.....ooo.mm.m paper fi /ttp:///wwe..e../.....aaa/aaaaa. er er-Conference.pd
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference ceedings of Machine Learning Research, pp. 19730-19780 19742. PMLR, 23-29 Jul 2023. URL m
Machine Learning. volume 202 of Pro-
https: http:///poceedddeee wtt.diiinnnnmm..ss.eeeeee www.srpppss...mmee v20 23q html
Yixuan Li, Xuelin Liu, Xiaoyang Wang. Bu Sung Lee, Shiqi Wang, Anderson Rocha, and Weisi Lin. Fakebench: Probing explainable fake image detection via large multimodal models. IEEE Transactions Information Forensics and Security, 20:8730- 8745, 2025b. doi: 10.1109/IIFS.2 025.359721 URI.
feee org
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In A. Oh,
T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 14992-34916. Curran Associates, Inc., 2023. URL /ttp::s:s//occceen/ntgg////////t cttp://sscoouriisccc.cc.coooa Sactcapprr_lleesseeeeeeeeeeee fttess/pperr2223///5//22 pdf
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Leava-next: Improved reasoning, ocr, and world knowledge, January 2024. httts:tts:/// //11 Accessed: 2025-05-13.
ava vl. github. ho/blog//022 01- 30 1lava n
Xuannan Liu, Zekun Li, Pei Li, Huaibo Huang, Shuhan Xia, Xing Cui, Linzhi Huang, Weihong Deng, and Zhaofeng He. Mmfakehench: mixed-source multimodal misinformation detection benchmark for Ivlms. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (eds.). International Conference on Learning Representations, volume 2025, pp. //proceedings. iclr http::/c.......aa..ooooooooeooaa
2025a. URL https:
8 OTFOOf//ccsaaaecc-2pee---Caaeeee
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Mmbench: Is your multi-modal model an all-around player? In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Güi Varol (eds.). Computer Vision - ECCV 2024, pp. 216-233, Cham, 2025b. Springer Nature Switzerland. ISBN 978-3-031-72658-3, URL https: : www. www.ee..t www.a net/p appessecces 2024 4/ 2024/papers. papers BCCV C0959.0..... pdd.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannanch Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (eds.), International Conference on Learning Representations, volume 2024, pp. 23439-23554, 2024. wtt http:::psps//ppooeeeeeeeeeee http:///..ee..aaa..oeoo////reeee ic wttp://....oopar....eeee///////e
Paper Soweer-oooffeenccepcoe
Meta. The Llama 4 herd: The beginning of a new era of natively multimodal Al innovation, April //ai meta. www.blog//////cccccoocooooo.oo.o llama mumt intelligend
2025. | ht
e/ Accessed: 2025-04-05.
OpenAl. Introducing our latest image generation model in the API, April 2025. URL https: // 7/oeeaii. com/ zh- Ha-Haans-To http///ddd/mmmaeeeeeeeeatataaaee
image- www.reneeeioioooe -ptpreeeeenneeaataaaaaaaa
Accessed: 2025-
04-23.
12

---

Published as a conference paper at ICLR 2026
OpenAl. OpenAl o3 and o4-mini System Card. Technical report, OpenAL, April 2025. URL
http:s///cch...ona.....///////22 0112222677550222777--------9-91- 03 33-and--o-mmii--ssytte---rrr---
Accessed: 2025-04-16.
OpenAl, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan
Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, et al. Gpt-40 system card, 2024. URL https:
arx.i.orgg//o........ao......... wwwp///s/111 21276
Bryan A. Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In 2015 IEEE International Conference on Computer Vision (ICCV), doi: 10. 10.1109/CCV 20115.333 URL ht tps htthhttt:////peeeeeee http///pppccccss...oppppo thecv
pp. 2641-2649, 2015,
ent icev 2015/papers//lnnngee Bactneer_Fiickee ting Iec___cc__2__00 2015 paper. pdf
Flickr30x Entities Entities_Colles
SenseTimme SenseNova V6 Miaohua System Card, April 2025, URL https:://plaa...ommm www.:eeww...aaaaaaaaaaaaaaaa.eee
senova https/aaannn/e www.neecooooccccccccccccccceoeoe http://whhooooooooooooooo 2025-04-10.
Accessed:
Yichen Shi, Yuhao Gao, Yingxin Lai, Hongyang Wang, Jun Feng, Lei He, Jun Wan, Changsheng Chen, Zitong Yu, and Xiaochun Cao. Shield: an evaluation benchmark for face spoofing and forgery detection with multimodal large language models. Visual Intelligence, 3(1):9, Jun 2025. ISSN 2731-9008 doi: 10. (0.1007//42277-05-000777
URL
544267-025-0077 025-7999--
Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. OpenAI GPT-5 System Card, 2025. UtL.tttps://arae http://////aaaiooooo////////////
03267
ByteDance Seed Team. Seed1.6 Tech Introduction, June 2025. URL WtLLhttps:///et http::::://eeee.yyyydde.e Accessed: 2025-06-25.
Ce de.cco/e///eeee
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report, 2025.
19786
Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang. Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, et al. Glm-4.5v and glmh...vtthinking: Towards versatile multimodal reasoning with scalable reinforcement learning, 2026. URL http:::tttsctctt /ttpoi:////xvv..c/o org/ wwwpor/g///a///o..ee/.../..eeeee 07.01006
Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang. Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, and Conghui He. Mineru: An open-source solution for precise document content extraction, 2024a. /aa_aattps///xx..oorrg//a/s//
abs 18839
Chien Yao Wang, Alexey Bochkovskiy, and Hong- Yuan Mark Liao. Yolov7: Trainable bag-of- freebies sets new state-of-the-an art for real-time object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7464-7475, June 2023. URL https: http:::://ppaacceeeeeeeeeeeeeeee theoff com/ content http:///Ve22222/apa/ http:////coccee/////////000//aa/ wwwp:///pppppprr//ee/eaaaaa /Wang NeW_Statteeeeeeeee 2tt_P__222_paaae_ppp_0 wawpr.pperp...o
OLOv7 Twwiirrrraaaallee_e oag-of-Freebiess Sets New State the- for
eal eal-Time wtt_mmmm__oe__eeeeeettteeee cww.oorr________ 2023
Jin Wang, Chenghui Lv, Xian Li, Shichao Dong, Huadong Li, Kelu Yao, Chao Li, Wenqi Shao, and
Ping Luo. Forensics-bench: A comprehensive forgery detection benchmark suite for large vision language models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 4233 4245, June 2025a. URL httpt:tp::// httttss:://peaaccessstss http//ppaaccccaeeeeeepee thecvf. com/ httpeeetttccvv222////a/a/ B per.pdf
Wang Forensics-Bench A Comprehensive
Forqery Detection Benchmark Suite for Sifor_Larrreeee Large_ Vision CVPR
Vision_cVPR
2025 2025
pa
13

---

Published as a conference paper at ICLR 2026
Shengkang Wang. Hongzhan Lin, Ziyang Luo, Zhen Ye. Guang Chen, and Jing Ma. MFC- bench: Benchmarking multimodal fact-checking with large vision-language models. In ICLR 2025 Workshop Building Trust in Language Models and Applications. 2025b. URL hthppt:: //openreview. net. for. d=a2
Mma LG3Z
Weiyun Wang. Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internv1.55: Advancing open-source multimodal models in versatility, reasoning, and efficiency, 2025c. URL /tttps://axxiv.oom org. abs/25
08. 62265
Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi
Wu, Haotian Liu, Sadhika Malladi, Alexis Chevalier, Sanjeev Arora, and Danqi Chen. Charxiv: Charting gaps in realistic chart understanding in multimodal llms. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, Tomezak, and C. Zhang (eds.), Advances Neural Information Curran Associates, Inc., 2024b. doi: 10.5 ccpppper files/pap
Processing Systems, volume 37, pp. 113569-11 110099-1136997.
2202/079017-3669, URL 22000//000...00..lrrrss//////eee er 11000///111.....eeeeee0000000000 Paper-apprr Saai--DDaattte an L.Bechharks__racck pdf
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoe V
Le, and Denny Zhou. Shhain-fftthoughe prompting elicits reasoning in large language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances Neural Information Processing Systems, volume 35, pp. 24824-24837. Curran Associates, Inc., 2022. URL htw.ltteeee//ooeeeednn...eeeedos http:////..ppp..eea/////////////
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9556 9566,9557,200 2024. URL https: //openaccess. thecvf..c com/ centent httptt/c/RR22////peeee moda Understandi ng and ReasoRRsnnnnnnee Bench ve Mu CBP____P____00 2atpa_2222___aa_pp___2__
Yue__MM__ MMMU A Massi
ma for__vpR_________
YiFan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, Liang Wang, and Rong Jin. Mmeerealworld: Could your multimodal IIm challenge high-resolution real-world scenarios that are difficult for humans? In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (eds.), International Conference Learning Representations, volume 2025, pp. 89655-89701 2025. URL URU/tttps://prrcceciness
1r. http:///aa....ee/////////////aa.
9d 9d-Paper-Conferencce pdE
Zhenfei Zhang, Mingyang Li, and Ming-Ching Chang. A new benchmark and model for challenging image manipulation detection. Proceedings of the AAAI Conference on Artificial Intelligence, 38
(7):7405-7413, Mar. 2024. doi: 10. 10.1609/aaii.v33888 v3817. 225511. https::::///
AAPP:/AAAA/AA http://p//..../ee.eeeee/e/eeee// 28871
Haokun Zhou and Yipeng Hong. Diffusyn bench: Evaluating vision-language models on roal-world complexities with
synthetic benchmarks, 2024. URL https:: /arxiv.
htg/abs/2a /2////bb./...... 2406. 04470
14

---

Published as a conference paper at ICLR 2026
Appendix Contents
Benchmark Construction and Task Design
17
A.1
Dataset Statistics
17
A.I.I
Basic Statistics
17
A.1.2
Paper Sourcing and Curation
17
A.1.3
Distribution and Visualization of Seven Scenarios
17
A.1.4
Distribution of Fraud Types
18
A.2
2
Real Data Construction
19
A.3
Synthetic Data Construction
19
A.3.I
Extraction and Parsing
19
A.3.2
Splicing Forgeries
20
A.3.3
Copy-Move Forgeries
22
A.3.4
Al-Generated Forgeries
22
A.3.5
Duplication Forgeries
23
A.3.6
Text-Image Inconsistency Forgeries
24
A.4
Evaluation Task Design
25
A.4.I
Single- Forgery Identification and Localization (SMF-L))
25
A.4.2
Composite Manipulation Operations Identification (CMO-I)
26
A.4.3
Cross-Modal Inconsistency Identification and Localization (CMI-IL)
27
B
Experimental Details and Setup
28
Evaluation Environment
28
B. 2
Benchmarked Models
28
Evaluation Metrics and Protocols
29
B.3.1
Task-Specific Metrics
29
B.3.2 Balanced Robustness Index (BRI)
30
Supplementary Results and Analysis
31
Evaluation on World Datal
Impact of Prompting Strategies
32
Analysis of Protesrroessiig Results
37
Detailed Results by Fraud Type
38
D.I Splicing
38
D.2 Copy-Move
39
D.3 Al-Generated
40
D.4 Duplication
D.5 Text- Inconsistency
42
15

---

Published as a conference paper at ICLR 2026
Case Study
43
Splicing: Chart (Success Case)
43
E.2
Splicing: Medical Imaging (Failure Case]
44
Splicing: Chart (Failure Case)
45
E,4
Copy-Move: Stained Micrograph (Success Case
46
Copy-Move: Stained Micrograph (Failure Case
47
E.6
Copy-Move: Stained Micrograph (Failure Case
2)
48
Copy-Move: Stained Micrograph (Fallure Case 3)
49
E.8
Al-Generated: Medical Imaging (Success Case
50
E,9
Al-Generated: Micrograph (Failure Case)
................................
51
10
Al-Generated: Diagram (Success Case)
52
11
Al-Generated: Chart (Failure Case
................................
53
12 Al-Generated: Physical Object (Failure Case)
54
13 Al-Generated: Chart (Failure Case
................................
55
14
Duplication: Micrograph (Global Success Case)
56
E.
15
Duplication: Stained Micrograph (Global Failure Case)
................................
57
16
Duplication: Stained Micrograph (Local Failure Case
...............
58
E.
17
Duplication: Stained Micrograph (Local Failure Case 2)
59
18
Text-Image Inconsistency: Others (Numerical Success Case)
................................
60
E.
19
Text Text- Inconsistccnss Others (Numerical Failure Case
61
E.20
Text- Inconsistency: Others (Numerical Failure Case 2)
62
E.21
Text-Image Inconsistency: Others (Trend Failure Case 1)
63
22
Text-Image Inconsistency: Others (Trend Failure Case 2)
F
Document Extraction and Parsing Details
65
Justification of Pipeline Selection
65
R2
Error Analysis and Manual Correction
66
Use of LLMS
69
16

---

Published as a conference paper at ICLR 2026
A BENCHMARK CONSTRUCTION AND TASK DESIGN
A.1 DATASET STATISTICS
A.1.1 BASIC STATISTICS
Table 4 summarizes the fundamental statistics of THEMIS, illustrating the scale of the dataset and the complexity of the manipulation operations (e.g., average number of operations per synthetic im- age). Note that the paper categories are not mutually exclusive, an individual source paper may provide both original (authentic) samples and their manipulated (synthetic) counterparts. Comple- menting this, Table [ detailed breakdown of the authentic retracted papers, highlighting the distribution of specific real-wor world forgery techniques involved in our benchmark.
Table 4: Basic statistics of THEMIS.
Table 5: Basic statistics of retracted papers.
Statistics
Value
Statistics
Value
Total Papers
920
# Retracted Papers
41
# Totta Images
6,025
# Total Panels in Retracted Papers
194
Papers Providing Authentic Samples
483
# Splicing Samples
11
# Papers Providing Synthetic Samples
744
# Copy-Move Samples
17
Papers with Real Issues
41
# Direct Reuse Samples
74
Inter-Element Images
5,145
# Rotation Samples
28
Manip. per Synthetic Image (Avg.)
2.08
# Flip Samples
9
Images per Scenario (Avg.)
860.71
# Scaling Samples
8
# Segments per Mask (Avg.)
4.84
# Parameter Modification Samples
11
A.1.2 PAPER SOURCING AND CURATION
We first collected 1,432 retracted papers from the open-source repositories Retraction Watch and PubPeer. To ensure the scientific validity of our fraud-related dataset, multiple annotators con- ducted a rigorous manual auditing pipeline. Specifically: (1) we examined retraction notices and excluded cases unrelated to scientific paper fraud (e.g., voluntary withdrawals, journal mergers, and publisher-side issues); (2) we carefully inspected image-related evidence and discarded cases that only involved minor typographical formatting errors; and (3) we adopted a cross-review mecha- nism, where each retained paper was independently verified by at least two annotators. Following this process. only 41 papers were ultimately confirmed to strictly match our definition of fraud types.
Beyond the retracted papers, we also adopted rigorous multi-stage screening process to curate the broader source corpus, which was initially sourced from the PubMed Central open-access repository, for synthetic data construction. Concretely, we (1) excluded all non-open-access or incomplete arti- cles, (2) filtered out papers with missing figures, low-resolution images, ambiguous captions, and (3) performed manual inspection by multiple annotators to retain only those satisfying the require- ments of clarity, integrity, and domain relevance. After this multi-stage filtering, we selected 879 high-qualit papers from an initial pool of 5,253, forming the foundation of our synthetic dataset.
A.1.3 DISTRIBUTION AND VISUALIZZTION OF SEVEN SCENARIOS
In order to ensure comprehensive coverage of visual contexts in scientific papers, we categorized all images into seven representative scenarios: Chart, Diagram, Micrograph, Stained Micrograph, Physical Object, Medical Imaging, and Others. The Others scenario further included equations, data tables, covers, UI screenshots, legends, Western blots, and multi-panel compositions, thereby complementing the main scenarios to span the full spectrum of scientific visual elements. To obtain reliable annotations, we adopted semi-automatic pipeline in which Roubao-Seed-1 (Team] 2025) was employed to provide preliminary classification. followed by careful refinement and verification by more than three human experts. As shown in Figure 6, the final distribution of annotated sam- ples is presented as follows: Chart (1,403), Diagram (286), Micrograph (854), Stained Micrograph (2,490), Physical Object (191), Medical Imaging (220), and Others (581).
17

---

Published as a conference paper at ICLR 2026
Medical Imaging
Stained Micrograph
Micrograph
Medical Imaging
Physical Object
Physical Object
Diagram
Others
Stained Micrograph
Micrograph
Others
Diagram
Chart
Chart
hih
Figure 6: Distribution and visualization of scenarios.
A.1.4 DISTRIBUTION OF FRAUD TYPES
Table(oJetailn the fine-grained distribution of fraud types and manipulation operations specifically for the synthetic portion our dataset. Because real-world retraction notices rarely provide fine- grained documentation of the exact manipulation operations (e.g., scaling color temperature mod- ification), the 41 authentic retracted papers are tallied separately based on their macro-level fraud methods, previously detailed in Table
Table 6: Detailed distribution of fraud types and manipulation operations across the synthetic data.
Scope
Fraud Type
Manipulation Operation
# Samples
Copy-Move
Copy-Move
92
intra-Element
Image Inference Forgery Targeted Region Editing
74
Al-Generated
550
Splicing
Splicing
534
Global Direct Reuse
104
Global 180" Rotation
283
Global Vertical Flip
315
Global Horizontal Flip
289
Global Brightness
Global Contrast
185
Global Saturation
203
Global Hue
190
Inter-Element
Duplication
Global Color Temperature
171
Local Direct Reuse
103
Local 180 Rotation
336
Local Scaling
168
Local Vertical Flip
356
Local Horizontal Flip
368
Local Brightness
358
Local Contrast
364
Local Saturation
342
Numerical Inconsistency
60
Text-Image Inconsistency
Trend Inconsistency
90
18

---

Published as a conference paper at ICLR 2026
A.2 REAL DATA CONSTRUCTION
Data Collection. We constructed our real-data subset by systematically collecting 1,432 retracted papers from the open-source repositories Retraction Watch and PubPeer. To ensure coverage and diversity, we first aggregated metadata (title, authors, journal, racracti on reason) and then obtained the full-text PDFs where available. Since not all retracted cases in- volve visual fraud, we applied preliminary filtering to remove non-open-acces articles, incomplete manuseripts, and papers without figures. Following this process, a total of 41 high-quality papers were retained from an initial pool of candidates, providing the source corpus for subsequent annota- tion and validation.
Dataset Annotation. To precisely capture fraudulent manipulations in retracted papers, we adopted a multi-annotator collaborative annotation protocol. Each paper was reviewed independently by at least three annotators with expertise in biomedical image forensics and scholarly integrity. The annotation primarily focused on panel-level issues, such as Copy-Move and Splicing. Annotators marked tampered regions using bounding boxes and masks, and additionally recorded the issue type, subtype, and supporting evidence a structured JSON schema. To facilitate fine-grained reasoning tasks, each annotation was further linked to metadata such as caption text, page number, and the sgure-panel hierarchy.
Quality Control. Ensuring the reliability of annotations is critical. We implemented three-stage validation pipeline: (1) cross-validation among annotators, requiring consensus between at least two annotators for an issue to be retained; (2) expert arbitration, where disputed cases were resolved by senior domain experts; and (3) consistency checks, including overlap verification between bounding boxes and masks, and completeness of metadata. In addition, a random sample of 10% of the annotated papers underwent blind re-annotation, yielding an inter-annotator agreement score above 0.87, demonstrating the robustness and reproducibility of the dataset. Ultimately, from the 1,432 retracted papers, only 41 papers were confirmed to strictly match our definition of fraud types, forming curated gtad-standard subset of real fraudulent cases.
A.3 SYNTHETIC DATA CONSTRUCTION
A.3.1 EXTRACTION AND PARSING
We collected 5,253 papers from the PubMed Central open-access repository and performed pre- liminary expert screening to remove samples with irregular layouts, poor image quality, or lacking experimental visual content, ultimately retaining 879 high-quality papers as the core data source. Based on the original PDF files, we designed systematic extraction and parsing pipeline to con- struet a standardized and traceable annotation framework. As shown in Figure the entire process consists of six steps:
Step 1: Figure and Textual Content Extraction. Each PDF was parsed page by page using the
Fitz library to extract high-resolution figure composites. In paraliel, GPT-4o mini was employed to assist human experts in parsing and extracting figure-related captions and associated sentences, which were then cross-checked and corrected by at least three human annotators.
Step 2: Panel Segmentation. We customized YOLOv7 (Wang et al.. 2023) with task-specific modifications, including anchor adjustments, a confidence threshold of 0.5 (balancing quality and recall), and an overlap threshold of 0.35 (suitable for multi-panel figures). The detector was trained on our in-house annotations to generate dedicated weights for panel-level segmentation, thereby enabling fine-grained subdivision of figure composites.
Step 3: Coordinate Mapping. A transformation function was applied to convert figure coordinates back into page coordinates, and all results were consistently recorded as pixel-level bounding boxes (bbox_page) patpppieeeepeeep)eepppppppppp.p" to preserve semantic layout consistency.
Step 4: Ordering and Renaming, All panels were systematically ordered and renamed following toottom and left-to-right reading logic ensure interpretabilityy
https::/ppmmpoffteeet reacthedocs...to.o io/en////////
1
19

---

Published as a conference paper at ICLR 2026
Da
dil
Express
Tickoadd (9I7) lumus tondelion dua
2gng
Light
Hewr E. cone
20011
Extraction and Parsing, The regions highlighted with red boxes correspond figures, while those highlighted with blue boxes correspond to panels. Green boxes are used to verify whether the visualization is effective.
Figure 7: Stage 1:
Step 5: Visualization and Verification. For each paper, visualization files with bounding box over- lays were generated and manually inspected to validate the accuracy of segmentation and coordinate mapping.
Step 6: Structured Archiving. All parsing outputs were consolidated into hierarchical structure spanning saper-page sertr-page---urree figure panel, which encapsulates file paths, coordinates, identifiers, and potential issue flags to facilitate downstream analysis.
A.3.2 SPLICING FORGERIES
Generation Pipeline. The Splicing pipeline is designed to generate panel-level synthetic forgeries that remain visually plausible and covert. ensure that the manipulation strategy both reasonable and effective, we systematically identified panel pairs exhibiting the highest degree of visual simi- larity across multiple dimensions. All candidate panels were first resized to a standard resolution of 224 x 224 pixels and converted to grayscale, which reduces the influence of color bias and ensures consistent feature extraction across the dataset.
To evaluate similarity, we adopted multi-dimennionns metric that integrates three complementary features:
(1) Histogram Correlation (40%). We computed 256-bin grayscale histograms with cv2 calchist () and measured correlation coefficients between histograms with cv2. wwwpareliisttt..c..co () This captures distributional similarities in pixel intensities, reflect- ing color and brightness patterns across images. A weight of 0.4 was assigned to emphasize the importance of distribuutonnlevel consistency.
20

---

Published as a conference paper at ICLR 2026
(2) Structural Similarity Index (SSIM, 40%). Following the simplified SSIM formulation, computed the mean values (41, (2), variances (oî. 02). and covariance (012) of paired images, in- corporating stability constants C 0.01² and C2 = 0.03². SSIM quantifies structural fidelity by comparing luminance, contrast, and structural information, thereby ensuring that the internal orga- nization of panels is aligned. With a weight of 0.4, this component plays dominant role in guiding similarity assessment.
(3) Size Similarity (20%). We measured geometric compatibility by comparing the relative differ- ences in height and width between two panels. Specifically, similarity is defined as
Saise (1
hh - h2l
(1 [1 - w2l
-
x
max(w11, (1-max(w1,,0))
(1)
max(h11 h2)
where h1, hy denote heights and W1, W2 denote widths. This component, weighted at 0.2, prevents severe scale mismatches while retaining moderate variability.
The overall similarity score is computed as a weighted sum of the three components:
Stotal W1 X Shist.norm + W2 X max(Sseim 0) + w3 X Saizer
(2)
where Shist,norm +batt1 normalizes histogram correlation into [0, 1], W1 0.4, w2 0.4, and wa = 0.2. The final score Stotal ranges between 0 and 1, with values closer to indicating higher similarity between panels.
Through exhaustive search over all possible panel pairs, the method selects those with the high- est similarity scores candidates for splicing. This multi-dimensional assessment guarantees that paired panels share consistent distributional, structural, and geometric properties, thereby providing a strong basis for visually convincing splicing forgeries.
For each selected pair (panel A, panel B), foreground objects were extracted from the source panel using a combination of thresholding, morphological refinement, and comnected-comoonen analysis. To ensure accurate object segmentation, we adopted the following procedure:
(1) Otsu's 's thresholding algorithm was applied with the cv2. threshold function using the cv2. .HREEHHBBINAR mal threshold to convert the grayscale panel into a binary mask, effectively separating foreground objects from the background.
cv2. THRESHLOTSU flag, which automatically determines the opti-
(2) Morphological operations were performed with a x 5 kernel via the cv2, m.mphoorpoooo ogyFx () function: closing operation (cv2. WWW.M..RPHCCOSS. was applied to fill internal holes within objects, followed by an opening operation (cv2 MORPH OPEN) to suppress small noise points, thereby improving segmentation quality and object regularity.
(3) Connected-componett analysis c www.ooocceeeoooooeeeetttttttttia bel all contiguous regions in the binary mask and compute statisties such as area via cv2.CC CtcCC__SAATT AREA. The region with the maximum area was selected as the primary object to ensure completeness of the extracted foreground.
() to la-
The resulting object mask was post- processedd (hole filling and removal of minor components), re- sized, and geometrically aligned to match the dimensions of the target background panel, providing a consistent and precise object mask for subsequent splicing synthesis.
Pixel-level composition was performed via alpha blending to ensure that the extracted foreground object could be seamlessly integrated into the background of the target panel. Specifically, the binary object mask was first normalized to the range [0, and then used the alpha channel for weighted pixel-wise fusion. The blending operation follows the equation:
syn (z. y) = Loty (x, 3) x Mnorm (z, v) + (xy y) x (1 - 11--nooomaa (2),),
(3)
where L (z y) denotes the synthetic image pixel position (z.z). (z, y) is the foreground object pixel, (nz(2, 3) is the background pixel, and Mnorm (x, y) is the normalized mask value at (x, y). computed represents the background mask, ensuring that the blending weights always sum to one, thereby preventing unnatural brightness artifacts at the splice boundary.
Mnorm (2. (z. = (a,, y) M). The complement (1 Mnorm (z. v))
To guarantee dimensional consistency, all images and masks were resized to the same spatial res- olution using ev2. resi i ze () with the cv2. INTER NEAREST interpolation method to preserve
21

---

Published as a conference paper at ICLR 2026
the discreteness of the binary mask. The single-channel mask was expanded into three-channel format via st.ckack/ () to match the RGB image representation. This process ensured that both the object and background were aligned at the pixel level, which facilitated precise and artifact-free composition.
To increase data diversity, two variants were synthesized for each panel pair: (1) the object from panel A composited onto the background of panel B, and (2) the object from panel B compos- ited onto the background of panel A. Each synthesis produced a structured triplet (original source panel, synthetic panel, object mask), along with metadata including panel identifiers, similarity scores, dojjecttto-bcckrruuddd area ratio, and transformation parameters.
Quality Control. To guarantee both the quality of the generated Splicing data and its practical util- ity for the Single-Mode Forgery Identification and Localization task, we adopted rigorous quality control and filtering protocol. Specifically, we identified and removed visually meaningless manipu- lations, such as cases where the inserted object was entirely incompatible with the target background or where the bacckroudb size ratio was severely distorted. In addition, we excluded sam- ples exhibiting overly obvious artifacts, including unnatural seams at the splice boundary, abrupt color transitions, or visible traces of post-processing. The filtering process integrated multiple crite- ria, including similarity score thresholds, object area ratio constraints, and post-componn composiiinnn quality evaluation, thereby ensuring that the retained Splicing samples were both visually plausible and suf- ficiently challenging. This guarantees that the final dataset provides high-quality and well-structured samples to support scientific paper fraud detection research. After these steps, a total of 534 high- quality Splicing samples were retained for downstream analysis.
A.3.3 COPY-MOVE FORGERIES
Generation Pipeline. This pipeline was developed to construct panel-level Copy-Move forgeries, simulating the common pattern of duplicating regions within scientific images. We employed the SAM (Kirillov et al. (023) to perform automatic object segmentation and mask generation. The raw masks were further refined through denoising, morphological opening and closing operations using cv2 mo morpho loyyEx (), and contour smoothing, resulting in clean and well-defined binary masks. From these masks, valid objects were selected based on an area ratio criterion, retaining only those whose area occupied between 10% and 60% of the panel region.
For each valid object, we computed its centroid and adaptively defined grid based on the ob- ject's width and height, with the grid step size determined as step = max(20, min(Wobj, hobj)/ /2) The farthest grid position from the source centroid was chosen as the target location for object placement. To avoid overly regular results, random noise bias was added to the computed coor- dinates, while enforcing strict boundary constraints to ensure that the duplicated object remained within the valid image region. During this process, corresponding binary mask of the tampered region was generated and saved. Each operation ultimately yielded structured triplet (orig oriigina image, forged image, tampering mask), providing standardized data for the Single- Mode Forgery Identification and Localization task.
Quality Control. To ensure the usability and realism of the synthetic data, a multi-stage screen- ing strategy was applied: (1) Objects that fell outside the designated area ratio threshold (smaller than 10% or larger than 60%) were discarded, as these produced unrealistic manipulations either too trivial or too dominant; (2) Synthetic images with duplicated regions that extend beyond valid panel boundaries or overlap with semantically meaningless background (e.g., blank margins, figure legends, or annotation text) were excluded; (3) Samples were filtered if the duplicated region intro- duced strong visual artifacts, such as unnatural seams, sharp edges, or visible copy boundaries that would make detection trivial; and (4) Human annotators performed manual validation stage to view a subset of the generated data, removing images that failed to preserve visual plausibility. After these steps, total of 92 high-quality Copy-Mov samples were retained for downstream analysis.
A.3.4 AI-GENERATED FORGERIES
Generation Pipeline. To construct an Al-Generated forgery dataset at the panel level, we designed controlled strategies to ensure both realism and diversity while retaining detailed fidelity to authentic academic images. Two representative forgery types were considered:
22

---

Published as a conference paper at ICLR 2026
(1) Image Inference Forgery:
The entire panel globally reconstructed to simulate complete Flux (Labs et al.) 2025) model.
replacement, primarily using the
Partial manipulations were performed while preserving the major- ity of image content, where localized regions were replaced or edited based on textual prompts or binary masks. This class of forgeries
(2) Targeted Region Editing:
implemented using mainstream generative models including Stable Diffusion (Esser et al. 2024), SenseNova V6 Miaohua (Sense'Time Time 2025). and GPT-Image-I (OpenAll 2025). During synthesis, we systematically recorded the original panel, the forged panel, and the corresponding region masks (the latter only for targeted editing), thereby form- ing structured triplets (original image, forged image, mask/ mask/metadas metadata). In total, more than 5,000 AI- Al-Generated panels were produced as negative samples.
Quality Control. To guarantee the authenticity and research value of the synthetic dataset, a rig- orous multi-stage quality control pipeline was adopted. At the automated stage, rules were applied to filter out invalid or failed generations, including samples with excessive blur, severe distortions, corrupted text, or local edits that conflicted with global semantics. At the manual stage, human annotators further inspected and excluded low-quality cases that evaded automated filtering but still exhibited implausible details. Through iterative refinement combining automated screening and human verification, total of 74 high-quality entirely Al-Generated images and 550 partially AI- Generated images were retained, forming reliable and challenging negative sample set for the subsequent Identification and Localization task.
A.3.5 DUPLICATION FORGERIES
Generation Pipeline. This pipeline focuses on constructing Duplication forgeries at the panel level, which are divided into two categories: Global Duplication and Local Duplication.
(I) Global Duplication: This process is performed by directly reusing the entire panel after applying a randomly selected geometric transformation, including horizon- tal flip (cv2.flip) (image, 1) vertical flip (cv2.flip (image, 0)), or 180 rotation (cv2. rotate (image, (OTATE-180) ). In addition, up to two random parameter modifications are optionally applied, such as brightness modification, contrast adjustment, saturation control, hue shifting, and color temperature variation. These operations are implemented via OpenCV routines (e.g., cv2. cevtollrrrroooooooo (), cv2. www.ertttcco...... lebbs (), and cv2. LUT () ), ensuring realistic duplication while preserving overall image integrity.
(2) Local Duplication: This category specifically targets stained micrographs, which typically ex- hibit highly self-similar textures and repetitive structural units. Such properties make localized duplications visually subtle and challenging to detect. To ensure sufficient detail, only panels with resolutions larger than 350 x 350 pixels considered. For each eligible image, initial over- lapping crop is performed by randomly selecting one of three geometric modes: diagonal, vertical window, horizontal window, producing two sub-panels (denoted panel A and panel B) with clear overlapping region.
Forgery generation then follows innovative crop-then-transform strategy. Based the initial crop, we adaptively perform secondary cropping step to precisely extract blocks containing the overlapping area. Panel A is preserved as reference, while panel B undergoes random manipulation operations, including geometric transformations (i.c., rotation, flip, or scaling) and parameter mod- ifications (i.e., brightness, contrast, saturation, hue, or color temperature). To maintain dimensional consistency under scaling, we design reverse cropping mechanism: given random target scaling factor, the required crop size inversely computed so that, after scaling, panel B's block perfectly matches panel A in size.
All manipulation operations are recorded in metadata, together with a key indicator, the duplication ratio, defined as the proportion of the reused area within panel A relative to its total block area. Each forgery sample is stored as a structured triplet (original panel, duplicated panel, metadata), supporting downstream model evaluation.
Quality Control. To guarantee usability and prevent trivial cases, multiple filtering mechanisms applied. Panels that fail the minimum resolution requirement (350 350 pixels) are excluded, as they lack sufficient texture for meaningful duplication. For Global Duplication, samples pro- ducing strong distortions obvious artifacts after geometric parameter operations discarded.
23

---

Published as a conference paper at ICLR 2026
For Local Duplication, pairs with excessively small overlapping areas, severe misalignment af- ter transformations, or visible boundaries at splice seams are removed. A manual inspection stage further eliminates unrealistic duplications, such cases where duplicated regions overlap with non- semantic areas (e.g., blank margins, legends, or annotation text). This multi-stage process ensures that the retained Duplication dataset exhibits both high visual plausibility and adequate difficulty for forgery detection research. After these steps, a total of 917 high-quality Global Duplication samples and 2,124 high-quality Local Duplication samples are retained.
A.3.6 TEXT-IMAGE INCONSISTENCY FORGERIES
Generation Pipeline. We adopt two controlled types of caption/relattdd manipulations the panel level: Numerical Inconsistency and Trend Inconsistency.
(1) Numerical Inconsistency: We randomly select the figure caption or the set of related sentences, and replace each selected numeric token with a different number (ensuring the replacement is not identical to the original).
three numeric expressions appearing in
(2) Trend Inconsistency: We randomly select one to three trend-describing keywords (e.g., in- crease, rise, and upward) and replace each with an appropriate antonym (e.g., decrease, fall, and downward).
The generation process is subject to the following constraints:
(1) A candidate token for modification must not be located inside the same punctuation-delimite substring (comma, period semicolon). Concretely, split captions and related sentences by the delimiters [.,;] ;| and only allow substitutions on tokens belonging to different segments so as to avoid altering multiple values/claims within single tight clause.
(2) A candidate keyword/number must not be directly present as OCR text within the image (i.c., must require reasoning beyond raw visual OCR). We verify this by running OCR on the panel and rejecting any candidate token that appears verbatim in the OCR output.
(3) For numerical replacements, we enforce syntactic and unit consistency (preserve units when present, avoid inserting non-numen numeric characters), and ensure the replaced value differs meaningfully (e.g., absolute or relative difference above a small threshold) to avoid near-identical noise edits.
(4) For trend replacements, substitutions are chosen from a curated antonym mapping and are ap- plied with part-of-speech checks preserve grammaticality.
The pipeline draws candidate modifications automatically (via regex for numeric tokens and curated trend-keyword list for trend tokens), applies candidate substitutions, and records for each generated sample the tuple (original text, modified text, modi if calificciion type (numerical/trend)) modified token indices, original token values, replacement values).
Quality Control. All generated Text-Image Inconsistency samples undergo manual proofreading by at least two human experts. During the quality control phase, pruned the candidate pool by filtering out nonsensical substitutions and manually adjusting borderline cases to ensure high fidelity. Through this manual validation process, guarantee that the final dataset retains 150 high-quality, reliable samples suitable for evaluating models' ability to identify and localize cross- modal inconsistency.
24

---

Published as a conference paper at ICLR 2026
A.4 EVALUATION TASK DESIGN
To evaluate the comprehensive perception and reasoning capabilities of MLLMs across the five fraud methods, we designed three core evaluation tasks. We provide the standardized prompt templates for each task below, including the specific system prompts and user instructions used in our benchmark.
A.4.1 SINGLE-MODE FORGERY IDENTIFICATION AND LOCALIZATION (SMF-IL)
This task corresponds to three forgery methods: Splicing, Copy-Move, and Al-Generated. requires the model to judge whether the input academic panel is forged and select the fraud type from five options (Splicing, Copy-Move, Al-Generated, No Forgery, and Not Sure). If forged, the model must further provide block-based localization of the forged region.
Minele-Mode Forgery Identification and Localization (SMF-IL)
System Prompt:
Role:
You are an image forensics expert. Detect the image forgery type and localize needed.
Task:
Single-choice task (pick EXACTLY one):
A) Splicing different sources of content combined in one image; cues: unnatural seams, boundary mismatches, inconsistent lighting/textures. Note: Splicing NEVER involves duplicated regions.
B) Copy-Move region copied and pasted within the SAME image; cues: duplicated patterns/objects with same noise/texture but shifted//otated//caleedd Reminder: any part of the image is duplicated, it MUST be classified Copy-Move not Splicing.
C) Al-Generated - image wholly or partially Al-genoraaeed/modiiiee cues: synthetic
textures, implausible details, inconsistent semanties.
D) No Forgery - the image appears genuine; no strong evidence of manipulation.
E) Not Sure insufficient evidence to decide.
Input:
ONE image (a panel/subfigure).
textual description of how the image is partitioned into numbered regions mms.ccpption).
Mask Rules:
mask IS REQUIRED and your choice is A/B/C - output indices of forged regions (comma-separated)
mask is NOT required (e.g., whole-image forgery) - <MASK> must be EMPTY, even if you chose A/B/C.
choice D/E - always leave <MASK> EMPTY.
STRICT OUTPUT (exact format, no extra text):
<MA.KK>commm-s<parrdce region numbers; EMPTY when not required when choice is D/E</MASX>
<EXPLANATION: Drr--rrffff reasoning: state what visual cues led to your choice; if A/B/C with mask, mention where forgery cues were observed.<
User Instruction:
Single-panel image for forensics.
(Non-global Al-generation for the input image)
Mask IS REQUIRED you choose A/B/C; output indices from mask_caption.
(Global Al-generation for the input image)
Mask is NOT required for this question; LEAVE <MASK> EMPTY.
mask caption: (mask.caption or [no caption]]
25

---

Published as a conference paper at ICLR 2026
A.4.2 COMPOSITE MANIPULATION OPERATIONS IDENTIFICATION (CMO-I)
This task corresponds Duplication. Given pair of panels, the model required to identify whether duplication relationship exists and the specific types of manipulation operations (Direct Reuse, Scaling, Rotation, Flip, and Parameter Modification) involved. Multiple selections are per- mitted, covering positive, negative, and uncertain cases.
Composite Manipulation Operations Identification (CMO-I)
System Prompt:
Role:
You are an image forensics expert specializing in cross-panel duplication.
Task:
Multi-select task (choose ALL that apply):
A) Direct - near-identical content across panels without geometric transform; many
keypoints align at similar scale/position.
B) Scaling content in one panel appears in another after zoom-in/zoom-our.
C) Rotated one panel a rotated version of another (e.g., 90%/1809).
D) Flipped
panel is mirrored (horizontal/verticee version of another. E) Parameter-Moddffed - duplication exists but with image-level adjustments (e.g.,
brightness, contrast, and color).
F) No Duplication - there is no convincing duplication across panels.
G) Not Sure evidence is insufficient ambiguous.
Rules:
Pick A/B/C/D/E for every duplication type that appears. Multiple types may co-exist (e.g., scaling + rotated = choose B,C).
none of A-E apply, choose F (no duplication).
If uncertain, choose G (not sure). G with A-E.
Do NOT mix
Input:
You will receive MULTIPLE panels (subfigures) from the same figure. A short text may describe panel numbering, but you do not need to output indices.
STRICT OUTPUT (exact format):
CCHOICES<comma-sen
letters from (A.BCC.D.E.FG)):
uppercase only< CHOOCCES>>>>
<EXPLANATION -INI-N-BBA Brief reasoning: which panels appear duplicated and why: mention like keypoint matches, geometric transforms, or parameter changes. </ecesss///PPP LEPALTINIONN
User Instruction:
Multiple panels for cross-panel duplication forensics. panel note: (mask caption [no caption]]
26

---

Published as a conference paper at ICLR 2026
A.4.3 CROSS-MODEE INCONSISTENCY IDENTIFICATION AND LOCALIZATION (CMI-IL)
This task corresponds Text-Image Inconsistency. requires the model judge whether Nu- merical or Trend inconsistency exists based on the input image and its corresponding text. If detected, the model must further localize the minimum sentence unit with inconsistency the text and provide corrected content.
Cross-Modal Inconsistency Identification and Localization (CMI-IL)
System Prompt:
Role:
You are an expert in text-image consistency for scientific figures.
Task:
Single-choice (pick EXACTLY one):
A) Numerical Inconsistent - the numeric valuees/assertios in text contradict the figure's numbers/labelsssaales.
B) Trend Inconsistent - the trend described in text (e.g., increase, decrease, peak, monotonicity ordering) contradicts the figure's ' visual trends.
C) Consistent text matches the figure.
D) Not Sure - image/text too ambiguous or unreadable to decide.
Rules:
Decide A/B/CDD.
you choose A B, localize ALL problematic parts.
Output <PARTS> with one or more entries: capti on if the issue is in the caption, related if the issue in the related text.
Output <SENTENCES> with ALL minimal inconsistent sentences.
minimal sentence is the smallest segment separated by '' ' "r 1 5. +.1
If you choose C or D, set <PARTS>none</ / and leave <SENTENCES> EMPTY.
Input:
You will receive:
-ONE figure image.
figure_caption: the figure's caption.
regure short context related to the figure.
STRICT OUTPUT (exact format):
<PARTS>comma-seppin list of cuptio//elaedhhoe<//PRR>>>>
<SENTENCES>list ALL minimal inconsistent sentences;
EMPTY for #DD</SENTENES>>>
<EXPLANATIONS Brief reasoning: what in the text contradicts which element in the figure; or why consistent/not Det.e///EXAAAAATOON
User Instruction:
Figure for text-image consistency checking.
figure caption: [figure.captionn or [empty]}
figure related: rriigure_reaate
[empty]}
Remember: choose A/B/C/D, and if A/B, return PART=caption/related and single minimal SENTENCE.
27

---

Published as a conference paper at ICLR 2026
B EXPERIMENTAL DETAILS AND SETUP
B.1 EVALUATION ENVIRONMENT
For evaluation experiments involving two input settings, most model inferences are conducted via the OpenRouter A Exceptions include Intern VL3.5-8B and the LLaVA series, which are down- loaded from Hugging Face and executed locally, as well as the Doubao series, which is accessed via the Volcano Engine AP
The system configuration is summarized below:
CPU: Dual-socket Intel Xeon Gold 6148 (2.40 GHz), 20 cores per socket, 80 threads total
GPU: NVIDIA A40 (48 GB VRAM each)
GPU Driver: 575.57.08
CUDA: 11.8
cuDNN: 8.9.6 (compiled with CUDA 11.8)
Operating System: Ubuntu 22.10
B.2 BENCHMARKED MODELS
We evaluate 16 MLLMs in total, including nine proprietary and seven open-source models. The proprietary models come from OpenAl (Singh al. 2025
OpenAI| 2025). Google (Comanici al.)
2025), Alibaba (Bai et al. 2023), ByteDance (Team, 2025). Anthropic (Anthropic 2025) and Zhipu
AT (Team al. 2026)), representing mainstream industrial progress. The open-source models
released by Google Team et al., 2025), Meta (Meta (025), Alibaba (Bai et al. (025). the LLaVA
community (Li al. 2025a Liu al. 2024) and OpenGVLab (Wang al. (025c). covering
different scales and architectures of open-source MLLMs. The following list details these models.
Table 7: Detailed information benchmarked MLLMs.
Provider
Version
Parameters
Access Links
Proprictary MLLMs
GPT-5
N/A
ht ht-pa://opanroutrr.oo /operal
OpenAl
OpenAl o4- mini-high
N/A
at http:///appnrrrttr..//cooaaaaaap 2.5 flash
Gemini 2.5 Flash
N/A
Google
Gemini
Alibaba
Qwen- ML-Ma
NIA
htsps://gesuooueee..////////// max
Doubao-Seed- Beecccd-----hineee 66-thinking
N/A
wwm. http://wwww........o///////eeaaa be
ByteDance
Doubao-See-- Seweede---vviiiiie 16-vision
NA
Anthropic
Claude Sonnet 4.5
NAA
Zhipu Al
GLM-4.5V
NA
htt.ps://opentootee.aa// si/glm 4.5v
Open-Source MLT.V
Google
Gemma 27B
27B
Meta
Llama Maverick
17B
htpps//gaarrotee..ansss-llll 11ama 4-navesick
Qwen2.5-VL-3228
32B
htpp:///peeuccueeee.///eeeeee http:/ttw...../e/e/eeeeee/oee v1 32b instruct
Alibaba
Qwen2.5-VL-722
72R
7B
LLaVA Community
n- 000-ppohh"
LLaVA-NeXT-34B
34B
ht http////aaggggggaaaaaooo///////r v1.6 34b hf
nt ot..
OpenGVL ab
IntemVL.3.5.RR
SB
https://openrouuer.cc
https://huggings
http://hhhgginnga....... ccce.coo
https://www.tt http://wwwww/ cengine.com com
28

---

Published as a conference paper at ICLR 2026
B.3 EVALUATION METRICS AND PROTOCOLS
B.3.1 TASK-SPECIFIC METRICS
To comprehensively assess model performance, we employ five core metries tailored to three tasks. For all metrics described below, the final performance reported in our results represents the mean score across all samples in the respective task.
Task 1: Single-Mode Forgery Identification and Localization (SMF-IL). This task involves two complementary metries:
(I) Forgery Type Identification. The model is required to categorize the input image into specific forgery types or mark it as authentic single-choice classification). We use Accuracy (ACC) as the primary metric:
ACC
1 v:), N
Σ
where is the number of samples, Di is the predicted class, and yi is the ground-truth label.
(2) Forged Regions Localization. For images identified as forged, the model must output binary mask indicating the tampered region. We quantify the overlap between the predicted mask M and the ground-truth mask M using Intersection-over-Umiioo (IoU):
M M M U M
IoU
(5)
Task 2: Composite Manipulation Operations Identification (CMO-I), Since single image may undergo multiple manipulation operations (e.g., scaling and rotation), this is formulated as a multi- label classification problem. We adopt the Set-based F1 Score measure the match between the predicted set of operations S and the ground-trun
| |$]
Recall |
Precision
(6)
[S]
2 Precision Recall
F1
(7)
Precision + Recall
Task Cross-Modal Inconsistency Identification and Localization (CMI-IL). This task in- volves two complementary metrics:
(1) Inconsistency Identification. The model determines the existence and type inconsistency (Numerical or Trend). We use ACC as the primary metric:
ACC
1 N
Σ 1(0 =1))
(2) Inconsistency Localization. The model must extract the specific text span containing the error. We compute the Text-based F1 Score by comparing the predicted text span /' with the ground-truth span T:
2. [
2.
Fl
(9)
Here, || denotes the length of the span.
29

---

Published as a conference paper at ICLR 2026
B.3.2 BALANCED ROBUSTNESS INDEX (BRI)
Beyond task-specific metrics, define composite score holistically measure both overall ac- curacy and robustness across tasks.
Score Normalization. Since different tasks use different metric scales (e.g., IoU ACC), first normalize the raw scores to ensure comparability, Let Ra,y denote the raw score of model i on task dimension j. The normalized score $ij computed via Min-Mac normalization across all M benchmarked models:
R minke RE
$1 (1..e REJ min My R
(10)
For each model i, we collect its normalized performance vector across the five metric dimensions:
S; 1 81,2; 81,3, 85,4, 81,5].
Index Calculation. We then compute the mean performance μ4 and the stability penalty A; (repre- senting the performance gap across tasks):
Si.j
(11)
j=1
A; max St.z) min(s.,).
(12)
Finally, the Balanced Robustness Index (BRI) is defined as:
BRI: = 14 X A.,
(13)
where is a tunable penalty weight (set to 0.25 in our main experiments to balance average per- formance and cross-task variance). A higher BRI indicates that model only achieves strong average performance but also avoids over-specialization (i.e., maintaining a small A:).
Sensitivity Analysis of X. To examine the stability of model rankings with respect the penalty weight X, we report BRI scores under different values in Table g
Table 8: Sensitivity of the BRI under different values of X.
Model
x=
A=
0=033
X=0.40
GPT-5
64.56
56.15
53.35
47.75
OpenAl o4-mini-high
63.57
52.34
48.60
41.11
Twen-VL-Man
57.31
49.83
47.34
42,35
Gemini 2.5 Flash
57.67
44.70
40.36
31.71
Doubao-Seed Swwd--eeedd---eeee 1.6tthinkin
48.43
37.14
33.37
25.85
Doubao-Seed- 1.6-vision
46.17
33.47
29.24
20.78
Gemini 2.5 Pro
46.10
31.97
27.25
17.83
GLM-4.5V
40.66
31.57
28.54
22.48
Claude Sonnet 4.5
38.74
29.96
27.03
21.18
Owen2.5-VLL-72.
58.60
47.16
43.34
35.71
Intern
51.58
38.73
34.45
25.89
Llama 4 Maverick
40.00
34.78
33.04
29.56
32.79
23.59
20.52
14.38
LLaVA-NeXT-34B
31.68
18.40
13.97
5.11
Qwen2.5-Ve L-L-32B
29.76
18.22
14.37
6.68
Gemma 3 27B
17.37
9.59
7.00
1.81
Overall, the relative ranking of top-performing models (e.g., GPT-5, OpenAl o4-mini-high, and remains stable across different penalty weights. As À increases, absolute BRI val- ues decrease, reflecting a stronger emphasis on stability over peak task performance.
30

---

Published as a conference paper at ICLR 2026
C SUPPLEMENTARY RESULTS AND ANALYSIS
C.I EVALUATION ON
DATA
Table 9: Performance comparison on synthetic versus real data. We report the Id Score (Iden- tification Score) and Loc Score (Localization Score) across subtasks corresponding to different fraud types.
Splicing
Copy-Move
Duplication Id Score
Model
Data Type
Id Score
Loc Score
Id Score
Loc Score
Proprietary MLLMs
Synthetic
43.51
16.67
72.73
36.41
33.32
GPT-5
Real
27.27
23.96
60.00
33.39
23.22
Synthetic
41.49
10.44
77.89
29.78
30.34
OpenAl o4-mini-high
Real
9.09
11.99
60.00
31.35
36.04
Synthetic Real
30.37
40.07
87.40
48.34
23.33
0.00
15.15
13.33
3.70
20.95
Synthetic
63.39
56.72
67.56
46.98
24.96
Gemini 2.5 Flash
Real
9.09
38.55
80.00
47.58
49.58
Open-Source MLL.Ms
Synthetic
36.40
51.66
77.27
55.16
16.75
Qwen2.5-VL-72B
Real
0.00
54.54
100.00
36.27
22.39
Synthetic
23.37
17.97
51.24
28.50
19.01
Ilama 4 Maverick
Real
0.00
8.66
6.67
2.22
39.75
Synthetic
25.39
27.78
28.87
32.80
12,20
Gemma 27B
Real
0.00
15.25
40.00
17.04
42.18
Synthetic
37.70
31.62
66.14
39.71
22.84
Avg.
Real
6.49
24.01
51.43
24.51
33.44
As shown in Table our synthetic data exert comparable visual fraud reasoning pressure to real data, they fully reflect diverse real-world fraud behaviors and cover the known fraudulent operations reported retracted papers. A detailed comparison across specific subtasks reveals several key observations:
Synthetic data surpass real data in difficulty for the Duplication subtask because they incorporate multiple compounded manipulations. These include geometric transformations (i.c., horizontal flip, vertical flip, and 180 rotation) and parameter modifications (i.e., brightness, contrast, sat- uration, hue, and color temperature). which are generally more complex than the direct reuse or scaling duplication commonly observed in real cases. For example, Gemini 2.5 Flash reached 49.58% Id Score on real cases, yet dropped to only 24.96% on synthetic data.
Model performance on synthetic and real data closely aligned for the Copy-Move subtask. For instance, GPT-5 achieved a 36.41 % Loc Score on synthetic data, compared to 33.39% real data.
The primary exception lies in the Splicing subtask, where real samples remain more challeng- ing due their inherently more sophisticated merging patterns. Real-world forgery operations often employ more complex and subtle splicing strategies, such as localized contrast, brightness adjustment, and texture smoothing the merged boundaries.
Moreover, model performance synthetic data serves reliable indicator of real-world capabil- ity. For example, GPT-5, the strongest model on synthetic visual fraud reasoning tasks, also exhibits relatively stable performance real samples. Thus, synthetic data not only provide comprehensive
31

---

Published as a conference paper at ICLR 2026
testing of reasoning capabilities but also function predictive signal for real scientific review scenarios.
The convergence of performance between synthetic and real data demonstrates that our fraud data generation pipeline can continuously produce high-quality and scalable fraud samples despite the scarcity and prohibitively high annotation cost of real-world cases. This enables robust evaluation of five core capabilities (i.e., Expert Knowledge Utilization, Visual Recognition, Spatial Reason- ing, Region Localization, and Comparative Reasoning) within an extensible testbed for multimodal scientific fraud assessment.
An improved splicing synthesis method will be incorporated in future work to better mimic these nuanced operations and more effectively expose current models' limitations in boundary perception.
C.2 IMPACT OF PROMPTING STRATEGIES
Table 10: Performance comparison of different MLLMs under various prompting strategies on synthetic data. We report the Id Score (Identification Score) and Loc Score (Localization Score) for 5 fraud methods: SPL (Splicing), CM (Copy-Mova) AIG (A--Genera,ed), DUP (Duplication), and THI (Text-Image Inconsistency). Llama: Llama 4 Maverick.
Prompting Strategy
SPL
CM
AIG
DUP
Model
Score
Loc Score
Score
Lae Score
Id Score
Loc Score
M Score
Id Score
Loc Score
None
43.51
16.67
72.73
44.26
33.32
GPT5
CaT
(+1.34)
25.16 (48.49)
74,46 (+1.73)
38.74 (+2.33)
(-0.90))-90)
30,10 30.10 (+10.77
33,95 (40.63)
66.00 (+5.33) 25.16 (-2.28)
Few-Shot
46.47 (+2.96)
18.80 (+2.13)
71.97 (40.76
31.47 (-4.4)
43.26 (-1.00)
27.46 (+8.13)
23.37
51.24
58,19
16.08
Llama
CaT
(1-4887))
49.34
(131.37)
60.21 (+8.97)
28.47 (-000))
59.08 (40.89)
34,06 (-114.3)
28.29 (49.28)
61.66 (47.66)
13.64 (-2.44)
Few-Shot
31.99 (+8.60)
9.25 (-8.72)
62.05 (+10.99
14.93 (-13.57)
40.90 (-17.39)
13.06 (-6.68)
We further compare two prompting methods, CoT (Wei al. 2022) and few-shot (Brown et al. 2020). across different fraud types. The results in Table indicate that:
CoT generally provides consistent gains, but the magnitude varies with model strength. For GPT- 5, the improvements were modest (e.g., +1.34% on SPL-Id and +1.73% CM-Id), suggesting that model with strong intrinsic reasoning ability benefits only marginally. By contrast, Llama Maverick showed large boosts with CoT (e.g.. +4.87% on SPL-Id, +8.97% on CM-Id, and +9. +9.28% on DUP-Id), demonstrating that stepwise prompting disproportionately enhances weaker models by guiding them toward structured reasoning steps.
Few-shot prompting, however, yields mixed and unstable effects. While improved certain scores such as SPL-Id (+8.60%) and CM-Id 10.79% for Llama 4 Maverick, it simultaneously caused severe drops in other subtasks, including AIG-Id (-17.39%) and AIG-Loc (-6.65%). GPT-5 exhib- ited similar trend: slight improvements in some subtasks (c.g., +2.96% on SPL-Id and 18.1%% on AIG-Loc) but declined in others (c.g.. -4.94% on CM-Loc). These inconsistencies suggest that the few-shot strategy may bias models toward local demonstration patterns rather than enabling generalizable reasoning, leading to degradation when facing diverse fraud types.
Overall, the evidence indicates that CoT more reliable prompting strategy for scientific im- age forensies, providing stable and interpretable improvements, whereas few-shot prompting lacks robustness and exhibits limited practical value in our setting.
We provide the enhanced prompting templates used in our benchmark, including CoT and few-shot strategies.
32

---

Published as a conference paper at ICLR 2026
Shain-of-Thoughh Single-Mode Forgery Identification and Localization
System Prompt:
Role:
You are an image forensics expert. Detect the image forgery type and localize needed.
Task:
Single-choice task (pick EXACTLY one):
A) Splicing different sources of content combined in one image; unnatural seams, boundary mismatches, inconsistent lighting/textures. Note: Splicing NEVER involves duplicated regions.
B) Copy-Move - a region copied and pasted within the SAME image; cues: duplicated
patterns/objects with same noise/texture but shifted/rotated/scalled Reminder: any part of the image is duplicated, it MUST be classified as Copy-Move, not Splicing.
C) AI-Generated image wholly or partially -l-geerrated/mmdiffe.. cues: synthetic
textures, implausible details, inconsistent semanties.
D) No Forgery the image appears genuine; no strong evidence of manipulation.
E) Not Sure insufficient evidence to decide.
Input:
ONE image (a panet/subfigure).
textual description how the image is partitioned into numbered regions (mask.caption).
Mask Rules:
If mask IS REQUIRED and your choice is A/B/C -> output indices of forged regions (comma-separated),
If mask is NOT required (e.g., whole-image forgery) <MASK> must be EMPTY, you chose A/B/C.
If choice is D/E - always leave <MASK> EMPTY.
Reasoning Requirement CChin--ff-Thoughty
First, carefully observe the image and list suspicious visual cues such textures, boundaries, lighting, duplication, and semantic plausibility. Next, compare against all five options (A-E) and explicitly explain why each is ruled in or ruled out.
Then, narrow down the most plausible choice. If A/B/C is chosen, explain where the forgery appears and why the mask is required or not.
Keep the reasoning concise but clearly step by step.
STRICT OUTPUT (exact format, no extra text):
<CHOICE> A/B/C/D//B////////////// CHOICE> <MA.K>>comma comma-s-aatt- separated region numbers; EMPTY when not required or when choice D/Ex W/K//MSK>>
<EXPLANATION Provide the step-by-step reasoning process as described above, ending with justification for the final choice, </EXE/ LANATION>
33

---

Published as a conference paper at ICLR 2026
Chain-of-Thoughh Composite Manipulation Operations Identification
System Prompt:
Role:
You are an image forensics expert specializing in cross-panel duplication.
Task:
Multi-select task (choose ALL that apply):
A) Direct near-identical content across panels without geometric transform; many
keypoints align at similar scale/position.
B) Scaling content in one panel appears in another after zoom-in/zoom-out,
C) Rotated - one panel a rotated version of another (e.g.. 90%/180")
D) Flipped one panel a mirrored (horizontal/verticaa) version of another.
E) Parameter-Modiffed - duplication exists but with image-level adjustments (e.g.,
brightness, contrast, and color).
F) No Duplication there is no convincing duplication across panels.
G) Not Sure evidence is insufficient ambiguous.
Rules:
Pick A/B/C/D/E for every duplication type that appears. Multiple types may co-exist
(e.g., scaling rotated = choose B,C).
none of A-E apply, choose F (no duplication).
If uncertain, choose G (not sure).
- Do NOT mix or G with A-E.
Input:
You will receive MULTIPLE panels (subfigures) from the same figure.
A short text may describe panel numbering, but you do not need to output indices.
Reasoning Requirement (Chain-of- Thhought)::
First, systematically compare all panels to check for visual similarities.
Next, analyze the nature of the similarity: decide whether it is a direct reuse, scaling, rotation, flip, parameter modification.
Then, evaluate each option A-E one by one, ruling them in or out.
none of A-E apply, consider F (no duplication).
If evidence is weak or ambiguous, consider G (not sure).
<EXPLANATION, provide step-by-step reasoning: what similarities were found, which transformations are involved, and why the final choice(s) were made.
STRICT OUTPUT (exact format):
CCHOICES>comma-eeps uppercase only</COOICES:
letters from (A.BC.DD.EF..)):
DOXPLANATTION-SSee-bbe reasoning: explain which panels appear duplicated, what transformations or adjustments are observed, and why the final choices are selected.< Doocttdd/EXXPLAAATIOOO
34

---

Published as a conference paper at ICLR 2026
Chain-of-Thought Cross-Modal Inconsistency Identification and Localization
System Prompt:
Role:
You are an expert in text-image consistency for scientific figures.
Task:
Single-choice (pick EXACTLY one):
A) Numerical Inconsistent the numeric vallees/assertions in text contradict the figure's numbers/labelsssaales.
B) Trend Inconsistent the trend descrihed in text (e.g., increase, decrease, peak, monotonicity ordering) contradicts the figure's 's visual trends.
C) Consistent text matches the figure.
D) Not Sure - image/text too ambiguous or unreadable to decide.
Rules:
Decide A/B/C/D.
you choose A or B. localize ALL problematic parts.
Output <PARTS with one or more entries: caption if the issue is in the caption, related if the issue in the related text.
Output <SENTENCES> with ALL minimal inconsistent sentences.
minimal sentence is the smallest segment separated by '., '?, T, 12 ".. ":.
If you choose C or D, set <PARTS>nones /PARTS and leave <SENTENCES> EMPTY.
Input:
You will receive:
ONE figure image.
figure caption: the figure's caption.
rigune_related: short context related to the figure.
Reasoning Requirement CChain-ofTThoughe
First, carefully observe the figure and extract key information (eg., numeric values, axis labels, visual trends, and peaks).
Next, analyze the caption and related text step by step, checking their numeric claims and described trends.
Then, compare systematically against all four options (A-D), explaining why each is ruled in out.
Finally, choose the most plausible option.
or B is chosen, clearly indicate whether the inconsistency appears in the caption or the related text, and list all minimal inconsistent sentences. D is chosen, set In <EXP <EXP<<EPPBB LIRLLLAATION write out the reasoning step step (observation checking text - option comparison - final decision).
and leave <SENTENCES> empty.
STRICT OUTPUT (exact format): <CHOICE>A/////DD/CCHH
-PARTS>comma-spprrtted list of cuptio/cclaedhooe<//PRR>>>>
<SENTENCES>list ALL minimal inconsistent sentences;
EMPTY for C/D< SENTENCES>
<EXPLANATION CETOON-Ste----steyy reasoning process as described above, ending with justification for the final choice. < EXPPANNTIONS
35

---

Published as a conference paper at ICLR 2026
Few-Shot Single-Mode Forgery Identification and Localization
System Prompt:
Role:
You are an image forensics expert. Detect the image forgery type and localize needed.
Task:
Single-choice task (pick EXACTLY one):
A) Splicing different sources of content combined in one image; unnatural seams, boundary mismatches, inconsistent lighting/textures. Note: Splicing NEVER involves duplicated regions.
B) Copy-Move - a region copied and pasted within the SAME image; cues: duplicated
patterns/objects with same noise/texture but shifted/rotated/scalled Reminder: any part of the image is duplicated, it MUST be classified as Copy-Move, not Splicing.
C) AI-Generated image wholly or partially -l-geerrated/mmdifeed cues: synthetic
textures, implausible details, inconsistent semanties.
D) No Forgery the image appears genuine; no strong evidence of manipulation.
E) Not Sure insufficient evidence to decide.
Input:
ONE image (a panet/subfigure).
textual description how the image is partitioned into numbered regions (mask.caption).
Mask Rules:
If mask IS REQUIRED and your choice is A/B/C -> output indices of forged regions (comma-separated).
If mask is NOT required (e.g., whole-image forgery) <MASK> must be EMPTY, you chose A/B/C.
If choice is D/E - always leave <MASK> EMPTY.
STRICT OUTPUT (exact format, no extra text):
<CHOICE> A/CI////////BIDDE//CCCCCECC
<MASK-ccomm--spparatte region numbers; EMPTYwhen not required or when choice is D/E</MASK>
LEXPAANTION> AriiiiiTO--Biifiiiii reasoning: state what visual cues led to your choice; if A/B/C with mask, mention where forgery cues were observed. < PEXPLANATIONS
Few-Shot Examples:
Example Al-Generated (C)
CHOIICE>C<<CHCCCE>>
<MASK></MASK>>
TEXPLAIAATIINN-SSytttttiiig artifacts and implausible micro-textures suggest
AIGC.</EXPLANATIION>
Example 2 Copy-Move (B)
<CHOICE>B</CHOICE>
<MASK></MASK>>
EEXPLANAATIOO--Duliiccaaee patch with the same noise/texture implies
Copy-Move.</EXPLAAATTT
Example - Splicing (A)
<CHOIC>>A</CHOICCB>
<MASK></MASK>
BOXPLANNATION-BBunaayy inconsistencies and lighting mismatch without duplication
indicate D/eeinng<EEXPLAAAAAE
36

---

Published as a conference paper at ICLR 2026
C.3 ANALYSIS POST-PROCESSING RESULTS
Table 11: Performance comparison of different MLLMS under various input perturbations synthetic data. We report the Id Score (Identification Score) and Loc Score (Localization Score) for fraud methods: SPL (Splicing), CM (Copy-Move), AIG (AI-Generated), and DUP
(Duplication). Gemini: Gemini 2.5 Flash.
Post-
SPL.
CM
AIG
DUP
Model
Processing
Score
Loc Score
Id Score
Score
Id Score
Score
Id Score
None
43.51
16.67
72.73
36.41
44.26
19.33
33.32
Gauss
23.37 (-20.14)
3.43 (-13.24)
34.85 (:37.88)
12.81 (-23.60)
37.22 (-7.04)
7,73 (-11.60)
20.84 (-12.48)
GPT-S
JPEG
33.59 (-9.92)
(-3.93)
48.86 (-23.87)
32.19 (4.222)
(-0.77)
(:5.01)
(-9.76)
Scaling
36.63 (-6.88)
14.05 (-2.62)
48.48 (-24.25)
42.37 (45.96)
41.11 (-3.15)
16.67 (-2.66)
25.08 (-8.24)
Nome
63.39
56.72
67.56
46.98
35.45
38.87
24.96
Gauss
48.13
33.63 (-23.09)
50.38 (-17.18)
39.74 (-7.24)
31.11 (-4.34)
23:20 (-15.67)
25.84 (+0.88)
Gemini
JPEG
64.82 (+1.4) 60.01 (43.29)
54.93 (-12.63)
42.46 (-4.22)
33,90 (-1.55)
39.93 (+1.06)
25.56 (+0.60)
Scaling
61.37 (-2.02) 57.46 (+0.74)
46.97 (-20.59) 43.86 (-3.12)
34.44 (-1.01) 40.75 (+1.88)
25.31 (+0.35)
We further introduce three common image perturbations (Gaussian blur, JPEG compression, and scaling) systematically evaluate model robustness under input distortions. Our analysis yields the following observations:
Gaussian blur directly disrupts fine-grained textures and boundary information, and the results showed that it had the most severe impact, causing substantial performance drops across nearly all subtasks. By contrast, JPEG compression primarily introduces quantization noise and artifacts, while scaling alters image resolution and aspect ratios; both operations also led to degradation, with the effects varying across models and subtasks.
Under Gaussian blur, GPT-5 experienced a drop of over 23% in localization performance for Copy-Move, while Gemini 2.5 Flash suffered substantial degradation in localization performance for both Splicing and Al-Generated.
These findings highlight that current MLLMs are highly vulnerable to mild perturbations, as they rely excessively on delicate low-level features. The lack of robust representations makes them susceptible to common distortions and post-processing operations, thereby limiting their reliability in practical forensic applications.
37

---

Published as a conference paper at ICLR 2026
D DETAILED RESULTS BY FRAUD TYPE
D.1 SPLICING
Table 12:
Experimental results THEMIS synthetic data for Splicing. We report the ACC, IoU, and Set-based F1. The best and
second-best scores are highlighted.
Splicing
Model
ACC
IoU
Set-based F1
Proprietary MLLMs
GPT-5
43.51
16.67
20.27
OpenAl 04-mini-high
41.49
10.44
12.81
Qwen-VI.n VL-Max
30,37
40.07
52.84
Gemini 2.5 Flash
63.39
56.72
65.08
Doubao-Seed -eendo Freed11.6e 6-thiinking
35.67
12.09
16.24
Doubao-Seek -ivvision
20.84
31,42
42.49
Gemini 2.5 Pro
30.60
46.65
59.50
GLM-4.5V
29.23
8.54
9.76
Claude Sonnet 4.5
29.71
14.95
17.20
Open-Source MILMs
Qwen2.5-VL-720
36.40
51.66
$7.60
Intern VL.3.588
30.97
43.38
46.37
Llama 4 Maverick
23.37
17.97
L. VaaVA--nterleaee-78
41.80
35.97
47.50
32.00
50.70
52.82
Gemma VL-32B
Qwen2.5-VL-32B
30.31
5.91
7.23
25.39
27.78
34.62
the performance of different MLLMs on Splicing reveals clear divergence between proprietary and open-source models:
As shown in Table 12
Among proprietary models, Gemini 2.5 Flash achieved the highest overall performance, with notable gains across all three metrics (ACC 63.39%, loU 56.72%, and Set-based F1 65.08%), indicating its strong capacity for both forgery identification and fine-graines localization. con- trast, GPT-5 reached the second-best ACC (43.51%) but lagged significantly in IoU (16.67%) and Set-based Fl (20.27%), suggesting that while it can often detect the existence of splicing, it struggles to precisely delineate tampered regions. Interestingly, Qwen-VL-Max VL-Max exhibited the opposite trend, with relatively low ACC (30.37%) yet competitive localization performance (IoU 40.07% and Set-based F1 52.84%), reflecting imbalance between visual recognition and region localization.
On the open-source side, Owen2.5-VL-72B delivered competitive localization scores (IoU 51.66% and betebased F1 57. 57.60%), in some cases rivaling or surpassing proprietary counterparts. However, its classification ACC remained modest (36.40%) highlighting a persistent gap in holis- tic decision-making.
These results collectively indicate that while proprietary models currently dominate balanced performance, open-source models have made notable progress in localization. key takeaway is that splicing forensics challenges models differently at the identification and localization levels, with many MLLMs excelling one but both. Bridging this discrepancy represents critical direction for advancing visual fraud reasoning.
38

---

Published as a conference paper at ICLR 2026
D.2 COPY-MOVE
Table 13: Experimental results on THEMIS synthetic data for Copy-Move. We report the ACC, IoU, and Set-based F1. The best and second-best scores are highlighted.
Copy-Move
Model
ACC
IoU
Set-based F1
Proprietary MLLMS
GPT-5
72.73
36.41
46.11
OpenAl o4-mini-high
77.89
29.78
39.32
Qwen-VI.-Mel
87.40
48.34
60.94
Gemini 2.5 Flash
67.56
46.98
$7.90
Doubao-Seed-1 16-thinking
74.17
13.58
18.47
Suabao-Seed-1.6-viiiiee
61.78
26.97
36.65
Gemini 2.5 Pro
47.46
46.61
60.28
GLM-4.5V
58.43
22.23
26.75
Claude Sonnet 4.5
75.66
44.12
50.31
Open-Source MLLMS
Qwen2,5-VL-72B
77.27
55.16
64.58
Intern VL.35VL
58,42
40.24
47.75
Llama 4 Maverick
51.24
28.50
35.47
LLaVA-Imeerlaave78
47.55
25.22
36.99
LLaVA-NeXT-34B
84.00
45.25
53.35
Qwen2.5- VL-32B
57.48
18.93
20.79
Gemma 27B
28.87
32.80
41.58
Table 13
presents the results of THEMIS on Copy-Move, revealing distinct trends compared to Splicing:
Among proprietary models, Qwen- WL-N-V achieved the highest identification ACC (87.40%) demonstrating strong capability in determining whether Copy-Move forgeries exist. However, its localization performance was moderate (IoU 48.34% and Set-based Fl 60.94%), indicating that while detection is reliable, precise delineation duplicated regions remains challenging. Inter- estingly, Gemini 2.5 Flash attained competitive balance with relatively strong localization (IoU 46. 46%98% and Set-based Fl 57.90% albeit at lower ACC (67,56%), suggesting better sensitivity to duplicated textures
when overall identification is less stable.
On the open-source side, Owen2.5-VL--2B 55.16% and Seseb F1 64.58%), outperforming most proprietary counterparts. This highlights the potential of scaling open-source MILMS to large capacities for improving spatial reasoning forensic analysis. Additionally, I.LAVA-NeXT-34 achieved competitive performance (ACC 84.00%, loU 45.25%, and Set-based F1 53.35%), positioning models capable of approaching the performance of proprietary models in overall effectiveness.
VL-72B showed the strongest localization capability (IoU
of the few open-source
Overall, these results suggest that open-source MLLMs are progressively bridging the performance gap with their proprietary counterparts in the Copy-Move subtask, particularly in terms of localiza- tion precision at the regional level.
39

---

Published as a conference paper at ICLR 2026
D.3 AI- GENERATED
Table 14: Experimental results on THEMIS synthetic data for Al-Generated. We report the
ACC, IoU, and Set-based F1. The best and second-best scores are highlighted.
Al-Generated
Model
ACC
IoU
Set-based F1
Proprietary MLLMS
GPT-5
44.26
19.33
22.21
OpenAl o4-mini-high
35.67
19.79
22.86
Owen-VL--Me
35.43
35.85
43.87
Gemini 2.5 Flash
35.45
38.87
44.76
Laabao--Seed--aee 16-thinking
36.57
15.19
18.09
Doubao-Seed-L.--visioo
27.54
29.90
35.11
Gemini Pro
47.20
54.66
GLM-4.5V
54.51
10.63
12.41
Claude Sonnet
30.43
20.98
25.00
Open-Source MILMS
Qwen2,5-VL-72B
51.06
35.57
41.58
IntemVL3.5-8B
67.00
33.19
37.77
Llama 4 Maverick
58.19
19.71
23.46
LJ JLaVA--nterleave-78
46.93
32.06
41.56
VA-NeXT3333
34.00
34.01
40.13
Qwen2.5- VL.32B
39.24
7.45
9.17
Gemma
34.23
26.44
32.92
The results in Table 14 indicate that AI-Generated forgery detection remains highly challenging for current MILMM:
Proprietary models struggled to provide consistent performance, with Gemini 2.5 achiev- ing the highest localization scores (IoU 47.20% and Set-based Fl 54.66%) but exhibiting very low ACC (14.90%). Gemini 2.5 Flash showed similar trend, reaching competitive localization (IoU 38.87% and Set-based F1 44.76%) while maintaining only moderate ACC (35.45%) These findings suggest that proprietary models can sometimes identify forged regions but lack robust discrimination between Authentic and Al-Generated panels.
In contrast, open-source models demonstrate relatively stronger classification ability. Intern VL3.5-8B attained the best ACC (67.00% significantly outperforming all proprictary coun- terparts, though its localization quality remains weak (IoU 33.19% and Set-based F1 37.77%) Qwen2.5-VL-72B struck a more balanced profile, combining strong classification (ACC 51.06%) with reasonable localization (IoU 35.57% and Set-based F1 41.58%), highlighting the benefits of scaling open-source architectures for Al-Generated forgery detection.
Overall, the results reveal a notable discrepancy between identification and localization performance. Proprietary models excel marginally at spatial consistency checks but fail reliably separate Au- thentic from AI- Al-Generated panels, whereas large open-source models achieve stronger binary dis- crimination but weak spatial grounding.
40

---

Published as a conference paper at ICLR 2026
D.4 DUPLICATION
Experimental results on THEMIS synthetic data for Duplication. We report the Set- based F1, EM (Exact Match), and SPC (Subset Partial Credit). EM requires strict equality with the ground truth; SPC awards credit for correct subsets only if no incorrect elements (false positives) are predicted. The best and second-best scores are highlighted.
Table 15:
Duplication
Model
Set-based F1
EM
SPC
Proprictary MILMS
GPT-5
33.32
19.05
22.72
OpenAl 04-mini-high
30.34
17.12
21.35
Owen-VL-Mar
23.33
12.13
15.29
Gemini 2.5 Flash
24.96
15.78
17.41
Doubao- Seendo--eeed--ae -ecod-1.6-tle L6-thinking
20.22
14.24
17.08
Doubao-Seed- 1--viiioon
45.13
17.99
19.95
Gemini Pro
21.49
15.63
19.63
GLM-4.5V
21.84
15.73
20.05
Claude Sonnet
21.86
15.30
17.98
Open-Source MLLMS
Qwen2,5-VL-72B
16.75
11.54
14.35
Intern IntemVLL3.588B VL3.5-8B
33.28
10.34
10.89
Llama 4 Maverick
19.01
13.90
16.07
8.01
0.33
0.33
LLaVA-NeXT-38
10.73
0.11
0.11
Qwen2.5 Dwen2.5-VL-32B 5-VB-B
15.26
14.28
14.95
Gemma 3 27B
12.20
9.57
10.35
Table 15 reports results Duplication. Overall, performance across all models is markedly weaker compared to other fraud types, underscoring the intrinsic difficulty of duplication forensics:
Among proprietary models, Doubao-Seed 111--116-vvii vision achieved the highest Set-based F1 (45.13%) suggesting relatively stronger discriminative ability, but its EM (17.99%) and SPC (19. (19.95%) re- mained low, indicating limited capability in producing strictly correct multi-label predictions. GPT-5 attained the second-best Set-based (33.32%) and led in SPC (22.72%), while also yield- ing the best EM (19.05%), showing modest advantage in producing fully correct predictions. Nevertheless, even these leading scores are far from satisfactory, reflecting the challenge of simul- tancously identifying multiple manipulation operations with high precision.
In contrast, open-source models exhibit consistently poor performance. Qwen2.5-VL-322 and Llama Maverick achieved slightly higher EM (14.28% 900% and SPC (14.95%/16.077)
compared to other open-source MLLMs, but remained substantially behind proprietary models. LLaVA-Interleave-72 and -.AV-NN---334 failed almost entirely, with EM and SPC close to zero, suggesting limited capability in reasoning about multiple manipulation operations.
Taken together, these results highlight Duplication the hardest subtask within THEMIS, where neither proprietary nor open-source models show satisfactory performance. The stark contrast with other fraud types reveals that detecting subtle geometric transformations and parameter modifica- tions across panels requires fine-grained relational reasoning and robust representation of geometric and parameter variations. Addressing Duplication thus remains critical bottleneck for advancing visual fraud reasoning with MLLMs.
41

---

Published as a conference paper at ICLR 2026
D.5
INCONSISTENCY
Table 16: Experimental results on THEMIS synthetic data for Tmxt-Image Inconsistency. We report the ACC, Macro-F1, and Text-based Fl. The best and second-best scores are highlighted.
Text Text-Image Inconsistency
Model
ACC
Macro-F1
Text-based F1
Proprictary MILMs
GPT-5
60.67
44.95
27.44
OpenAl o4-mini-high
66.33
48.58
32,22
Qwen-VL-Mas VL-Max
56.00
33.74
15.36
Gemini 2.5 Flash
36.33
25.30
28.24
Doubao-Seed-1 16-thinking
60.00
44.56
31.71
Doubao-Seed-I 1-6-vsion
37.00
27.98
30.43
Gemini 2.5 Pro
44,67
31.42
37.31
GLM-4.5V
53.67
39.83
22.69
Claude Sonnet 4.5
35.67
27.33
27.42
Open-Source MLLMS
Qwcn2.5-VL-772
61.33
36.51
12.32
Intern VL3.5-8B
55,00
25.61
4.78
Llama 4 Maverick
54.00
33.95
16.08
LLLVA--nterleaveh
50,00
16.78
12.57
ALaVA-NeXT-3AA
41.33
24.74
4.28
Qwen2.5 VL--2BB
58.33
38.71
17.94
Gemma 3 27B
31.67
16.91
21.41
Table T6 summarizzss the results on Text- Inconsistency, evaluated with ACC, Macro-F1, and Text-based Fl:
Compared to other fraud types, this subtask poses unique challenges as it requires models to in- tegrate textual semantics with fine-grained visual reasoning. Proprietary models showed stronger overall performance, with OpenAl o4-mini-high achieving the highest ACC (66.33%) and Macro- F1 (48.58%), suggesting robust alignment between textual cues and image content. GPT-5 fol- lowed closely, ranking second in Macro-Fl (44.95% while also maintaining competitive ACC (60.67%) Notably, Gemini 2.5 Pro achieved the best Text-based F1 (37.31 %), indicating rela- tive advantage in grounding inconsistencies to specific regions, although its overall ACC (44.67%) lags behind leading models.
On the open-source side, Qwen2.5- performed competitively in terms of ACC (61.33%) nearly matching proprietary leaders, but its Macro-F1 (36.51 %) and Text-based F1 (12.32%) were substantially weaker. This discrepancy highlights a gap between classification accuracy and bal- anced, fine-grained reasoning across categories. Other open-source MLLMs such as LLaVA- NeXT-34B exhibited consistently low performance, with Macro-Fl and Text-based Fl dropping below 25% and 5%, respectively, reflecting their limited capacity for multimodal semantic con- sistency reasoning.
Overall, the results reveal that while certain proprietary MLLMs (e.g., OpenAl o4-mini-hin high and GPT-5) demonstrate promising capability in detecting Texc- Inconsistency, the subtask re- mains far from solved. The relatively low Text-based F1 scores across all models indicate that fine-grained grounding of inconsistencies is especially challenging, underscoring the need for future models better integrate visual fraud reasoning with spatial localization.
42

---

Published as a conference paper at ICLR 2026
E CASE STUDY
We provide 22 representative cases selected from the outputs of GPT-5, our best-performing model, comprehensively cover the full scope THEMIS. These cases span all seven academic scenarios (Chart, Diagram, Micrograph, Stained Micrograph, Physical Object, Medical Imaging, and Others) and are systematically organized across the five fraud types: Splicing. Copy-Move, AI-Generated (Image Inference Forgery/Targeted Region Editing), Duplication (Global/Local), and Text- Text-Image Inconsistency (Numerical/Trend).
E.1 SPLICING: CHART (SUCCESS CASE)
Splicing: Chart
2
3
4
5
6
Question
The lask requires model act image forensics expert, determine the of forgery the given panel, accurately mark the tampered regions when necessary, and strictly follow the specified output format.
Options
Splicing different souroes content combined in one image: cues: unnatural seams, boundary mismatches,, inconsistent lighting/texturrss Note: Splicing NEVER involves duplicated regions. image; cues: duplicaled patlerss/ojjects same noise/texture but shiftedirotated/scaleed Reminder: if any part of the image is duplicated, it MUST be Copy-Move, Splicing.
Capy-Move region copied and pasted
C) Al-Generated - image wholly or partially Al-generatedimocilleed cues: synthetic textures, implausible
details,
semantics.
Fargery - the image appears genuine; no strong evidence manipulation.
Not Sure insufficiet evidence decide.
Slicing Rule
The image was divided into 3-row by 2-column equal-area grid, resulting in 6 blocks. The numbering goes left right, bottom: Block top-left Block top-right Block middle-left Block 4= middle-ight Block bottom-left Block 6 bottom-right.
Issue A) Splicing
[2,4,6]
(GPT-5) Model Answer: A) Splicing
Model Explanation
grid background, uniike the left white area. This abrupt boundary and missing underfying details indicate an overiaid (splicing).
sharp verticall
right
region from
Region Localization
[2.4.6]
Figure Success case by GPT-5 on Splicing.
43

---

Published as a conference paper at ICLR 2026
E.2 SPLICING: MEDICAL IMAGING (FAILURE CASE)
Splicing: Medical Imaging
4P1Q
0.91 LV Length LVid 62 (AIC)
B
1000ml 45%
40m
3
5
6
Question
The task requires the model to act as an image forensics expert, determine the type forgery given panel, accurately mark tampered regions when necessary, stricily follow the specified output format.
Options
Splicing different sources content combined one image; cues: unnatural seams, boundary mismatches, inconsistent ghhting Note: Splicing NEVER involves duplicated regions.
duplicaled patternskabjects same noise/exxture but shitedrrtatethscaldd Reminder: if any part of the image is duplicated, MUST be
Copy-Move region copied
image;
classified Copy-Move Move, not Splicing.
Al-Generated image wholly or partilly Algennerteedmoddfedd cues: synthetic textures, implausible
details, Inconsistent semantics.
Forgery image appears genuine; no strong insufficient evidence to dedid..
manipulation.
E) Not Sure
Slicing Rule
by 3-column equal-area grid, resulting blocks. The numbering goes left to right, top to battom: Block top-left, Black 2 top-middle, Block top-right, Block bottom--iddle. Block 6 bottom-right.
The image divided
battom-lefl, Block 5
Issue A) Splicing
[1,2,3,455.]]
(GPT-5) Model Answer: A) Splicing
Explanation
largee triangular region the center is unnalurally uniform black, lacking the specklennsise expected in mismalched overlays- indiicavvieee of pasted black petch covering content. This spiiced area spens blocks 2-3 (top centerlighh) and most of the bottom (blocks
ultrasound view. has eharp diagonal boundaries
Region Localization
[2.3,4,5.6]
Analysis
Error Analysis
The annotation indicates that all blocks (1-6) were spliced, but the model only marked blocks 2-6, missing follows: first. the black region Block limited coverage blumed edges and low contrest, making less prominent and easier to overlook compared to other blocks; second, the block contains parameter text and interface elements, which the model tands regard normal UI, thereby
Black The
large triangle with the regular 2x3 grid and only slightly touches Block leading the model to misjudge this block as unaltered uncertain, model adopts to avoid faise positives, which explains why it did not classify Block as forged region.
weakening perception splicing traces; moreaver, spliced
align
during grid-based mapping: finally.
strategy under-marking
Lack Five Core Reasoning Capabilities
Region Localization, Visual Recognition
Figure 9: Failure case by GPT-5 on Splicing, The model correctly identified the forgery type but mislocalized it.
44

---

Published as a conference paper at ICLR 2026
E.3 SPLICING: CHART (FAILURE CASE)
X
Splicing: Chart
2
3
5
Question
The lask requires the model to act as an image forensics expert, determine the lype of forgery in the given panel, accurately mark the tampered regions when necessary, and strictly follow the specified output format.
Options
unnatural boundary mismatches,, inconsistent lighting/texttrrs. Nate: Splicing NEVER involves duplicated regions.
Splicing different of content combined in
Copy Move region copied and pasled within the SAME cues: duplicalled patiermstobjecls with Reminder: if any part of the image duplicated, it MUST
same nolse/texture but shiftedrotatediscale diassified Copy-Move, not Splicing.
Al-Generated image wholly or partially Al-generatedmoddiled; cues: synthetic textures, implausible
details, inconsistent semantics.
Forgery the image appears genuine; no strong evidence manipulation.
E) Not Sure insufficietl evidence to decide.
Slicing Rule
The image was divided into 2-row by 3-column equal-area grid, resulting in blocks. The numbering goes left right, top to bottom: Block = first row. left; Block first row, middle: Block 3 first right: Block second row,
row, middle; second row, right.
A) Splicing [1,2,34,5,6]
(GPT-5) Model Answer: D) No Forgery X
Model Explanation
The line chart shows multiple series with consistent gridlines and styling: no clear duplicated regions, unnatural seams,, syntheliofmplausbble textures. strong evidence manipulation.
Reglon Localization
Emoty
Analysis
Error Analysis
Large models inherent limitations in visual forgery delection. First, there clear training data distribution blas: most models leamn forgery cues from natural images, relying on edge breaks, lighting inconsistencies, or
texture repetitions, while low texture and highly structured line charts rare in training, leaving the models coupling insufficient. Models tend to rely only pixel-level surface inspection, whereas anomalles in line charts often *st data logic (such pixel discontinuitess logether "visual slyle source inference splicing likelihood." they easily overlook problems. Third, models evidence
lacking sensitivity to "chart splicing" features. Second, their perception-ressonte
inconsistent styles or differing sources)
ability
they prefer to posilives, and they are also insensitive to whole chart splicing mask annolations, leaning inslead toward searching localized enomalies
conservative judgment
forgery'
leading misclassification.
The Lack of Five Reasoning Capabilities
Expert Knowledge Utillizaiion, Visual Recognition, Spatial Reasoning, Comparative Reasoning
Figure 10: Failure case by GPT-5 on Splicing, The model misidentified and mislocalized.
45

---

Published as a conference paper at ICLR 2026
E.4 COPY-MoVE: STAINED MICROGRAPH (SUCCESS CASE)
Copy-Move: Stained Micrograph
Merge
Question
The task requires the model act as image forensics expert, determine the type of forgery in the given panel, accurately mark the tampered regions when necessary, and strictly follow the specified output
Options
Splicing different of content combined image; unnatural boundary mismatches, inconsistent hightinghextures. Note: Splicing NEVER involves duplicated regions. copied and pasted within the SAME image; duplicated patternskobjects with same nolse/texture but shifeddrolteddsceleed Reminder: If any part of image duplicated, MUST be classified Copy-Move, not Splicing.
Copy Move
Al-Generated image wholly or partially Alggmeeratsd/moodffeee cues: synthetic textures, implausible
details, inconsistant semantics.
Forgery the image appears genuine; strong
manipulation.
E) Not Sure insufficiett evidence to decide.
Slicing Rule
2-raw by 1-column equal-area grid, resulting numbering goes top to boltom:: Block = tap Block 2 batlom
image
Issue B) Copy-Move
[1,2]
(GPT-5) Model Answer: B) Copy-Move
Model Explanation
cell-shaped surrounding orange speckles/blue appear duplicaled (indicated by the dolled boxes), showing identical lextures/ncise hallmark of copy-move. The duplicated content spans image, blocks flagged.
regions with same
Region Localization
[1,2]
Figure 11: Success case by GPT-5 on Copy-Move.
46

---

Published as a conference paper at ICLR 2026
E.5 COPY-MoVE: STAINED MICROGRAPH (FAILURE CASE 1)
Copy-Move: Stained Micrograph
Question
The task requires the model to act as an image forensics expert, determine the type of forgery in the given panel, accurately mark the tampered regions when necessary, and strictly follow the specified output format.
Options
unnalural seams, boundary mismatches, inconsistent tightinggexttures. Note: Splicing NEVER involves duplicated regions.
Splicing different sources of content combined in
B) Copy-Move region copied and pasted within the SAME image; cues: duplicated pattemslotjects with any part image duplicated,
same noise/txttrre but shiftedirolatdd/saal daassified as Copy-Move, nol Splicing.
Al-Generated image wholly partially Al-generaledmmodfied;; cues: synthetic implausible
details, inconsistent semantics.
Forgery image sppears genuine; no strong E) Not Sure insufficient evidence to decide.
manipulation.
Slicing Rule
3-column equal-ares grid, resulting right, top to bottom: Block = first row. left Block 2 = first row, middle Blook 3 row, right Block 4 second Block second midale Block 6 second right.
image
numbering goes
Issue Copy-Move [1,3,4,6]
(GPT-5) Model Answer: B) Copy-Move
Model Explanation
Copied clusters purple nells with identical shapes/Nextures appear pasted within the same image, especially upper-left and lower-left areas. These palches have sharp, irregular edges and blend inconsistently with background, indicating copy-move forgery in Blocks and
Region Localization
Analysis
Error Analysis
Although the model successfully recognized manipulation type copy- failed precise region localization. duplicated clusters stained cells are indeed present, bul the model's predicted blocks [1, 4] not slign with the actual menipulated patches. This error stems from two key issues: (i) the model over-relies on detacting sharp boundaries and irregular contours, which caused to misinterpret natural variations dense cell regions duplicated blocks; and (i) lacks fine-grained self self-similarit analysis, leading to inaocurate indexing forged
The Lack of Five Core Reasoning Capabilities
Region Localization, Recognition
Figure 12: Failure case by GPT-5 Copy-Move. The model correctly identified the forgery type but mislocalized it.
47

---

Published as a conference paper at ICLR 2026
E.6 COPY-MOVE: STAINED MICROGRAPH (FAILURE CASE 2)
Copy-Move: Stained Micrograph
Question
The task requires the model to act as an image forensics expert, determine the lype of forgery in the given panel, accurately mark the tampered regions when necessary, and striclly follow the specified output
Options
unnatural boundary mismatches,, inconsistent lightinghextures. Note: Splicing NEVER involves duplicated regions.
Splicing different of combined in
Copy-Move- region copied and pasled within the SAME cues: duplicaled patiems/objecls with same noiseltxttrre but shifeedrotateddssaled Reminder: if any part of the image is duplicated, MUST be
classified Covy-Move, not Splicing.
Al-Generated image wholly or partially Alggenenttedmoddlledd cues: synthetic textures, implausible
details, inconsistent semantics.
Forgery - the image appears genuine; no strong evidence manipulation.
E) Not Sure insufficient evidence to decide.
Slicing Rule
The image was divided into 2-row by 2-column equal-area grid, resulting in 4 blocks. The numbering goes left to right, top to bottom: Block = top-lett, Block 2 top-rghtt, Block 3 bottom-laft. Block bottom-right
Issue B) Copy-Move
[1,2,3,4]
(GPT-5) Model Answer: A) Splicing
X
Explanation
Two sharp vertical seams/bars split the scene into separate sections with abrupt changes in texture and object distribution across the boundaries- classic spice seams. duplicated regions suggest copy-move,
details look photographic rather than Al-generated.
Region Localization
[1.2.3.4]
Analysis
Error Analysis
The model misciassiiied the manipulation because fixated on the two artificial black divider lines, incorrecily interpreting them splice seams. These lines dominate visual field and overshadow the forensic cue
large duplicated cellular regions across Blocks 1-4, GPT-5 ignored the regeated morphology pattems and instead relied on superficial "visible boundary splicnng' heuristic, This shows failure to prioritize duplication
avidence when strong but irrelvvant structural edges are present, leading to the wrong forgery type.
The Lack of Five Core Reasoning Capabilitios
Expert Knowledege Utilization, Visual Recognition
Figure 13: Failure case by GPT-5 on Copy-Move. The model correctly localized the forgery region but misidentified it.
48

---

Published as a conference paper at ICLR 2026
E.7 COPY-MOVE: STAINED MICROGRAPH (FAILURE CASE 3)
Copy-Move: Stained Micrograph
Question
The lask requires the model to act as an image forensics expert, determine the lype of forgery in the given panel, accurately mark the tampered regions when necessary, and striclly follow the specified oulput format.
Options
unnatural boundary mismatches,, inconsistent lightinggtxxtures. Note: Splicing NEVER involves duplicated regions.
Splicing different
of content combined
Copy- Move region copied and pasled within the SAME cues: duplicaled patlems/objecls with same noise/texture but shiftedirotatedscalee Reminder. any part of the image is duplicated, MUST be classified Copy-Move, Splicing.
Al-Generated image wholly or partially Al-generatedmmddifed:: cues: synthetic textures, implausible details, inconsistent semantics.
Forgery image appears genuine; no strong evidence manipulation.
E) Not Sure
insufficienl evidence to decide.
Sticing Rule
The image was divided into 2-row by 3-columi equal-area grid, resulting in blocks. The numbering goes left right, top to bottom: Block = first row, left; Block first row, middle; Block 3 = first right. Block row,
row, middle; second row, right.
B) Copy-Move
[1,2,4.6]
?
(GPT-5) Model Answer: A) Splicing X
Model Explanation
Visible pasted-locking potch on the left side with hard, polygonal edges and tonal mismatch; seams cut across appears that palch. These indicate
the small reclangular splicing.
Region Localization
[1.4]
Analysis
Error Analysis
copy- move, and instead fixated single tonal irregularity in Blocks and 4. GPT-5 misinterpreted natural staining splicing cues not
madel overlooked duplicated
variabiiity compression noise
edges" "seams," leaading actually present, By anchoring its judgment on minor contrast artifacts rather than structural repelition, the sssigned the wrong manipulation type.
model not only missed half the manipulated regions
The Lack of Five Core Reasoning Capabilities
Expert Knowleddge Utilization, Visual Recognition. Region on Localization
Figure 14: Failure case by GPT-5 on Copy-Move. The model misidentified and mislocalized.
49

---

Published as a conference paper at ICLR 2026
E.8 AI-GENERATED: MEDICAL IMAGING (SUCCESS CASE)
AI-Generated: Medical Imaging
Question
The fask requires the model to act as an image forensics expert, determine the lype of forgery in the given panel, and strictly follow the specified output format.
Options
combined image; unnatural seams, boundary mismatches, inconsistent lightingtexttures Note: Splicing NEVER involves duplicated regions. duplicated pattern/cojjetts with same nolsenaaxure but shitec/hotateddscaleddd Reminder: if any part of the image is duplicated, it MUST be classified Copy-Move, Splicing.
Splicing
sources
Copy-Move region copied and within SAME
C) Al-Generated image whally or partially N-ggeneratedmmodffeddd cues: synthelic textures, implausible
details, inconsistent semantics.
Forgery - the image appears genuine; no strong evidence manipulation.
E) Not Sure insufficient evidence to decide.
Issue C) Al-Generated
(GPT- Model Answer: C) Al-Genrrtee
Model Explanation
The image has Al-generation cues: implausible anatomy and vessel branching. overly smoothimelted toes, synthatic rather than x-ray detail. No duplicated regions suggesting capy move.
inconsistent glow halos
Figure 15: Success case by GPT- Al-Generated. The panel is entirely forged via Image Inference Forgery.
50

---

Published as a conference paper at ICLR 2026
E.9 AI-GENERATED: MICROGRAPH (FAILURE CASE)
X
AI-Generated: Micrograph
Question
The lask requires the model to act as an image forensics expert, determine the lype of forgery the given panel, and strictly follow the specified output format.
Options
combined image; cues: unnatural searns, boundary miamatches, inconsistent tightingttxxtures. Note: Splicing NEVER involves duplicated regions. duplicated patters/obijetts with same nolsehaxture but shitec/rotateddscalddd. Reminder: if any part of the image is duplicated, it MUST be classified Copy-Move, Splicing,
Splicing
sources
Copy-Move region copied and within SAME image:
C) Al-Generated image wholly or partially Aegenerateddmoddffed cues: synthelic textures, implausible
details, inconsistent semantics.
Forgery - the image sppears genuine; no strong evidence manipulation.
E) Not Sure insufficient evidence to decide.
Issue C) Al-Generated
(GPT-5) Model Answer: No Forgery X
Model Explanation
No duplicated regions artifacts; lighting. textures, and depth-of-filld look consistent with natural macro photo. strong indicators of generation.
Analysis
Error Analysis
model rejected AIGC because the image lacks typical generative artifacts such visible duplicated pallerns, warped structures, inconsistent lighling. GPT-5 over-relied on global pholorealism cues- smoothh depth-of-field, coherent lighting, natural macro like textures and failed detect subtie synthetic traits biological close-ups, which olten resemble real micrography.
The Lack of Five Core Reasoning Capabilities Expert Knowledge Utilization
Figure 16:
Failure case by GPT-5 Al-Generated. The model misidentified the forgery type. The panel is entirely forged via Image Inference Forgery.
51

---

Published as a conference paper at ICLR 2026
E.10 AI-GENERATED: DIAGRAM (SUCCESS CASE)
AI AI-Generated: Diagram
Pryooahiic BA
Pordtins
Repp
udd ci1
Semattd palhway
Question
The lask requires the model to act as an image forensics expert, determine the lype of forgery in the giiven panel, accurately mark the tampered regions when necessary, and strictly follow the specified output format.
Options
unnaturall seams, boundary mismatches, inconsistent lighting/texturrs. Note: Splicing NEVER involves duplicated regions.
Splicing different sources of content combined in
Copy Move region copied and pasled within the SAME cues: duplicalled patiernstobjecis with Reminder: if any part of the image duplicated, MUST
same noise/texture but shiftedrotateddscaaee classified Copy-Move, not Splicing.
Al-Generated image wholly or partially al-generatedmmoddffeee cues: synthetic textures, implausible
details, inconsistent semantics. Forgery the image appears genuine; strong evidence manipulation.
E) Not Sure
insufficienl evidence to decide.
Slicing Rule
equal-area grid, resulting blocks. numbering goes to right, tap to botlom: Block = top-lefl Black 2 = lop-middile Block = lop-right Block 4 bottom-lefi Block 5 bottom-
image
2-row
bottom-right.
C) Al-Generated
[1,2,3]
(GPT-5) Model Answer: Al-enneatee
Model Explanation
The lop banner lext is gibberish with obvious Al-like hallucinated words and malformed lypography, inconsistent with the coherent scientficc labels elsewhere. This nonsensical, synthetic-looking text indicates partial peneraaioniiooiiiicaiion concentraleed
Reglon Localization
[1,2.3]
Figure 17: Success case by GPT-5 AI-Generated. The panel is partially forged via Targeted Region Editing.
52

---

Published as a conference paper at ICLR 2026
E.11 AI-GENERATED: CHART (FAILURE CASE 1)
AI-Generated: Chart
sequercee identity
DNA
Protein
1
sapiens
100%
100%
Whol Dana
80 han
D00 Pera
Crecland
secedbre
80%
95%
gallus
80%
83%
80%
89%
Question
The task requires the model to act as an image forensics expert, determine the type of forgery in the giiven panel, accurately mark the tampered regions when necessary, and strictly follow the specified output format.
Options
unnatural seams, boundary mismetchee, inconsistent lighting/texturre. Note: Splicing NEVER involves duplicated regions. Copy-Move region coplad and pasted within the SAME image; cues: duplicated patters/objects with Reminder: if any part of the image duplicated, MUST
Splicing different sources of content combined in
same noise/texture but shiftedrolated
classified Copy-Move, not Splicing.
Al-Generated image wholly or partially Al-generatedmmoddfied; cues: synthetic textures, implausible
details, inconsistent semanlics.
Forgery image appears genuine; strong insufficiett evidence to decide.
manipulation.
E) Not Sure
Slicing Rule
numbering goes to right, top to batlom: Block 1 first row, left Block 2 first row, middle Block = first row, right Block 4 second row,
image
2-row 3-column equal-area grid, resulting
row, Block second right.
Issue C) Al-Generated
[2,3,5,6]
(GPT-5) Model Answer: A) Al-Generated
Model Explanation
The right-side table shows Al-ike artifacts: gibberish headers ('Wholls Dena," '80 hen." "DOO Peria").
inconsistent symbols (random squares and vartical bars), and odd semantics (red '437" with clear meaning).
These synthelic, repelitive textures and implausible text suggest Al generation localized the rightmost blacks.
Region Localization
[3.8]
Analysis
Error Analysis
The model correctly identifled the manipulation AIGC but missed half the manipulated regions. GPT-5 focused only on most visually abnormal areas- BlocBBlkks
6-where gibberish headers and synthetic
symbols are strongest. contrast, Blocks and 5 contain more subile AIGC artifacts embedded within clean scientific: layout. Their
formatting appear relatively cohereni, reducing salience compared corrppted rightmost columns. As result, the model over-relied on degree of distortion rather than the full spatial extent generation, leading partial localization.
highly
The Lack of Five Core Reasoning Capabilities Region Localization, Spatial Reasoning
Figure 18: Failure case by GPT-5 on Al-Generated. The model correctly identified the forgery type but mislocalized it. The panel is partially forged via Targeted Region Editing.
53

---

Published as a conference paper at ICLR 2026
E.12 AI-GENERATED: PHYSICAL OBJECT (FAILURE CASE)
X
AI-Generated: Physical Object
Question
The task requires the model to act as an image forensics expert, determine the type of forgery in the given panel, accurately mark the tampered regions when necessary, and strictly follow the specified output format.
Options
unnatural seams, boundary mematccess inconsistent lighting/texttres. Note: Splicing NEVER involves duplicated regione. Copy-Move region copied and pasted within the SAME image; cues: duplicated patternslobjacts with Reminder: if any part of the image duplicated, It MUST
Splicing different sources of content combined in
same noise/texture but shittedrrtattedscale
classified Copy-Move, not Splicing.
Al-Generated image wholly or partially al-generatedmmodifeeed cues: synthetic textures, implausible
details, inconsistett semantics.
Forgery image appears genuine; strong E) Not Sure - Insufficient evidence to decide.
manipulation.
Slicing Rule
image
equal-area grid, resulting numbering goes top-left Block 2 tightight Block 3 bottom-left Block boltom-right.
to right, top to battom: Block
Issue C) Al-Generated
[1,2]
(GPT-5) Model Answer: A) Splicing X
Model Explanation
The top band with uniform orange circles/text appears pasted: it has different sharpness/nolse and lighting from the there straight horizontal boundary separating from photographed strips clasw-classic splicing cues in the top half (Blacks and 2).
Region Localization
[1.2]
Analysis
Analysis
The model correctly localized the manipulated region but misclassified its type due AIGC-Splicin confusion. edges sharply
averly uniform colors, with the noisy, naturally lil photographic region below. GPT-5 misinterpreted this domain gap and the straight croes-image compositing. typical splicing cue. reality, artifacts arise from generalive over-regguarizatioon rather than pasted content, leading to correct delection but incorrect attribution.
Al-generated top
textures,
horizontal boundary
The Lack of Five Reasoning Capabilities
Expert Knowledge Utilization
Figure 19; Failure case by GPT-5 on Al-Generated. The model correctly localized the forgery region but misidentified it. The panel is partially forged via Targeted Region Editing.
54

---

Published as a conference paper at ICLR 2026
E.13
AI-GENERATED: CHART (FAILURE CASE 2)
AI-Generated: Chart
College or above
1
2
3
4
5
6
Question
The task requires the model to act as an image forensics expert, determine the type of forgery in the given panel, accurately mark the tampered regions when necessary, and strictly follow the specified output format.
Options
Splicing different sources of content combined in one mismatches, inconsistent lightingtexturrs, Note: Splicing NEVER involves duplicated regions.
unnatural seams, boundary
Copy-Move region coplad and pastec within the SAME image; cues: duplicated patternslobjacts with same naise/texture shiftediooated classified as Copy-Move, not Splicing.
Reminder: if any part image duplicated, MUST
Al-Generaled image wholly or partially Al-generatedmmdiffedd
synthetic implausible
delails, inconsistent semanlics.
Forgery image appears genuine; strong E) Not Sure insufficiett evidence to decid..
manipulation.
Slicing Rule
3-column equal-area grid, resulting to right, top to bottom: Block = first row. laft Block 2 first row. midalee Block 3 first row, right Block middle Block second right.
image
numbering goes
second Block
Issue C) Al-Generated
[1,3,4,5,6]
(GPT-5) Model Answer: D) No Forgery X
Model Explanation
The image is deen chart/plot with consistent elements and na duplicated regions, unnatural seams, or Al-stylc artifacts. No strong evidence of manipulation.
Reglon Localization
Empty
Analysis
Analysis
The model misclassifed the image 'No Forgery' because focused on the overall structural consistency of shading, aligned axes overlooking Al generation. in this case, the filled confidence interval and curve rendering exhibit syrthhtic artifacts such as typical of charts directly exported from standard statistical software. Moreover, the model falled to reconcile the semantic caption-provided grid references and the visual elements, causing ignore region- level cues across Blacks 1, 3, 4, 5, and 6.
smooth curve,
signs
overly uniform gradients, imprecise boundary blending, unnatural
mismatch
The Lack of Five Core Reasoning Capabilities Region Localization, Expert Knowledge Utilization, Spatial Reasoning
Figure 20: Failure case by GPT-5 on AI-Generated. The model misidentified and mislocalized. The panel is partially forged via Targeted Region Editing.
55

---

Published as a conference paper at ICLR 2026
E.14 DUPLICATION: MICROGRAPH (GLOBAL SUCCESS CASE)
Duplication: Micrograph
Question
In this task, the model is required to perform croee-panel duplication forensics, Each input conslsts of multiple sub-figures (panels) that that originate from the same scientific figure. The model must determine whether any panels duplicated, so, identify specific type(s) duplication.
Options
A) Direct near-identical content panels without geometric transform; many keypoints align at similar
scalerppositio.
B) Scaling: content panel appears in another after zoom-in/zoom-out.
C)
panel
version another (e.g. 90"/180").
DY Flipped: one panel is mirrored (harizotaalvertical version of anolher.
image-level adjustments (e.g.. brightness, color). F) No Duplication: there is no convincing duplication across panels.
Parameter-Me Modified: duplication
G) Sure: evidence insufficiett ambiguous.
Issue Flipped
(GPT-5) Model Answer: D) Flipped Duplication
Model Explanation
The left and right panels depict the same cell after left-right mirror flp, Distinctive features-the the irregular "hook"-like protrusian the cell edge, the ovall dark center with the same center darker patch, and several small background speckles (including trio near the lower edge) align when one image is horizontally flipped.
No evident scaling rotation.
Figure 21: Success case by GPT-5 on Duplication. These panels resulted from Global Duplication with flip.
56

---

Published as a conference paper at ICLR 2026
E.15 DUPLICATION: STAINED MICROGRAPH (GLOBAL FAILURE CASE)
Duplication: Stained Micrograph
Question
In this lask, the model is required perform cross panel duplication forensics. Each inpul consists of multiple sub-figures (panels) originate
scientific figure. must whether any panels duplicated, and if so, identify the specific type(s) of duplication.
Options
A) Direct: near-identical content across panels without geometric transform; many keypoints align similar scala/position.
B) Scaling: content one panel appears
zoom-inizoom-on
C) Rotated: one panel rolated version another (e.g.. 90'/180").
Flipped: panel mirrored (horizontallveticcl
another.
E) Parameter-MMddifed duplication exists but with image-level adjustments (e.g., brightness, contrast, and color).
F) No Duplication:
is convincing duplication G) Not Sure: evidence is insufficient or ambiguous.
Issue C) Rotated & E) Parameter-Modified
(GPT-5) Model Answer: Rotated Duplication X
Model Explanation
'The two panels match after 180" rotation: the same green/blue texture pallern, distinclive white verfical bar shifts from left right, the purple band moves from bottom, and two dashed circles with "ES/SS" swap positions exactly as expected under 180 rotation.
Analysis
Error Analysis
While the model correctly identifiedd the rotated duplication, failed to detact the parameter-modifed manipulation. The saturation shift subtle and globally applied, making less visually salient than the
duplicated circular regions. GPT-5 over-focused on geometric transformations and overlooked the uniform color-
intensity distertion across photometric allerations.
revealing bias toward siructural anomalies while missing global
The Lack of Five Core Reasoning Capabilities Spatial Reasoning, Comparative Reasoning
Figure 22: Failure case by GPT-5 on Duplication. These panels resulted from Global Duplication with rotation and parameter modification.
57

---

Published as a conference paper at ICLR 2026
E.16 DUPLICATION: STAINED MICROGRAPH (LOCaL FAILURE CASE 1)
Duplication: Stained Micrograph
Question
this task, model required perform cross-panel duplication forensics. input consists muttiple sub-figures (panels) that oriiintee from the same scientific figure. The model must determine whether panels are duplicated, so, identify specific type(s) duplication.
Options
Direct: near-identica content panels without geometric transform, many keypoints similar scalelposition.
B) Scaling: content in one panel appears in another after zoom-in/zoomoout.
Rotated:
g. 90"/180").
Flipped: one panel is a mimored (horizontallvertica) version of another.
Parameter Modiifid:: duplication exists with image-leve adjustments (e.g.. brightness, contrast, and color).
No Duplication: there is no convincing duplication acroes panels.
Not Sure: evidence is insufficient embiguous.
Issue C) Rotated
X
(GPT-5) Model Answer: A) Direct Duplication
Explanation
The two panels show the same histology field the same scale and orientation. Distinct gland contours and clusters blue puncta align when translated (cropped differently), with no rotation, flip, scale change,
noticeable parameter adjustments.
Analysis
Error Analysis
The model misclassifed rotational duplication as direct reuse. It correctly perceived similar gland contours and clusters of blue puncta, but its visual recognition failed register the orientation change. terms spatial reasoning, assumed
same scale crientation, attributing differences only to cropping. Its comparative reasoning was Imited, relying on a single translation elignment rather than evaluating manipulation
panels
rotation altemative explanation. Finally, did activate expert knowledge patterns, where rotation combined with crapping frequent histological images. These shorkcomings led ta the erroneous conclusion of direct dupiication.
The Lack of Five Core Reasoning Capabilities
Expert Knowledge Utilization, Visual Recognition, Spatial Reasoning, Comparative Reasoning
Figure 23: Failure case by GPT-5 on Duplication. These panels resulted from Local Duplication with rotation. The duplicated region is indicated by the yellow boxes.
58

---

Published as a conference paper at ICLR 2026
E.17 DUPLICATION: STAINED MICROGRAPH (LOCaL- FAILURE CASE 2)
Duplication: Stained Micrograph
Question
required to perform cross-panel dupiication Each input multiple
sub-figures (panels) that originate from the same scientriic figure. The model must determine whether any panels duplicated, and identify the specific type(s) of duplication.
Options
A) Direct: near-ideniical content across panels without geometric transform; many keypoints align al similar scalelposition.
B) Scaling: content in one panel appears in another efter zoom-inzzoom-out
Rotatad:
rotated version of another (e.g. 90"/180").
DJ Flipped: one panel mirrored (horizontalvertical) version another.
Parameter-Moote
duplicalion exists but with image-level adjustments (e.g.. brightness, contrast, and color). there no convincing dupiication across panels.
Duplication:
G) Not Sure: evidence is insufficiett ambiguous.
Issue Parameter-Morde Moddifee
(GPT-5) Model Answer: B) Scaling & Parameter-mocdiied
X
Model Explanation
two panels share same histologic
(matching curved sY"-shaped pale
small circular lumina) al the same relative arrangement, but the right panel appears slightly zoomed in and stronger exposure/contras) magenta shift. indicates duplication with scaling image-lavel adjustments.
Analysis
Analysis
The duplicated marked the yellow boxes occupies different proportions in the panels, appearing
more compact one image more dispersed in other. The model incoreetly attributed change
propartion scaling rather than to differences in background tissue that had been cropped or obscured. geametric transformation nuclear orientations remained consistent, thus mistakenly applying a "scaled" interpretation instead of the simpler of direct duplication exposure adjustment.
Seeking more complex explanation,
the duplicated
The Lack Five Core Reasoning Capabilities
Visual Recognition, Spatial Reasoning, Comparative Reasoning
Figure 24: Failure case by GPT-5 on Duplication. These panels resulted from Local Duplication with parameter modification. The duplicated region is indicated by the yellow boxes.
59

---

Published as a conference paper at ICLR 2026
E.18 TEXT-IMAGE INCONSISTENCY, OTHERS (NUMERICAL SUCCESS CASE)
Immaee Inconsistency: Others
7.45
Si(weh)
6.17
Image Caption
EDS analysis at the grain boundaries Fe-6.5 wi% steel strip samples: 1.0 wi% Cu, (b) 1.5 wi% Cu, and (c) 2.0 wt % Cu.
Image Related Sentence
Further SEM microstructural examination revealed that precipitates absent in the wt% Cu
specimen (Figure 8a), but became visible in the 1.5 wt % Cu specmen (Figure 8b). The precipitates in the 1.5 boundaries. Cu-rich precipitats continuous or semicontinuous at grain boundaries as the Cu dosage was raised to 1.0 wi%, as shown Figure Bc.
Cu specimen tiny, imegularty scattarad
Options
A) Numerical inconsistent the numeric values/asserlions text contradici the ligure's numbersihabelssssaale
B) Trend Inconsistent the frend described text (e g. increase, decrease, peak, or monotanicity ordering) contradicts the figure's visual trends.
Consistent text matches figure.
Not Sure image taxt too ambiguous unreadable to decide.
Issue A) Numerical Inconsistent
(GPT-5) Model Answer: A) Numerical Inconsistency
Model Explanation Observation
1.0 wi% Cu. 1.5 wt% Cu, and 2.0 wi% Cu. The EDS lables show approximate contents of 0.98, 1.73, and 7.42 w5% for (a), (b). and (c), consistent with those sample labels. Checking text The related text states that 1.5 w5% corresponds to Figure 8a and that "raised to 1.0 wr% corrssonnds Figure Comparison with options These are mismatches between and the figurelcaption, ie., numerical inconsistencies, not trend discrepancies. The caption itself malches the A) Numerical Inconsistent, listed showing incorrect figure composition mappings.
The figure and its caption label panels
figure, issue lies
sentences
Sentence Localization
Further SEM microstructural examinstion revealed that Cu rich precipitates were absent in the 1.5 wt %/ Cu
specimen (Figure
visible the 1.5 % Cu specimen (Figure 8b).
Cu-rich precipitates were continuous or semicontiuuous at grain boundaries as the Cu dosage was raised to 1.0 Figure
Figure 25: Success case by GPT-5 Text- Numerical Inconsistency.
60

---

Published as a conference paper at ICLR 2026
E.19
TEXT-IMAGE INCONSISTENCY, OTHERS (NUMERICAL FAILURE CASE 1)
Text- Image Inconsistency: Others
Image Caption
Figure 1. NTCP inhibition by Myrcludex B increases plasma bile acid and GLP-1 levels. (A) Unconjugated, conjugated, and total plasma bile acid levels hours after placebo or Myrcludex (2.5 mg/g) injection male obese WT and Oalpfs/1b KO mice 9-10)...
Image Related Sentence
To evaluste the in vivo impact of bile acids on GLP-1 secretion, we used the NTCP inhibitor... increase in temporary, they completely normalized within 24 hours after injection. Inhibition NTCP by Myrcludex B increased fasting GLP-1 levels 10-fald (Figure 1B), Nexi, we evaluated the Myrcludex B B-mediated GLP-1 increase affected glycemic control...
plasma bile acids
Options
A) Numericaal Inconsistent the numeric values/assertions in text contradict the figure's numbers/lbeessssalesss
Trend Inconsistent the trend described in text (e increase, decrease, peak, or monotonicily ordering) contradicts the figure's visual trends.
C) Consistent text matches the figure.
imageltext ambiguous
decide.
A) Numerical Inconsistent
(GPT-5) Model Answer: A) Numerical Inconsistency
Model Explanation
Figure 1A plats bie acids in uM and does not support "1.6 vs 6.13 mM total bile acids. nor the switch between
"unconjugated" and "total." Figure shows roughly -2x GLP-1, 4-fold
Sentence Localization
In line with previous research, Oatp1a/tb KO mice display slightly elevated (unconjugated) bile acid levels compared with WT control animals (average 1.64 KO 6.13
bile acids) (Figure 1A).
Inhibition of NTCP Myrdudex increased fasting GLP-1 levels 4-fold (Figure 1B).
Analysis
Error Analysis
The model relied trend matching rether than numerical reasoning, focusing only whether the direction consistent without comparing the stated fold-changes to actual magnitudes shown the figure. As result, overlooked the clearly unrealistic "10-fold" discrepancy, and it failed apply basic biological knowledge that such
a large increase GLP-1 is highly uncommon physiological experiment.. Because not estimate approximate values from Figure 1A and 18, it could not identify which numbers were inconsistent with the visual
The Lack of Five Core Reasoning Capabilities Visual Recognition, Comparative Reasoning, Expert Knowleege Utliizttion
Figure 26: Failure case by GPT-5 on Text- Numerical Inconsistency. The model identified but mislocalized.
61

---

Published as a conference paper at ICLR 2026
E.20 TEXT-IMAGE INCONSISTENCY. OTHERS (NUMERICAL FAILURE CASE 2)
Text -Imaee Inconsistency: Others
Image Caption
Fig. 1. Long linear DNA fails transfer by acousfic dispensing. PicoGreenTD dsDNA dye was used to measure the fluorescent signal from long, linear lambde DNA (A) supercolled plesmid DNA (B) range of concentrations
sample was transferred either by manual pipetting acoustically using Echolo, tripiictte. At concentrations above 2.5 ugiml, lambda DNA does not transfer well on the Echoll..
Image Related Sentence
tested a selection plasmid samples in the miniaturised dsDNA quantifiction assay, with samples transferred either manually by acoushic dispensing (Fig. 1C and D)...
Options
numeric values/assertions text contradict the figure's numbersllabllsssalles B) Trend Inconsistent the trend described in text (e increase, decrease, peak, manotonicity ordering) contradicts figure's visual trands.
Numerical Inconsistent
CJ Consistent text matches the figure.
DJ Not Sure imageltext too ambiguous unreadable decide.
Issue A) Numerical Inconsistent
(GPT-5) Model Answer: C) Consistent
X
Model Explanation
The caption and related text correctly describe all panels: lambda DNA shows poor Echo transfer above 2.5 ugimi (A), plaamid DNA transters well acress concentrations (B), manuel transfers yleld expected concentrations samples 1-3 are variable for sample (D), gel indicates gDNA in lanes (E),
(C). transfers
and the standard curve matches (F). No numerical trend contradictions observed.
Sentence Localization
X
None
Analysis
Error Analysis
model lacked numerical threshold reasoning: requirement to examine the data points at and beyond the 2.5 uglml mark on the x-axis. Had it done so, would
interpret textual claim 'above 2.5 uglmi
measurements above this concentration. contradicting the propoeed cutoff. Instead, it relied on coarse trend matching the higher concentration range-and therafore
observed Echoo)
continue increase closely
the methods allgn consistency without verifying whether 2.5 uglml correspanded to any actual inflection or failure point the plot.
overall
The Lack Five Core Reasoning Capabilities Recognition, Comparative Reasoning. Expert Knowledge Utiiizaiion
Figure 27: Failure case by GPT-5 on Text-Image Numerical Inconsistency. The model misiden- tified and mislocalized.
62

---

Published as a conference paper at ICLR 2026
E.21
TEXT-IMAGE INCONSISTENCY, OTHERS (TREND FAILURE CASE
Text-Image Inconsistency: Others
Image Caption
FIGURE 3. Covi-iDPRE evokes Ca increase human Jurkat cells after UV photolysis. A, Co-EIDPRE (200 by cIDPRE (200 μM)
(M))
Fluo- Moaded cells were incubated in E, in Fluo loaded Jurkat cells, thapsigargin (10 μM] pretreatment promoted
weaker Ca increases after photolysis
CosiaIDPRE (120 pMy-induedd Ca increases the absence extracellular Ca Data quantifications of
[Ca""] "] peak induced by drug treatment A,
were expressed mean S.E., n cells, p Co-icIDPRE throughout experiments.
D,
continuously
Image Related Sentence
It has been shown that cADPR targets the RyR, but which specific isaform targets remains to be determined Individual knockdown
RyR2 RyR3 expressee isoforms inhibited cIDPRE- inDPRiinduced Ca
(1-3).
Options
A) Numerical Inconsistent - the numeric valueelasserionn in text contradict the figure's numberslabalsiscales.
B) Trend Inconsistent trend described text (e g.. increase, decrease, peak, manatonicity ordering) contradicts figure's visual trends.
C) Consistent text matches the figure.
Not image/text ambiguous
B) Trend Inconsistent
(GPT-5) Model Answer: B) Trend Inconsistency
Model Explanation
panel D (leff), LC3II bands are increased for XRK3F2 alone and for XRK3F2+Blz versus control, indizating
higher LC3II/LCI
Sentence Localization
Co-i-cIDPRE (200 μM) inciled much weaker Ca2+ increases after UV photolysis than that by cIDPRE (200 μM)
cells.
Analysis
Error Analysis
one point, showing a partial ability judge However, it failed to detect key inconsistency: the effect thapaigergin (10 UM) pretreatment on Ca* panel
successfully
first sentence, preventing its degradation."
p62 levels by inducing expression and
The Lack of Five Core Reasoning Capabilities Recognition, Comparative Reasoning
Figure 28: Failure case by GPT-5 on Text-Image Trend Inconsistency. The model identified but mislocalized.
63

---

Published as a conference paper at ICLR 2026
E.22
TEXT-IMAGE INCONSISTENCY, OTHERS (TREND FAILURE CASE 2)
Text Inconsistency: Others
A
MAO
NBVC
1POC
Image Caption
Figure Levels of inflammatory factors in different groups. (a-5) IgA, IL-4, IL-8, IL-17, TNF-expressio levels different groups. NotesCompared with the control group, "P: 0.05, ""P 0.01: Compared with the model group, Pi 0.01.
Image Related Sentence
PCs Altenuated the Infiammnatory Readioon HSP Serum-Induced CelisHSP serum remarkably enhanced the levels of IgA, IL-4, IL-8, IL-17 and TNF- of HUVECs (P 0.05) (Figure These infiammatory indicators induced by HSP serum, however, exhibited an similar trend the presence of PCs (P 0.05); that was, PCs prominently out down the expression of IgA, IL-4, IL-8, IL-17 and TNF- in the different concentration PCs groups (P 0.01) (Figure nevertheless, huge significant difference was among different concentration PCs groups (P) 0.05).
Options
A) Numerical Inconsistent the numeric values/aserrions in text contradict the figure's numbers/labelsssalles.. Inconsistent the described text (e. increase, decrease, peak, or monotonicity orderin)) contradicts the figure's visual trends.
ligure.
D) Not Sure - image/text too ambiguous or unreadable to decide.
Issue B) Trend Inconsistent
(GPT-5) Model Answer: A) Numerical Inconsistency
X
Model Explanation Observation
figure showe model groups higher controls
groups, levels
model. Significance markers indicate (P 0.01) for most markers when compared with the model, but for IL-17 (panel only single is shown, indicating weaker significance (P 0.05), P 0.01. Checking text: The related text states that PCs prominently reduced all five markers with P 0.01. ComparisonFor IL-17, the figure indicates 0.05, whereas text claims 0.01 for markers. This numeric (p-value) contradiction Option analysis: A Numerical Inconsistent fits due to p-value mismalch for IL-17. B Trend Incorsistent trends jincrease with model, decresse with PCS) match. Consistent not true because p-value discrepancy. Thersfore, Als selected.
Sentence Localization
that PCs prominently concentration PCs groups (P 0.01) (Figure 4)
the expression IL-4, IL-8, IL-17 and TNF- the different
X
Analysis
Error Analysis
The related seniences contain obvicus trend error logical contradiction. However, the model failed detect these issues, and its entire reasoning focused instead on the subite difference in the reported p-value
IL-17 significance another sentence.
The Lack of Five Core Reasoning Capabilities
Visual Recognition, Comparative Reasoning
Figure 29: Failure case by GPT-5 on Text- Trend Inconsistency. The model misidentified and mislocalized.
64

---

Published as a conference paper at ICLR 2026
F DOCUMENT EXTRACTION AND PARSING DETAILS
F.1 JUSTIFICATION OF PIPELINE SELECTION
Although recent document parsing tools (such as MinerU (Wang et al. 2024a)) offer more con- venient interfaces for PDF structural analysis, these tools are not designed for constructing a sci- entific paper fraud forensics benchmark. In particular, they do not satisfy the following three key requirements of THEMIS: (1) figure-level extraction, (2) accurate caption association, and (3) panel-level segmentation.
In contrast, Fitz YOLOv7 (Wang et al. 2023) pipeline is widely used in production vironment, Our YOLOv7 model was trained on a 200K dataset of manually annotated academic panels and has passed commercial deployment-level reliability validation. In evaluation, the Fitz + YOLOv7 pipeline achieved 94.10% precision, 93.94% recall, 94.02% FI score, and 92. average IoU. The examples in Figure 30gure330shov show typical failure cases observed when using MinerU, as well as the correct extraction results produced by our pipeline on the same samples.
MinerU
Ours: Fitz YOLOv7
Extract Figure
Associate Caption
Sogment
Extract Figure
Associate
Cannet Extract Figurs.
Image Caption Pair
"Tgum Alls cowtico
Wrong Caption:
TRAD and liphotas setoot
Wrong Image:
Image:
Figure 30: Left: Extraction and Parsing by MinerU. The green highlighted regions indicate MinerU's panel-level parsing outputs, while the blue regions represent caption parsing results. MinerU fails to extract complete figures and frequently mismatches image caption pairs. Right: Extraction and Parsing by our pipeline. The regions highlighted with red boxes correspond to figures, while those highlighted with blue boxes correspond panels. (Green boxes are used solely to verify the effectiveness of the visualization.) Our extraction pipeline correctly identifies both full figures and their corresponding panels without structural or visual degradation.
Furthermore, across multiple document parsing methods evaluated in production environment, we found that the Fitz YOLOv7 pipeline consistently preserves the visual fidelity of all extracted figures and panels, without introducing noticeable degradation in resolution clarity. In contrast, other document parsing tools (such as MinerU) often perform panel slicing directly on rasterized PDF renderings, leading to visible loss of clarity.
65

---

Published as a conference paper at ICLR 2026
F.2 ERROR ANALYSIS AND MANUAL CORRECTION
Our document extraction and parsing pipeline first calls Fitz to obtain complete figure regions from each PDF page, and then applies our YOLOv7-based detector to parse sub-panels within each figure.
Within the THEMIS construction pipeline, all automatically extracted figures and panels undergo rigorous human verification. Any panel-level extraction errors, such boundary offsets, under- segmentation, or missed sub-panels, are manually corrected before inclusion in the final benchmark. This ensures that every figure and panel used THEMIS is accurate and complete.
These examples illus- trate typical error modes, including overlapping panels, weakened boundary cues, and dense textual interference, and demonstrate how these cases arise.
We further analyze representative parsing failure cases in Figures 33
Parsing Error: Overlapping Panels
Soft-age Coinies (6)
150
MDA231
150
MDA468
150
SUM159
MCF7
missingg_annel
200
200
100
100
100
150
150
50
50
50
Softagar
100
50
50
WP
Tumorr growt (mrml)
P.00001
Control
shPL-1
Tumor weight (g)
9 10
PLSCR1
60
0.2
p-tubulin
50
10
14
18
22
26
30 (d)
F
0001
P- 0.000001
Po0.7157
0001
Pro 0930 P-000000000000
#*114
SH181
e=102
n+96
PLSCR1 miRAA
nr32
m=180 m228
PL PLSCR1 MRNA
ax78
n=101
m=119
PLSCR1 MRNA
GSEZ5066
S 2 S
NK02995
AKC295
GSE7390
80
PLSCR1
0353
P-O 0043
Tumor size (om) less than 2 2
less 2an 2
Error Analysis
The YOLOv7 detector suppresses the inner sub-panel when panels overlap, keeping only the outer box and resulting in missing panel error.
Figure 31: One missing panel highlighted with a yellow box.
66

---

Published as a conference paper at ICLR 2026
Parsing Error: Minor Boundary Deviation
extra panel
missing_panne pannel
LDH release (%6
60-
LDH (%)
10-
(μIM
P-STAT3
STAT3
20-
8
GAPDH
Poputatin A+TFP (6pM)
TFP (8pM)
Pepslatin A+TFP (6μM)
Ctrl TFP (BuM)
H1
Melmet
Pepstatin
Pepstatin
SISTAT3+ siCM/+TTP
Galectin-3
LDH release (%)
68-
LDH release (%) 18
15 to
(6
TPP yiM)
Melmet
TFP (BμM)
sTSTAT3 RISTAT3-1FP (9pM)
Ctrl
Ctrl TFP (BuM) sISTATS
RISTATS-TFP (GuM)
Error Analysis
The panels are correctly detected, though slight boundary offsets cause a mismatch with the ground truth. This reflects a small geometric variation rather than a substantive parsing error.
Figure 32: Two missing panels highlighted with yellow boxes. Two extra panels highlighted with red boxes.
67

---

Published as a conference paper at ICLR 2026
Parsing Error: Partially Missed
missing pane
Error Analysis
this example, dense textual elements and complex layout weaken the visual cues of panel boundaries, causing the YOLOv7 detector to miss several sub-panels during parsing.
Figure 33: Three missing panels highlighted with yellow boxes. One extra panel highlighted with red box.
68

---

Published as a conference paper at ICLR 2026
G USE OF LLMS
In the preparation of this manuscript, LLMs were used to assist with language editing to improve readability and fluency. Additionally, generative models played role in the construction of our benchmark, specifically for synthesizing controlled fraud data. Detailed methodologies regarding their usage in data generation are provided in Appendix We emphasize that all research concepts, experimental protocols, and analyses were human-led and validated. The authors reviewed all AI- assisted content and bear full responsibility for the scientific integrity of the work.
69


Reviews:
[{"review_id": "QVF3dLigcM", "reviewer": "Reviewer_bNnH", "summary": "This paper constructs a benchmark for forgery forensics in academic papers, covering various forms of academic misconduct across multiple research domains. The benchmark includes both simple manipulations such as copy-move and fully AI-generated forgeries, and provides a comprehensive evaluation of existing MLLMs. The results show that current MLLMs perform poorly on this benchmark, with localization tasks performing significantly worse than attribution tasks, highlighting the substantial room for improvement in this area.  \n**As I am not deeply familiar with this specific domain, I would prefer to refer to the opinions of other reviewers when determining the final score.**", "questions": "Please refer to the *Weakness* section for detailed explanations.  \nIf the authors can adequately address my concerns, I would be very willing to raise my score.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The constructed benchmark covers a wide range of academic disciplines and forgery types, and the evaluation includes a relatively comprehensive set of model categories.  \n2. Although there are a few typos, the paper is overall well-written, and the figures and visual presentations are clear and well-designed.", "weaknesses": "1. The authors mention using the **Fitz** library and **YOLOv7** for information extraction and segmentation. However, to my knowledge, there are now more accurate tools in the document extraction domain, such as **dots.ocr** and **MinerU**. Given the diversity of samples, the effectiveness of simply applying Fitz (PyMuPDF) and YOLOv7 is questionable. Since subsequent steps rely heavily on accurate information extraction, this stage could be improved to ensure higher benchmark quality and reliability.  \n2. The appendix should include more **case examples** from different categories within the benchmark to give readers an intuitive understanding of its scope and difficulty level.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d3p6Mxt1XZ", "reviewer": "Reviewer_f9Wq", "summary": "This paper proposes a new benchmark, termed THEMIS, to comprehensively evaluate the capability of current MLLMs in scientific paper fraud forensics. Compared with existing visual fraud reasoning benchmarks, the proposed THEMIS exhibits several advantages, including real-world scenarios & complexity, tasks diversity & granularity, and multi-dimensional capability evaluation. Based on THEMIS, the authors extensively evaluate recent open-source and proprietary MLLMs and conduct in-depth analysis.", "questions": "In lines 1443-1444, \"However, performance drops markedly on synthetic data, with some tasks almost completely failing (e.g., LLaMA approaches zero on both splicing and copy-move)\". This statement may be inconsistent with Table 10.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. THEMIS defines a comprehensive taxonomy for the field of scientific paper fraud forensics.  Specifically, THEMIS covers 7 academic scenarios, 5 tasks, 16 manipulation operations, and 5 core reasoning capabilities, which is more diverse than existing visual fraud reasoning benchmarks.\n2. The data quality of THEMIS is high. The synthetic data is rigorously reviewed by human experts. Moreover, THEMIS contains real samples in addition to synthetic samples, which makes it closer to real-world applications. \n3. The authors provide the details of data curation and question generation, which lays a soild groundwork for future research.\n4. The authors conduct comprehensive evaluation and in-depth analysis based on THEMIS. The MLLMs involved includes 6 proprietary ones and 5 open-source ones. The evaluation results demonstrate the significant challenges posed by this benchmark. Furthermore, the analysis conclusions provides deep insights into the related field.", "weaknesses": "1. The MLLMs evaluated in this paper are not comprehensive enough. It is recommended to supplement the results of InternVL3.5, GLM4.5V, Gemini-2.5-Pro, Claude, etc.\n2. There is a lack of comparsion on difference parameter sizes of the same series of MLLMs (e.g., Qwen2.5-VL-3B/7B/32B/72B).\n3. The conclusion in lines 373-375 is not well explained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BD79wc0DHZ", "reviewer": "Reviewer_W3Ck", "summary": "This paper presents THEMIS, a novel multi-task benchmark for evaluating Multimodal Large Language Models (MLLMs) in scientific paper fraud forensics. It features over 4K questions across 7 real-world academic scenarios (from retracted papers and synthetic data), covers 5 fraud tasks with 16 fine-grained manipulations, and maps tasks to 5 core reasoning capabilities. Experiments on 11 leading MLLMs show even top models (e.g., GPT-5 with 56.29% BRI) fall below passing thresholds, revealing limitations in handling compound manipulations and imbalanced capabilities.", "questions": "* In your opinion, how should the community use your benchmark? Is it to evaluate a model's ability to detect cheating and then select an excellent model to serve as a judge? Or are there more ambitious goals?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "* The starting point is novel and has practical application value.\n* The data construction is complete, clear and reproducible.", "weaknesses": "* Typo: There is an issue with the citation in 4.4 Appendix\n* From the perspective of the benchmark, it is quite well done. However, in the long run, this topic should be more suitable for optimizing, training, and fine-tuning models. If there are fine-tuning results, the value of this benchmark will be even higher.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)