FakeXplain: AI-Generated Image Detection via Human-Aligned Grounded Reasoning
Authors: Yikun Ji
Yan Hong
Qi Fan
jun lan
Huijia Zhu
Weiqiang Wang
Liqing Zhang
Jianfu Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Vision Language Models
Image Forensics
AIGC Detection
Primary area: foundation or frontier models, including LLMs
TL;DR: This work fine-tunes a Vision Language Model based on human-annotated data to classify AI-generated images and pinpoint where and why it considers so.

Abstract: The rapid rise of image generation calls for detection methods that are both interpretable and reliable. Existing approaches, though accurate, act as black boxes and fail to generalize to out-of-distribution data, while multi-modal large language models (MLLMs) provide reasoning ability but often hallucinate.
To address these issues, we construct \textbf{FakeXplained} dataset of AI-generated images annotated with bounding boxes and descriptive captions that highlight synthesis artifacts, forming the basis for human-aligned, visually grounded reasoning. Leveraging \textbf{FakeXplained}, we develop \textbf{FakeXplainer} which fine-tunes MLLMs with a progressive training pipeline, enabling accurate detection, artifact localization, and coherent textual explanations. Extensive experiments show that \textbf{FakeXplainer} not only sets a new state-of-the-art in detection and localization accuracy ($98.2\%$ accuracy, $36.0\%$ IoU), but also demonstrates strong robustness and out-of-distribution generalization, uniquely delivering spatially grounded, human-aligned rationales. The code and dataset are available at: \href{https://github.com/Gennadiyev/FakeXplain}{https://github.com/Gennadiyev/FakeXplain}.

Published as a conference paper at ICLR 2026
FAKEXPLAIN: AI-GENERATED IMAGE DETECTION VIA HUMAN- ALIGNED GROUNDED REASONING
Yikun Ji
Yan Hong Qi Fan' Jun Lan
Huijia Zhu? Weiqiang Wang? Liqing Zhang Jianfu Zhang
Shanghai Jiao Tong University Ant Group {da-kun, lqzhang, c. e.sis]esjtu.
yelam. yelam. 1eilam..jeantgroup.ccm
ABSTRACT
The rapid rise of image generation calls for detection methods that both in- terpretable and reliable. Existing approaches, though accurate, act as black boxes and fail generalize models (MLLMs) provide reasoning ability but often hallucinate. To address these issues, we construct FakeX epliied dataset of AI Al-generated images annotated with bounding boxes and descriptive captions that highlight synthesis artifacts, forming the basis for human-aligned aligned visually grounded reasoning. Leveraging FakeX- plained, we develop FakeXplainer which fine-tunes MILMss with progressive training pipeline, enabling accurate detection, artifact localization, and coherent textual explanations. Extensive experiments show that FakeXplainer not only sets a new state-of-the-arr in detection and localization accuracy (98.2% accu- racy, 36.0% IoU), but also demonstrates strong robustness and auttof-diitribbtiin generalization, uniquely delivering spatially grounded, human- aligned rationales. The code and dataset available at: https///gttaba.oommmmmmmma/////p
-itto--distribution data, while multi-modal large language
INTRODUCTION
The past decade has witnessed rapid progress text-to-image generation, evolving from Genera- tive Adversarial Networks to Diffusion Models, which now capable of producing images nearly indistinguishable from real photographs (Goodfellow et al.) 2014] Peebles & Xie 2023). These ad- vances have led to explosion of highly realistic Al-generated content, raising pressing concerns about misinformation. authenticity, and trust in digital media. Most existing detection methods cast this task as binary classification problem, leveraging convolutional neural networks vision trans- formers (Wang et al. 2020 Ojha et al. 2023 Park & Owens 2024). However, binary labels offer limited insight into why an image classified as Al-generated. real-world applications, especially those involving legal, journalistic,
ethical implications, explainable detection is essential. An ef- fective detection system should not only identify whether an image Al-generated but also pinpoint the specific visual cues or logical inconsistencies that betray its synthetic origin. Such explainability promotes user trust, supports verification workflows, and enables more informed decision decision-making.
The rise of Multi-modal Large Language Models (MLLMs) has enabled cross-modal inference, allowing models to generate human-readable explanations about Al-generated images. Recent ef-
forts (Li al. 2024 Zhang al. 2024 Zhou et al. 2025 Gao et al. 2025 Xu al. 2024 Liu
et al. 2024 et [d.]5202)) have advanced interpretable textual reasoning using MLLMs. However,
these methods either depend heavily on prompt engineering. model-generated explanations, plug- in segmentation modules (Kirillov et al. 2023) to delineate manipulated regions. As illustrated in Figure D existing MLLM-based detectors may hallucinate false claims or provide reasons without spatial grounding, since their explanations are not validated by human annotations. Without proper visual grounding or human-ale auman-aligned supervision, it remains unclear whether the generated rationales truly reflect the image content or derive from model hallucinations. To improve human alignment, fine-grained multimodal supervision, such as region-level annotations and captions, essential. Yet, the lack of such high-quality datasets poses a major challenge building reliable and interpretable MLLM-based detection systems. In this paper, we present FakeXplained, a dataset high-quality "t Corresponding authors.

---

Published as a conference paper at ICLR 2026
Grounding Rearview mirror not connected to vehicle
Tags
Tire broken (left)
Takckplaa Sapaalaainr
Tire too thin (right)
Art Style
Ceneerated
Unrecognizable
Text distorted and identifiable
Handrail misplaced
Wrong Attributes
Texture
Anamolies
L pon examining this image, found the following artifacts:
This golf cart image has fairly consistent sunlight and reasonably natural shadow cast. ... Non-specifie reasons
Generated
bledbooss
Other bosoei
The left rear wheel the golf cart is missing has unnstural appearance.
Traditional Method
Q Real
Hallucination (False claims)
Figure I: Comparison of our method (FakeXplainer) with traditional classification -based detectors (without explanations) and other MILM- MiLeed methods (with hallucinated or non-specific reasons). FakeXplainer is trained to localize flawed regions and explain why the image appears Al Al-generated.
Al-generated images with fine- linegrained, numma--roounded annotations, together with FakeXplainer, an RL fine-tuning pipeline for MLLMs that achieves state-of-the- detection accuracy and ground- ing performance. As shown in Figure [ training FakeXplained enables FakeXplainer provide comprehensive rationales for fake image detection, performing on par with human experts. Our major contributions threefold:
The FakeXplained dataset is a curated dataset 8,772 Al-generated images from diverse state- of- fi-the-art generative models, annotated with bounding boxes and concise captions that highlight visual anomalies and illogical details.
The FakeXplainer detector: By fine-tuning MLLMs on FakeXplained, we build an end-to- end system that not only detects Al-generated images but also explains them. Fine-tuning on FakeXplained enables FakeXplainer to perform fine-graine -r-igriindd visual reasoning and articulate clear, human-aligned observations.
Robust performance with explainability: FakeXplainer answers "where and why does this image look fake?" with reliable, spatially grounded explanations. Extensive experiments show that it achieves state-of-the-arr detection accuracy, generalizes well to out-of-distribution images, and remains robust under perturbations while providing human-aligned, interpretable reasoning.
2 RELATED WORKS
Detection of AI Al-generated and manipulated images. Detecting Al Al-generated images has gained prominence with the improving fidelity of synthetic images from GANs (Goodfellow et al. 2014 Esser et a 202T), autoregressive transformers (Van Den Oord et al.
2017). diffusion-baser mod-
els (Le et al. 2025 Ye et al.) 2025 Wang et al. 2025b; Li et al. 2025a Chadebec et al., 2025 Song ct al. 2020 Ho et al. 2020 and DiTs (Peebles & Xiel
2023 Chen al.
2023). Deep learning meth-
ods such as ResNets and Vision Transtormers trained on real and synthetic data (Wang et al. 2020
Tan & Le 2019 Park & Owens 2024 Chang et al. 2023 Yan et al.)
2024) leverage strong feature extraction to learn discriminative patterns. However, generalization to unseen models remains chal- lenging (Bi et al. 2023 Yan al. 2024). As generation techniques evolve, artifact-based cues alone become increasingly unreliable. A complementary research direction focuses on model explainabil- ity, most detectors offer only binary classification without indicating how or where synthetic cues are found. Recent efforts towards fine-grained or localized detection include using branti systems for multi-level labels (Bi et al., 2023), computing local intrinsic dimensionalities (Lorenz) et al., 2023), or using gradient visualizations (Selvaraju et al. 2017). Despite these advances, existing MLLM-based methods still face limitations providing human-aligned grounded expla- nations and in generalizing across rapidly evolving generative techniques. While several recent datasets leverage MLLMs to generate rationales or global labels (Zhang et al., 2024 Zhou et al.) 2025 Huang et al. 2025b; Wen et al. 2025 Li et al. 2025b), their reliance automatic anno- tations raises concerns about hallucination and weak visual grounding. Although FakeBench (Li al. 2024) incorporates human refinement, its explanations are initially generated by GPT-4V
2

---

Published as a conference paper at ICLR 2026
Text Prompt
this iooge rest fake! Way?
this
realistic image
FLUX
Vaiso Eaceder
Tekonieer Embedding
FakeXplainer
Generated Imaaee
<a3ine
upan abservation think
Manuel selection
High-Quality Images
The stone has overly shadows
Visien Projecter
LeRA
Language Decoder
LoRA LM Heed
</thinko
count incerrect
[29, 365, #28, 685/ Crub leg
too abviaus:
128,
Hermit crabs should one has only
ctapp </ug
ImageNet
Trained Experts
PGRPO
Ourpat Formatting
FakeNplained
Woong Pempactive
Classification Accuracy
Tharefore, this image Licely AZ-generated. </verdict>
(BBoxes, Captions, Tage)
Art Style Unancogrtzadle
J Grounding Accuracy
Wrung Alributts
(a), FakeXplained Construction
(b). FakeXplainer Fine-tuning Inference
Figure 2:
Overview of: (a) Fakexplained: Construction of dataset with numan-aligned bounding boxes and captions, and (b) FakeXplainer: Progressive fine-tuning pipeline of MILMs, which integrates SFT and GRPO to achieve accurate detection, grounding, and interpretable reasoning.
and remain purely textual, without any region-level grounding; moreover, the full annotations are not publicly released. LOKI covers multiple modalities but relies GPT-4o for ane-grained la- bels, which weakens its alignment with humans. Other datasets such as So-Fake-Set (Huang et al.) 2025b) focuses social-media scenarios and similarly lack region-level annotations, limiting their suitability for training grounded, human-aligned detectors. Methodologically, recent detectors such as AIGI-Holmees (Zhou et al., 2025) combine NPR with MLLMs to improve interpretability. whereas ielizaationfocussee approaches like FakeShield (Xu et al. 2024) and LEGION (Kang et al. (025) depend on external segmentation modules (e.g. SAM (Kirillov al., 2023 ) leaving the intrin- sic grounding capabilities of MLLMs underutilized. Meanwhile, methods without additional mod- ules, such as So-Fake (Huang et al. (2025) and FLKeVLM (Wen et al. 2025), still cannot generate spatially localized explanations. The absence of high-quality datasets remains key obstacle to building reliable and interpretable MLLM-basen detection systems. Without proper grounding or numan-alignes supervision. it is unclear whether generated rationales truly reflect image content are derived from hallucinations.
Training & fine-tuning reasoning-capable MILMs. Enhancing the reasoning capabilities of MLLMs is crucial for tasks requiring nuanced understanding (Wu et al. 2025a by Yang et al.] 2025a Fang et al., 2025 Chen et al.) (022c).. Initial strategies involved converting images into formalized textual representations to enable structured, language-driven reasoning (Yagg et al. 2025b). Subse- quent research has focused on instilling deeper cognitive abilities, including self seff-verification, self- correction, developing "slow thinking" capabilities (Wang et al. (202a)). and managing reasoning depth address phenomena like "overthinking" (Xiao et al. (025). Efforts also explore construct- ing high-quality multi-modal Chain-of- Thought (CoT) datasets (Huang et al.) 2025a) to guide rea- soning processes. Reinforcement Learning (RI. has become pivotal these advancements, with many sophisticated reasoning developments relying on RL methodologies. Fine-tuning methods have spurred significant interest in RL-based multi-modal reasoning (Chen et al. 2025). RL, par- ticularly when combined with structured reward functions, e.g. using Intersection over Union (IoU) for tasks involving image grounding (Shen et al. 2025) markedly improves multi-modal align- ment, visual reasoning, and human interpretable decision making, demonstrating RL's capability of advancing model performance in complex vasion-language tasks.
3 THE FAKEXPLAINED DATASET
Our objective is trustworthy and interpretable detection of Al-generated images. This requires detec- tors that generalize to unseen generative models, remain robust to perturbations, and provide human- understandable explanations. Conventional detectors often lack interpretability, while MLLMs, though promising, tend to produce unreliable explanations with frequent hallucinations when used without training (see Table A. To address this, require models that not only detect Al-generated images but also explain their decisions in natural language for reliability. Achieving this demands dataset that supports both visual grounding and textual reasoning. Therefore, introduce the
3

---

Published as a conference paper at ICLR 2026
FakeXplained dataset, illustrated in Figure aa), to train MLLM to produce trustworthy planations. consists of high-qualin synthetic images paired with fine- grained, human annotations that indicate the underlying flaws and artifacts responsible for detection fake.
3.1 AI-GENERATED IMAGES SELECTION
Semantic alignment. To ensure content diversity and semantic alignment. we generated images (a) across 1,000 ImageNet categories and (b) based on MS COCO (Chen et al. 2013) captions (2017 split). For ImageNet-1K I classes, the prompt used is:
"A realistic image of "class_name))
Model variety. We ensure source diversity by using 28 text-to-image generation models across architectures. The models used are listed as follows:
Diffusion- Belfusioo-BBaed Generators: Midjourney Midjourney (2023) Stable Diffusion models Rom-
bach al. (20226); Esser et al. (2024), DDIM Song et al. (2020), DDPM et (2020), DALL-E
OpenAl 2023). GLIDE Nichol et al.
(021), and VQDM Gu et al. 2022).
GAN-Based Generators: GALIP Tao et al. (2023).
StyleGAN
Karras et al. (2018) VQGAN
Esser et al. (202T), and BigGAN Brock et al. (2018).
DiT-Based Generators: PixArt Chen al. (2033 2024b a) and DiT Peebles Xie 2023).
Other Generators: VAR Tian et al. (2024), Infinity Han et al. (2024), MaskGIT C Chang et al.
2022), and LlamaGin Sun al. (2024).
Most generated images are at resolution of 512 x 512. For methods that do not natively support this resolution, the 1024 1024 resolution is used, and the images are downscaled to 512 512 before entering annotation stage.
Quality assurance. All generated images underwent manual quality screening. this stage, low- quality images that depict non- con-identifiable objects, or images where no part is real, are removed from the dataset. After screening, 8,772 Al-generated images were selected for subsequent annotation.
3.2 IMAGE ANNOTATION
To support interpretable reasoning and help models understand what constitutes an Al Al-generated image, we provide detailed annotations for synthetic images. Real images not annotated because they lack synthesis flaws.
Flawed regions and explanations. Precise regional annotations and corresponding textual descrip- tions are essential for visual grounding and interpretability. We recruited 23 trained annotators to label the high-quality Al-generated images selected from the previous stage. Their primary task was to identify and describe all regions within each image that exhibited signs of being fake (detailed guidelines provided in Appendix A. Prior annotation, all participants underwent standardized training focused on identifying visual cues of Al-generated content. The training emphasized the identification of fake regions. which are defined as areas within an image that either violate common sense or exhibit noticeable Al-generated artifacts. Examples of common sense violations include anomalies such as "a flamingo with three legs" or "bird feathers with metallic appearance". Com- mon AIGC artifacts include "repetitive patterns on a blanket" or "blurred or illegible text".
Annotators were also introduced to a structured annotation rubric to ensure consistency and align- ment with the dataset's objectives. Each annotation consists of one or more fake regions, where each region is represented by a tuple (R,; T:), where RG denotes a rectangular bounding box en- capsulating the region, and T; provides textual description of the identified anomaly or artifact. On average, an annotated image in the dataset contains 5.42 such (R, T)) pairs, which serve as the foundation for grounding and reasoning in downstream model training.
Image-level tagging. In addition to region-level annotation, annotators were asked to tag images based on broader perceptual attributes. These attributes include texture quality, overall realism, cor- rectness of attributes, recognizability of objects, and the presence other significant defects not explicitly listed (e.g., : the occurrence of multiple sub-images within single image). These tags C;
4

---

Published as a conference paper at ICLR 2026
mutually independent, allowing annotators to assign multiple tags to each image ap- propriate. This tagging framework allows the dataset to capture holistic image quality assessments, particularly in cases where visually realistic Al-generated images may lack distinct localized flaws.
3.3 QUALITY CONTROL.
To ensure the reliability the annotations, we implemented a quality control protocol involving both manual inspection and algorithmic validation. A subset of annotations was compared against reference set of fake region annotations curated by the research team. Given the inherently subjective nature of visual interpretation, we adopted tolerant validation criterion to accommodate diverse perspectives among annotators. Specifically, a minimum Intersection over Union (IoU) threshold of 20.0% was applied for bounding box overlap, and an accuracy threshold of 3/ was used for image- level tagging. These metrics were assessed on a validation set comprising 5% of the annotated images. The IoU metric is used to assess the spatial agreement between annotated and reference bounding boxes. Let Ry represent the rectangular bounding box annotated by volunteer, and R, represent the corresponding reference bounding box from the reference set. The Intersection over Union (IoU) is computed as:
R O R loU Rc)
Ro U Hr!
where Ra, n R denotes the area of the intersection between R. and Rr, and R., U Ryl denotes the area of their union. The loU value ranges from 0 to I, with higher values indicating stronger alignment. This quality control procedure ensures a baseline level of annotation fidelity while pre- serving the diversity of human interpretations. The resulting dataset, enriched with both region-level annotations (R, 7) and image-level tags C, offers robust foundation for analyzing the semantic in- consistencies and perceptual flaws of Al-generated images.
METHODOLOGY: FAKEXPLAINER
We propose a training methodology named FakeXplainer for MLLMs designed to detect AI- generated imagery, localize relevant artifacts, and articulate the rationale for their predictions. The training and inference pipeline is shown in Figure 2(.). Inspired by DeepSeck-No -Mthh (Shao et al. 2024). the training pipeline begins with Supervised Fine-Tuning (SFT) (Ouyang al. 2022) to provide basic reasoning ability and ensure structured outputs. This initial phase is succeeded by Reinforcement Learning from Human Feedback (RLHF), which is implemented using progressive GRPO (pGRPO), leveraging our proposed FakeXplained dataset.
Before training, each image's annotations are reformatted into dialogue between user and an assistant, using prompt structure designed for localization-aware fine-tuning. Region-level anno- tations (R,, T;) are enclosed within <think> markers, image-level tags C2 within <tag> markers, and the final verdict is wrapped in <verdict> markers.
4.1 COLD START WITH SUPERVISED FINE-TUNING
The cold start phase of FakeXplainer uses SFT to establish a stable foundation before proceeding RL. During this phase, linear layers of the vision encoder. projector, and language model components in the MILM are fine-tuned based on the supervision signals from the data. This initial fine-tuning is crucial for stabilizing the model prior to full-scale reinforcement learning training, preventing instabilities that might arise from pure RL-based updates (Guo al. 2025).
The SFT process focuses on teaching the model to produce coherent reasoning patterns with a clear structure. The training emphasizes the consistent use of the designated marker format with <think>, <tag>, and <verdict> fields, ensuring format clarity in the model's reasoning out- puts. This structured Chain-of-Thougie (CoT) format reduces errors and improves explainability, providing a solid foundation for subsequent GRPO stages that will refine the model's performance specific metrics.
5

---

Published as a conference paper at ICLR 2026
4.2 DESIGN OF REWARD FUNCTIONS
Reward design is a critical component of RLHE, guiding the MILMS to learn not only how to detect fake images, but also how to localize relevant regions and provide coherent reasoning. We define three core reward functions for this purpose.
Classification accuracy (Label). To ensure the model produces the correct verdict, we extract the classification decision from within the verdict ct marker and compare it with the ground-trune -tuth label. Let denote the textual output of the MLLM, we have:
[I,
if (()
Rc(o)
0
o.w.
where (o) is a regex match for the verdict, and y the ground-truth label of whether the image is real or generated.
Grounding accuracy (loU). To reward alignment between model-predictes and human-annotatedd regions, use a relaxed version of the Intersection over Union (IoU):
Ro(o) = loU** = min (1, 1 (oR(o),,,
where R(o) is the region extraction function that parses textual output to bounding boxes, Ry the annotated region, and 17 relaxing constant. The relaxation based on the observation that human annotators have slight discrepancies regarding the borders of annotated regions. This relaxation reward ensures full credit to the model when the regions annotated by the model are in good correlation with human-annotated ones.
Output format validity (Format). To ensure the model understands the structural requirements of the task, we introduce format reward that encourages outputs conforming to the expected syntax. valid output must include correctly structured <think nk>, <tag>> and <ve Everdicn ct> markers, as well as bounding boxes and captions that are syntactically well-formed and can be parsed using regular expressions. Formally, the reward is defined
[1. 0,
if (V, R, T, (o) parsable
Ro(
o.w.
where T(o) and C(o) extract the regional captions and image-level tags from respectively.
4.3 RLHF WITH GROUP RELATIVE POLICY OPTIMIZATION
Following SFT, we employ Group Relative Policy Optimization (GRPO) Shao et al. (2024) to pro- gressively align the MILM with our objectives of interpretable and reliable take image detection. GRPO combines structured supervision from the dataset with targeted reward signals through carefully designed training process. The reward function is formulated as:
R wa + WcRe + WrRr,
where weights wc wp 1.0 remain constant throughout training, while wG(() increase linearly from 0.5 to 1.0 over the training process.
Our approach employs continuous linear interpolation for the localization weight:
(t 0.5 + 0.5 (t/T),
where represents the current training step and T denotes the total number of training steps.
The linear weighting strategy addresses challenges observed in preliminary experiments and offers three benefits. First, without IoU weight adjustment, models trained with equal weights from the start tend to over-optimize localization rewards, producing many small fragmented bounding boxes that achieve high IoU scores but fail to capture meaningful regions. Down- wown-wwighting the localiza- tion reward in early training prevents this issue. Second, the progressive scheme enables natural curriculum learning. With reduced localization weight at the beginning, the model first learns out- put formatting and classification accuracy. As these skills stabilize, the gradually increasing IoU reward improves localization on top of this foundation. Third, continuous weight adjustment avoids reward spikes and stabilizes training, allowing smooth adaptation of optimization objectives. Exper- iments show that linear reward weighting outperforms static schemes, confirming the effectiveness of gradual reward shaping for training MILMSs in complex visual understanding tasks.

---

Published as a conference paper at ICLR 2026
100
176
08.0%
Hlse Madel Prefrence Rate
Art: (AA
Haie 3orld
Deet
M-7
Metrie (56)
600
Iell
Am:
35.07
40-
GRPO Stage
SP
23.4
11.3
372
$2.0
23.0
6G.6
Ron
SFT
GRPO
Training Stages
Bort
Ogpazina Nices
Figure 4: Accuracy and loU curve of the FakeX- plainer during the training process.
Figure 3: Human preference matrix.
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
We adopt Qwen2.5 5- (Qwen Team 2025) the base model of our FakeXplainer for its Both the SFT and GRPO stages last for three epochs, strong pre-trained grounding capabilities.
with a batch size of 1. We use 1) = 1.1 and the number of samples to G 4 during the GRPO stage. We use 8x NVIDIA A100 GPUs in the SFT stage; 16x NVIDIA A100 GPUs in the GRPO stage. All baseline methods are trained on one NVIDIA A100 GPU. More experimental details are provided in the Appendix B For baseline comparisons, we use the same training data as the Fak- explainer setup. We train SegFormer (Xie al. 202T) and ObjectFormer (Wang et al.) 2022) under segmentation + classification setting the FakeXplained dataset by converting bounding boxes to binary masks. For classification-oni. methods, including NPR (Ojha et al.) 2023). DMD (Corvi et al. 2023), ComFor (Park & Owens 2024), AfPr (Chang al. 2023), and DIRE (Wang al. 2023 , only image-level labels are used during training. We additionally evaluated state-of-the-iee art ML-L-bbased detection methods, specifically FakeShield (Xu et al. 2024), LEGION (Kang al.) 2025), and VLM (Wen et al., (025). For FakeShield, we utilized the pre-trained weights pro- vided by the authors without further fine-tuning. For LEGION, we adhered to the training protocol specified in the original paper, training the model on the SynthScars dataset with identical hyperpa- rameters and experimental configurations as reported.
labels
5.2 OVERALL PERFORMANCE
To ensure robustness and mitigate dataset bias, all models are evaluated using four-fold cross- validation. During training, the detection model is exposed 75% of the images from the Fak- eXplained dataset along with an equal number of real samples. Evaluation is conducted on the remaining 25% of synthetic images, again paired with the same number of real images. We port both classification accuracy and localization performance using the IoU metric on Al-generated images. Robustness tests against perturbations are provided in Appendix C
Comparing to other methods. Quantitative results are reported in Table D comparing FakeX- plainer with traditional detectors. For MILMs, Table presents the post-finetuning performance of LEGION (Kang et al.) 2025) and FakeShield (Xu et al. 2024) across different pre-trained models. Our bert-penn model achieves an overall classification accuracy 98.2%. demonstrating strong robustness and consistent performance across different image generators. For localization, segmentation- based baselines. This indicates that FakeXplainer identifies fake regions more consistently with human annotations than competing approaches. Reasoning Quality. Table 2 shows the BLÉU-2 and ROUGE-L (Li et al.) 2024) of model responses against the FakeXplained dataset. The results verify the training effec- tiveness of FakeXplainer, considerably outperforming the base model in explanation generation, indicating that both the regions and their reasons are generated accurately.
the model achieves an loU score 36.0%. outperforming
Generalizability of FakeXplainer on other MLLMs. To assess generalizability beyond Qwen-2.5- VI., further evaluated it on several bate-of-the-art MILMS with diverse architectures and capa-
7

---

Published as a conference paper at ICLR 2026
Table 1: Experimental result for current Al-generated image detectors and our FakeXplainer across different image generation methods.
FakeXplainer
ObjectFormer
SegUormer
NPR
DMD.
Comf'or
AIPs.
AEROB.
DIRE:
LL
Generators
IoU
Acc.
Acc.
Acc.
Acc.
Ave.
Acc.
Acc.
D
2
2022
0.986
0.360
0.957
0.251
0.942
0.285
0.907
0.934
0.877
0.892
0.823
0.916
0.908
DALLE
0.365
0.949
0.258
0,950
0.912
0.942
0.907
0.915
DDIM
NZU
0.974
0.345
0.954
0.285
0.945
0.280
0.917
0.928
0.879
0.915
0.839
0.912
0.902
DDPM 20.0
0,979
0.350
0.951
0.293
0.947
0,288
0.903
0.931
0.876
0.898
0.836
0.917
0.906
1-03
2024
0.988
0.362
0.958
0.299
0,940
0.295
0.922
0.937
0.874
0.779
0.843
0.919
0.938
FLUX 1-schtel
0.972
0.343
0.953
0.287
0.943
0,283
0.926
0.929
0.882
0.805
0.827
0.913
0.941
GLIDE
2021
0.970
0.340
0,950
0.289
0.946
0,286
0.913
0.935
0.873
0.661
0,922
Midjoumes
2023
0.990
0.364
0.956
0.296
0.949
0.294
0.908
0.939
0.869
0.878
0.814
0.925
0.926
Midjoumey
VS
0.992
0,366
0.959
0.273
0.941
0.297
0.902
0.943
0.871
0.851
0.718
0.927
0.932
SD 1.4 2022a
0.968
0.338
0.952
0.286
0,944
0.282
0.921
0.970
0.880
0.852
0.951
0.909
0.918
SD
0.975
0.347
0.955
0.294
0.948
0,290
0.916
0.949
0.875
0.866
0,966
0.915
0.921
SD 2.1
20226
0.980
0,352
0,951
0.291
0,942
0,287
0.911
0.938
0.872
0.881
0.833
0.918
0.913
SD 3.5 ZU24
2024
0
0.365
0.954
0.294
0.945
0,293
0.914
0.944
0.870
0.934
0
0.924
0.929
SD Large
0.368
0,950
0.906
0.947
0.837
0.935
VQDM 2022
0.973
0,342
0.953
0.288
0.943
0.284
0.927
0.932
0.877
0.938
0.932
0.914
0.909
Diffuston
0.983
0.356
0.954
0.287
0.945
0,290
0.913
0.941
0.874
0.864
0.842
0.920
0.919
BigGAN 2018
0.965
0.335
0.950
0.280
0.941
0.278
0.918
0.892
0.903
0.933
0.861
0.887
0.894
GALIP 2045
0.882
0.353
0,941
0.279
0.941
0,289
0.882
0.882
0.941
0.353
0.706
0.882
0.876
VQGAN 2027
0.967
0.337
0.954
0.282
0.943
0.907
0.889
0.908
0.921
0.932
0.885
0.858
StyleGAN XANGAAN--EE20 2018
0.960
0.330
0,951
0.278
0.940
0,275
0.914
0.884
0.980
0.928
0.939
0.879
0.680
GAN
0.337
0.950
0.280
0,941
0.912
0.890
0.866
0.827
PixAstAlphal 2023
0.987
0.357
0.956
0.295
0.947
0.291
0.908
0.912
0.891
0.934
0.927
0,903
0.861
Pexar
0,984
0.354
0,953
0.292
0.943
0,289
0.921
0.909
0.893
0.939
0.922
0.899
0.896
PixArtSigia 20744
0.989
0.360
0.957
0.296
0.949
0.293
0.919
0.915
0.889
0.924
0.931
0.905
0.904
Di1 2023
0
0.349
0.952
0.290
0.942
0,287
0.913
0.907
0.896
0.928
0.938
0.897
0.892
0.983
0.954
0.293
0.945
0.289
0.914
0.910
0.893
0.931
0.931
0.900
0.889
VAR 2024
0.976
0.346
0.954
0.287
0.945
0,283
0.928
0.893
0.901
0.934
0.927
0.889
0.886
Infinity
0.344
0.289
0.941
0.914
0.897
0.899
0.872
MaskGiT 2022
0.972
0.342
0,955
0.288
0.948
0,284
0.909
0.895
0.904
0.923
0.933
0.886
0.854
LlamaGion 2024
0,980
0
0.953
0.429
0.944
0,289
0.923
0.899
0.897
0.929
0.938
0.892
0.867
Others
0.978
0.348
0.953
0.369
0.944
0.287
0.920
0.898
0.898
0.931
0.933
0.889
0.870
Real Images 2009
0.985
0.956
0.946
0.918
0.903
0.882
0.934
0.854
0.896
0.763
Overall
0.982
0.360
0.954
0.299
0.945
0.289
0.914
0.928
0.882
0.887
0.873
0.911
0.828
Table 2: Performance comparison of FakeXplainer across different base MILMS and against other MLL.M- methods. Post-finetuning results are underlined.
Method
FakeXplainer
FakeShield
LEGION
Backbone
VL3-8
Intem\VL3-114
Ovis2.5-9B
MiMo-VL-7B-RI
Qwen-2.5-VL
Acc.
0.584
0.928
0.568
0.951
0.624
0.909
0.515
0.920
0.734
0.982
0.801
0.583
IoU.
0.039
0.134
0.043
0.289
0.044
0.360
0.028
0.098
BLEU-2
0.061
0,232
0.098
0.235
0.058
0.203
0.083
0.249
0.080
0.267
0.004
0.072
ROUGE-L L.
0.059
0.225
0.092
0.219
0.050
0.184
0.076
0.239
0.076
0.251
0.003
0.055
The consistent performance gain of FakeXplainer across architectures, whether 2025) or without (Ovis2.5 (Lu et al. 2024). MiMo (Xiaomi 2025), ) validates the model-agnostic nature of our pipeline.
bilities in Table 2
with grounding capabilities (InternVI.3 (Zhu et al.
Out-of-distribution (OoD) evaluation. We also evaluated the models on five OoD datasets. Fake-
Clue (Wen et al.; 2025). Chameleon (Yan et al. 2024), FaceForensics++ (Rössler et al., 2019).
images generated by GPT-Image-1 (OpenAl 2025 Rapidata 2025) and AMFRR-Datase (eval) pro- posed by FakeReasoning (Gao al. 2025). As shown in Table 3 our model consistently outper-
forms all other methods across OoD datasets, demonstrating considerable generalization to unseen image domains.
Table 3: Accuracy external datasets for out-of- generalization testing.
Sources
EXP.
ObjForner.
SogFormer
NPR
DMD.
ComFor.
AfPr.
AEROB.
DIRE
FakeShicld
LEGION
FakeClud 2025
0.852
0.462
0.485
0.833
0.734
0.766
0.849
0.239
0.727
0.550
0.172
Chamelenn p1z2
0.843
0.485
0.508
0.794
0.721
0.757
0.803
0.291
0.752
0.587
0.197
GPT PPTT-maae-Te
0,513
0.538
0.790
0.735
0.636
0.597
0.458
0.793
0.752
0.238
FaceForensees
2019
0.864
0.598
0.716
0.861
0.562
0.429
0.746
0.681
0.850
0.773
0.395
MMFR- Dataset
0.874
0.653
0.657
0.569
0.619
0.595
0.786
0.685
0.624
0.710
0.193
8

---

Published as a conference paper at ICLR 2026
Original Image
FakeXplained
FakeXplainer
LEGION
FakeShield
(no mask)
Fake The shoclaces blend into each other.
Fake 1. The laces don't cross over euch other, but blend into the shoe material.
Fake .o.To elaborate, have found that: Left shoe lnce: The left shoe lace missing section.
Real Natural reflection and shudows, no blurring jaggedness,
(AI-Generated)
the text on the shoe, Running ATT,
Fake 1. Blurry nametag, 2. The buttons are missing, and unnatural.
Fake Text and Insignia looks blurry. The pocket long and asymmetrical.
Fake The man's right car is missing. Men's left The rim and earlobe of the are together.
Fake There might be unnutural blending or unclear edges where fabric textures overlap around pocket.
(AI-Generated)
Pocket too large.
Figure 5: Comparison responses visualized from the FakeXplainer method, the ground truth from the FakeXplained dataset, and LEGION (Kang et al. 2025 and FakeShield (Xu et al. 2024)
Table 4: Performance comparison of FakeXplainer under different training configurations.
FakeXplainer
No-FT
Training Stratogy (32B)
Partial Data (32B)
Metric
3B
7B
32B
32B
SFT
GRPO
GRPO
no-bbox
no-caption
no-tags
label-only
Acc.
0,842
0.958
0.982
0.734
0.893
0.937
0.974
0.952
0.942
0.962
0.937
0.255
0.265
0.323
0.265
0.358
BLEU-2
0.195
0.246
0.267
0.080
0.183
0.257
0.261
0.164
0.243
0.076
0.160
Qualitative evaluation. Figure shows two samples from the FakeXplained test set. We found that our model prefers outlining smaller regions than human annotators, demonstrating fine- graingd localization capability. Compared with LEGION and FakeShield, FakeXplainer not only provides correct predictions but also delivers reliable, grounded explanations. Our user study also shows that responses from FakeXplainer preferred 99.75% of the time when compared to LEGION and FakeShield. More qualitative examples will be provided in the Appendix A.2
Human preference evaluation. While IoU and classification accuracy provide objective metrics for detection performance, they do not fully capture the qualitative aspects of region-caption alignment. In fact, the model may, some instances, generate annotations that surpass those of the original human annotators. To comprehensively assess the quality and relevance of the generated explana- tions, we conducted human preference study involving an independent group evaluators. this study, participants were shown pairs of outputs for the same image, each with different bounding box annotations and associated captions. With no metadata given, evaluators were asked to choose the annotation that demonstrated better alignment between the region and caption, as well as higher overall quality. no clear preference emerged, neutral option was available.
We received 1,525 non-neutral preference votes. In Figure the "Humans" category represents annotations from the FakeXplained dataset. When compared with FakeXplainer, human annota- tions were preferred in 52.9% of cases, indicating that FakeXplainer achieves near-parity with hu- man annotators in producing region-grouse -roinddd explanations, demonstrating the effectiveness of our framework in generating high-quality visual-textual reasoning.

---

Published as a conference paper at ICLR 2026
5.3 ABLATION STUDIES
We ablate each training component, including each data component and each training segment, and report the results in Table 4
Additional ablation results are provided in Appendix D
Model size of MILM. Model size has clear impact on the performance of FakeXplainer. With the same training pipeline, the 7B variants can identify the authenticity of the image in 95.8% of the cases, but the 3B variant fails to surpass most traditional methods in detection accuracy and cannot effectively localize fake regions. The 7B variant also outputs a good rationale according to the BLEU and ROUGE-L metrics, making it good balance between performance and speed.
Effects of different training stages. To analyze the impact of each training stage, we report both accuracy and loU metries throughout the training process Figure 4 Without GRPO, SFT alone yields marginal improvements over the base model, especially in localization. The GRPO stage with a constant Wg = has a higher IoU than the linear scheme but struggles to train effectively in later steps, demonstrating the cumulative benefit of the progressive reward design. By the completion of the RLHF stage, the model reaches an accuracy of 98.2% and an IoU of 36.0%.
2.5n-2555 2.5-vt-32B-Instruee achieves only 73.4% accu- racy. SFT improves this to 89.3%, and adding GRPO further increases the performance to 98.2%, demonstrating the critical role of our two-stage training pipeline and the FakeXplained dataset.
Fine-tuning impact. Without fine-tuning,
Data components. We evaluate three data components: image tags, region annotations (bounding boxes + captions), and binary labels. Using only binary labels yields 93.7% accuracy the lowest among partial variants but still exceeding DMD's s 92.8%. Removing bounding boxes or captions reduces accuracy by 3.5%, with caption removal severely impacting IoU (-9.5%). While tag removal has minimal effect on both metrics. These results confirm that structured reasoning information, particularly region-leve level annotations, substantially improves detection performance.
Reward weighting mechanism. Fixed reward weighting (GRPOwg=1 and GRPOwc=0.s) under- performs our progressive GRPO approach across all metries. Notably, the iisalizatton-priortiizee GRPOwg=1 also shows inferior loU, validating the necessity of textual explanations and dynamic reward weighting for the step-by-step acquisition of classification, localization skills, and overall interpretability.
6 CONCLUSION
In this work, we present FakeXplainer, an explainable Al-generated image detection approach uti- lizing MLLMs that provides grounded, numan-interpretablle explanations alongside the detection results. The system achieves strong performance metrics (98.2% accuracy, 36.0% IoU) through progressive training pipeline, establishing foundation for transparent visual media authentica- tion. Although grounded explainability improves FakeXplainer's generalization, its dependence on human-perceptible artifacts reflects an inherent limitation shared by all current explainable AIGI detectors. Fully realistic synthetic images without semantically describable flaws represent a fun- damentally different detection problem and fall outside the scope of this work. At the same time, our results show that grounded, human-aligned reasoning provides clear advantages over black-box classifiers: models that output only binary label exhibit substantial degradation under distribu- tion shift, whereas FakeXplainer maintains strong performance across multiple out-of-distribution benchmarks. These findings suggest that explicit localization and human-aligned supervision offer more robust and verifiable signal than classification alone.
7 ACKNOWLEDGMENTS
This work was supported in part by the National Natural Science Foundation of China (Grant Nos. 62302295, 62595733, and 62561 160155). the Shanghai Municipal Science and Technology Major Project (Grant No. 202ISHZDZZX0122) This work was also supported by Ant Group.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Xiuli Bi, Bo Liu, Fan Yang, Bin Xiao, Weisheng Li, Gao Huang, and Pamela C. Cosman. Detecting generated images by real images only, 2023.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In Proceedings of the International Conference on Learning Representations, 2018.
Clement Chadehec, Onur Tasar, Eyal Benaroche, and Benjamin Aubin. Flash diffusion: Accelerat- ing any conditional diffusion model for few steps image generation. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025.
Huiwen Chang, Han Zhang. Lu Jiang, Ce Liu, and William Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11315 11325, 2022.
You-Ming Chang, Chen Yeh, Wei-Chen Chiu, and Ning Yu. Antifakeprompt: Prompt-tuned vision- language models are fake image detectors. arXiv preprint arXiv:2310.114411 2023.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-a: Fast training of diffusion transformer for photore- alistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023.
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao. Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Weak-to to-sonng training of diffusion transformer for 4k text-to-image generation. In Proceedings of the European Conference on Computer Vision,
pp. 74-91. Springer, 2024a.
Junsong Chen, Yue Wu, Simian Luo, Enze Xie, Sayak Paul, Ping Luo, Hang Zhao, and Zhenguo Li. Pixart-6: Fast and controllable image generation with latent consistency models. arXiv preprint arXiv:2401.05252, 2024b.
Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. RI-v: Reinforcing super generalization ability in vision-language models with less than $3, 2025.
Xinlei Chen, Hao Fang, Tsung-Yi Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C. Lawrence Zitiikk Microsoft coco captions: Data collection and evaluation server. ArXiv, abs/1504.003255, 2015.
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024c.
Riccardo Corvi, Davide Cozzolino, Giada Zingarini, Giovanni Poggi, Koki Nagano, and Luisa Ver- doliva. On the detection of synthetic images generated by diffusion models. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1-5, 2023. doi: 10. D..109//CASS449335..00 2023. 10095167.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, K. Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 248-255, 2009.
Patrick Esser, Robin Rombach, and Bjom Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 12873- -1883, 2021.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion En- glish, Kyle Lacey. Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis, 2024.

---

Published as a conference paper at ICLR 2026
Wenlong Fang, Qiaofeng Wu, Jing Chen, and Yun Xue. guided milm reasoning: Enhancing milm with knowledge and visual notes for visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter Recognition, 2025.
Yueying Gao, Dongliang Chang, Bingyao Yu, Haotian Qin, Lei Chen, Kongming Liang, and Zhanyu Ma. Fakereasoning: Towards generalizable forgery detection and reasoning. arXiv preprint arXiv:2503.21211 2025.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Y. Bengio. Generative adversarial networks. Advances in Neural Informa- tion Processing Systems, 3, 06 2014. doi: 10.1 10.1145/342262..
Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to extttto-mmage synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10696-10706, 2022.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprini arXiv:2501. 12948, 2025.
Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. arXiv preprint 2xXiv:2412.044311 2024.
Jonathan Ho. Ajay Jain, and P. Abbeel. Denoising diffusion probabilistic models. ArXiv, abs/2006. 1239, 2020.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025a.
Zhenglin Huang, Tianxiao Li, Xiangtai Li, Haiquan Wen, Yiwei He, Jiangning Zhang, Hao Fei, Xi Yang. Xiaowei Huang, Bei Peng. et al. So-fake: Benchmarking and explaining social media image forgery detection. arXiv preprint arXiv:2505. 18660, 2025b.
Yikun Ji, Yan Hong, Jiahui Zhan, Haoxing Chen, jun lan, Huijia Zhu, Weiqiang Wang, Liqing Zhang, and Jianfu Zhang. Towards explainable fake image detection with multi-modal large language models, 2025.
Hengrui Kang, Siwei Wen, Zichen Wen, Junyan Ye, Weijia Li, Peilin Feng, Baichuan Zhou, Bin Wang, Dahua Lin, Linfeng Zhang, et al. Legion: Learning ground and explain for synthetic image detection. arXiv preprint 2xXxv:2503.5200 2025.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1396-4403 2018
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et Segment anything. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision, pp. 4015-4026, 2023.
Black Forest Labs. Flux, 2024.
Duong H Le, Tuan Pham, Sangho Lee, Christopher Clark, Aniruddha Kembhavi, Stephan Mandt, Ranjay Krishna, and Jiasen Lu. One difffusion to generate them all. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2025.
Jia Li, Lijie Hu, Jingfeng Zhang, Tianhang Zheng, Hua Zhang, and Di Wang. Fair text-to-image diffusion via fair mapping. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025a.
Yixuan Li, Xuelin Liu, Xiaoyang Wang. Shiqi Wang, and Weisi Lin. Fakebench: Uncover the achilles' heels of fake images with large multimodal models. ArXiv, abs/2404. 13306. 2024.
12

---

Published as a conference paper at ICLR 2026
Yixuan Li, Yu Tian, Yipo Huang. Wei Lu, Shiqi Wang, Weisi Lin, and Anderson Rocha. Fakescope: Large multimodal expert model for transparent ai-generated image forensics. arXiv preprint arXiv:2503.24267. 2025b. 2025b.
Jiawei Liu, Fanrui Zhang, Jiaying Zhu, Esther Sun, Qiang Zhang, and Zheng-Jun Zha. Forgerygpt: Multimodal large language model for explainable image forgery detection and localization. arXiv preprint arXiv:2410. 10238, 2024.
Peter Lorenz, Ricard Durall, and Janis Keuper. Detecting images generated by deep diffusion models using their local intrinsic dimensionality. Proceedings the IEEE/CVE International Conference on Computer Vision Workshops, pp. 448 459, 2023.
Shiyin Lu, Yang Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Han-Jia Ye. Ovis: Structural embedding alignment for multimodal large language model. arXiv:2405.20797, 2024.
Midjoumey. Midjourney, 2023.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint
2021.
Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards universal fake image detectors that generalize across generative models. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 24480-24489, 2023.
OpenAI. Dall-e 3 system card, 2023.
OpenAl. Introducing 4o image generation, Mar 2025.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744, 2022.
Jeongsoo Park and Andrew Owens. Community forensics: Using thousands of generators to train fake image detectors, 2024.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4105-4205, 2023.
Qwen Team. Qwen2.5-v1, Jamuary 2025.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022.
Rapidata. Rapidata openai 4o preference, Mar 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 10684-10695, June 2022a.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 10684-10695, June 2022b.
Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. FaceForensics++:: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2019.
Ramprasaath Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based lo- calization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 618-626, 2017. doi: 10. 19//CC/22.1.7.7.
13

---

Published as a conference paper at ICLR 2026
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Jun-Mei Song, Mingehuan Zhang, Y. K. Li, Yu Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. ArXiv, abs/2402.03300, 2024.
Haozhan Shen, Peng Liu, Jingcheng Li. Chunxin Fang, Yibo Ma. Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang. Ruochen Xu, and Tiancheng Zhao. Vlm-rl: stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07611 2025.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. ArXiv, abs/2010.02502, 2020.
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024.
Mingxing Tan and Quoc V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. ArXiv, abs/1905. 1994, 2019.
Ming Tao, Bing-Kun Bao, Hao Tang, and Changsheng Xu. Galip: Generative adversarial clips for text-to-image synthesis. In Proceedings the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 14214-14223, 2023.
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng. and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in Neural Information Processing Systems, 11884839-88866 2024.
Aaron Van Den Oord, Oriol Vinyals, al. Neural discrete representation learning. Advances in Neural Information Processing Systems, 30, 2017.
Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu. Fangzhen Lin, and Wenhu Chen. V1- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504. 25000:255000...7877, 2025a.
Junke Wang. Zuxuan Wu, Jingjing Chen, Xintong Han, Abhinav Shrivastava, Ser-Nam Lim, and Yu-Gang Jiang. Objectformer for image manipulation detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
Sheng- Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot... for now. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 8695-8704, 2020.
Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. Dire for diffusion diffusion-generatte generated image detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 22445-22455, 2023.
Zhendong Wang, Jianmin Bao, Shuyang Gu. Dong Chen, Wengang Zhou, and Houqiang Li. Design- diffusion: High-quality text-to-design image generation with diffusion models. In Proceedings of the IEEE/VVF Conference on Computer Vision and Pattern Recognition, 2025b.
Siwei Wen, Junyan Ye, Peilin Feng. Hengrui Kang, Zichen Wen, Yize Chen, Jiang Wu, Wenjun Wu, Conghui He, and Weijia Li. Spot the fake: Large multimodal model-based synthetic image detection with artifact explanation. arXiv preprint 2xXv:2503..4900 2025.
Mengyang Wu, Yuzhi Zhao, Jialun Cao, Mingjie Xu, Zhongming Jiang, Xuehui Wang, Qinbin Li, Guangneng Hu, Shengchao Qin, and thi-Wing Fu, lem-assistant: instuccionntuning multimodal large language models for rule-based explainable image content moderation. In Proceedings of the AAA/ Conference on Artificial Intelligence, 2025a.
Shengqiong Wu, Hao Fei, Liangming Pan, William Yang Wang, Shuicheng Yan, and Tat-Seng Chua. Combating multimodal Um ballucination via bottom-up holistic reasoning. In Proceedings of the AAAI Conference Artificial Intelligence, 2025b.
14

---

Published as a conference paper at ICLR 2026
Wenyi Xiao, Leilei Gan, Weilong Dai, Wanggui He, Ziwei Huang, Haoyuan Li, Fangxun Shu, Zhelun Yu, Peng Zhang, Hao Jiang, et al. Fast-slow thinking for large vision-language model reasoning, arXiv preprint arXiv:2504.18458, 2025.
LLM-Core-Team Xiaomi. Mimo-vl technical report, 2025.
Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Seg- former: Simple and efficient design for semantic segmentation with transformers. In Advances in Neural Information Processing Systems, 2021.
Zhipei Xu, Xuanyu Zhang, Runyi Li, Zecheng Tang, Qing Huang, and Jian Zhang. Fakeshield: Explainable image forgery detection and localization via multi-modal large language models. arXiv preprint ar urXiv:2410.0222 2024.
Shilin Yan, Ouxiang Li, Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Weidi Xie. A sanity check for ai ai-generated image detection. arXiv preprint arXiv:2406. 194355, 2024.
Fan Yang, Ru Zhen. Jianing Wang, Yanhao Zhang. Haoxiang Chen, Haonan Lu, Sicheng Zhao, and Guiguang Ding. Heie: Milm-based hierarchical explainable aige image implausibility evaluator. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2025a.
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et R1-onevision: Advancing generalized multimodal rea- soning through mooss-modal formalization. arXiv preprint arXiv:2503, 10615, 2025b.
Zilyu Ye, Zhiyang Chen, Tiancheng Li, Zemin Huang. Weijian Luo, and Guo-Jun Qi. Schedule on the fly: Diffusion time prediction for faster and better image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
Zicheng Zhang. Haoning Wu, Chunyi Li, Yingjie Zhou, Wei Sun, Xiongkuo Min, Zijian Chen, Xiaohong Liu, Weisi Lin, and Guangtao Zhai. A-bench: Are Imms masters at evaluating ai- generated images? arXiv preprint 2000002400033070,, 2024.
Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, et al. Swift: scalable lightweight infrastructure for fine-tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 29733-29735, 2025.
Ziyin Zhou, Yunpeng Luo, Yuanchen Wu, Ke Sun, Jiayi Ji, Ke Yan, Shouhong Ding, Xi- aoshuai Sun, Yunsheng Wu, and Rongrong Ji. Aigi-holmes: Towards explainable and gen- eralizable ai-generated image detection via multimodal large language models. arXiv preprint arXiv:2507.02666 2025.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint a0X4v.2504.10479, 2025.
A APPENDIX FOR FAKEXPLAINED
The FakeXplained dataset contains 8.772 high-quality Al-generated images annotated with fine- grained bounding boxes and descriptive captions that highlight synthesis artifacts and logical incon- sistencies. This dataset addresses the critical gap in explainable and spatially grounded Al -e-gererte image detection.
A.1 ANNOTATION PROCESS
We recruited a team of experts who are trained to identify fake regions accurately. All them have prior experience in photography, have seen Al-generated images before, possess fundamental photographic literacy understanding, and are familiar with related concepts such as "saturation," "shadow." "perspective." and "noise." During the training process, provide all annotators with detailed instruction handout with examples. The handout contains positive and negative examples for each global tag, detailed bounding box annotation guideline, along with quality control metrics.
15

---

Published as a conference paper at ICLR 2026
Instructions on fake regions. The rule for annotating fake regions is, if through observation of the selected regions of interest, humans should be able to clearly determine that the image is not an authentic photograph. Fake regions primarily show objects that do not follow the natural physics laws or contradict common sense. Common image generation artifacts are also encouraged to be annotated. After selecting local area in the image, it is necessary to describe the reason for identi- fying it as a generated image. The descriptive sentence must start with a noun, followed by one or several adjective phrases short clauses, and must exclusively describe content that appears in the region.
Definition of tags. We refer to the most prominent depicted object in the non-background portion of the image as the image subject. There are exactly six different tags that annotators can attach to an image. Their definitions are listed as follows:
Perspective errors: Indicates that the image has unnatural viewing angle, emrors in perspective, vanishing points. Incorrect occlusion and shadow errors do not constitute perspective errors, but be considered fake regions instead.
Artistic styles: If the overall image presents any artistic style, including but not limited to oil painting, ink painting, or manga style, then select the "Artistic Style" tag. If only certain part of the image contains content an artistic style, this tag should not be selected.
Unknown objects: Indicates that the subject of the image does not exist in the world, is obviously unreasonable. There may be unusual insects and furniture with strong design elements. Judgment should be based on intuition; unfamiliar or rare subjects do not necessarily indicate unreasonable non-existent objects.
Structure/attribute errors: Indicates that the subject of the image has a structure that is inconsistent with common knowledge, or has attributes inconsistent with common knowl- edge. Examples include green flower petals, pink elephants, bent iron spoon handles, hu- mans with more than two legs, and asymmetrical shapes. For erroneous attributes that only occupy a small portion of the image subject, such as an incorrect number of fingers on a human hand, fake regions should be marked as well.
If obvious texture errors appear in the image, this tag needs be selected. For example, the texture of the entire image is blurry, or a portion of an object has a repet- itive, tilted, distorted texture. Unreadable text does not qualify should be labeled as fake region instead. If "Artistic Style" has already been marked, this tag is usually omitted.
Texture errors:
texture error and
Other anomalies: there are very obvious global errors in the image that do not belong to any of the above categories, check this item. This tag can also be marked even if other tags have already been chosen.
Keywords Annotations We analyze the captions of the bounding boxes to find the most fre- quent phrases. Their occurrences shown in Figure Since filter for the highest-quality im- ages, it hard to find a deciding bounding box for some cases. The contours and depth of field are more likely give the image away, leading high frequency of related captions. FakeXplainer manages to align with most of the FakeXplained traits, with higher detection rate for abnormal object textures.
Quality Check ensure quality, we cross-referenced the annotators' proposals against control samples marked ourselves. We enforced rejection policy where average IoU under 20% or a tag agreement lower than will result in the exclusion of all previous data provided by the annotator.
Figure 7 qualitatively demonstrates the 20% IoU criteria of the screening process. The criteria selected to accommodate individual understandings over Al-generated images while preventing obvious fake regions from being omitted without notice. When the annotation process finishes, the annotations and QC samples reached loU of 42.35% and the tag-agreement rate was 79.67%.
16

---

Published as a conference paper at ICLR 2026
Comparison of Visual Artifact Attribution
blurred contours
lig hting or shadow issues
color inconsistencies:
FakeXplained FakeXplainer
unknown objects
text in intennistencies
physics incousistencies
unrealistic textures
4%
6%
8%
Percentage (%)
Figure 6: Keyword analysis for FakeXplained and the FakeXplainer responses.
QC Sample
IoU 38.1% accepted
QC Sample
IoU 25.7% accepted
QC Sample
IoU 18.4% rejected
QC Sample
IoU 17.9% rejected
Figure 7: Samples from the quality check process. When regional annotation has a lower than 20% IoU with the annotation proposal, we reject the annotation.
A.2 MORE SAMPLES
Figure 9 presents more annotated Al-generated images from FakeXplained. The left column dis- plays the human annotations FakeXplained. The right column shows the inference results of our best model, FakeXplainer. The center bar indicates the proportion of human preference votes from our user study. Note that since the "neutral" option was allowed, although the third annotated image received 46.2% of the votes, the human annotator is still rated higher than our model response. Our model demonstrates the ability to generate clearer, more descriptive captions for fake regions and reliably identifies content that contradicts common sense. For instance, in the lock-and- lock-and-keyhole ex- ample (row 5), the model successfully detects that the key is not inserted into the correct keyhole. In the volcano example (row 2). in addition to identifying the "broken mountain body" in the human annotation, the model also detects a subtle issue: the disconnection of the lava flow, highlighting its fine-grained visual reasoning capabilities.
17

---

Published as a conference paper at ICLR 2026
R;, 0)
from FakeXplained
cref-objects, <bbux- tags will be reploced with actuol data from FakeXplamed during training.
Template
Template
Fromge
System
AIGC detection expert. image, inspect curefully and pay attention to defects within the image. Provide reasons after thinking.
need determine this image is Alageaertted or real. Please pay attention the naturalness and details of objects in the ecene and give rearons after thinking.
provide
<image Please inspect this image. Focus on local details, find logical ereves or Al Algerrrrtioon artifacts. Point out regions with provide short caption for each of any, pleuse unswer
<image Is this image Al generated or real? Please think curefully und focus on local details. I there ure any abvious the corzespurding reasens, please point their location and Otherwise,
Prompe
ireational cccurences these regions. If obvious errors local parts, After inspection, this image may be real."
final please answer "No obvious errors local parts. After insgection, this image may be real."
Remponee
<think>
ozed Shbuet
Seefobbect><bbbux
cved chjeett-choppt
<ved object-bboocs
Associated <happstrocturrepppppee crzonssitags.
Overall, this has <iaghstructurs properryy onrorssitag>. <ithink>
<ithink>
<vendiot> After cureful inspection, believe this image Al-
Sverdiut> After cureful inspection, believe this image is generated. <iverdact>
generated. <iverdict>
Figure 8: An example showing two different chat templates branched from one annotation entry.
A.3 ETHICAL CONSIDERATIONS
All generated images are synthetic with no real individuals. Annotators provided informed consent to this annotation job, allowing us to use the annotated dataset for training. We explicitly ask the annotators not to leave any personal or sensitive information in annotations.
A.4 KNOWN LIMITATIONS
Language. Currently, all annotations are in one language. It is hard to translate the short annota- tion sentences to other languages without a manual check for language inconsistencies.
No Real Images. FakeXplained does not contain real images for the time being, as defining gions for real images can be more subjective than Al-generated images.
B ADDITIONAL TRAINING DETAILS
B. TWO-STAGE TRAINING
We use ms-swift Zhao et al. 2025) for fine-tuning Qwen-2.5-V6 VL models.
In the LoRA SFT stage, noticed that freezing either the projector the vision encoder leads to marginal improvement over the base model without training. To achieve optimal SFT performance, both modules must be fine-tuned jointly.
After the SFT stage, we use GRPO instead of PPO. As noted in Shao et al. (2024), GRPO obviates the need for additional value function approximation as in PPO, and instead uses the average reward of multiple sampled outputs. For each query g. GRPO samples G outputs {01, ......,......................... oa) from the old policy model Th... and uses the relative advantage to optimize the MLLM, making particu- larly well-suited for
reasoning tasks where absolute reward calibration is challenging.
We set the initial learning rate to 10 for the SFT stage and 10 15 for the RLHF stage. Reward signals fluctuated at the beginning of GRPO but quickly converged as the model is generating more human-aligned explanations, confirming the effectiveness of our reward design and training strategy.
B.2 COMPUTATIONAL RESOURCES
The full training procedure took 41.0 hours 8x NVIDIA A 100 (80G) GPUs.
18

---

Published as a conference paper at ICLR 2026
Table 5: Comparative performance analysis under compression artifacts, spatial transformations, and resolution changes.
Degradation & Metric
FakeXplainer
OhjFormer:
SegFonser
FakeShield
LEGION
NPR
DMD
ComFor
ATPr.
AFROB.
DIRE
JPEC Compression
Acc.
0.979
0.940
0.927
0,782
0.544
0,820
0.508
0.840
0.871
0.842
0.884
Quality)
loU
0.353
0.284
0.211
0,022
Compresaion
Acc.
0.781
0.897
0.784
0.814
(30% Quality)
loU
0.339
0.267
0.198
0.078
0.059
Random Cropping
Acc.
0.962
0.943
0.934
0.756
0,519
0,903
0.915
0.829
0.879
0.858
0.891
loU
0.314
0.217
0.176
0.067
0.061
Downsampling
Acc.
0.853
(0.5%)
loU
0.362
0.259
0.254
0.092
0.076
Original
Acc.
0.982
0.954
0,945
0.901
0,583
0,914
0.928
0.882
0.887
0.873
0.911
Images
ToU
0,360
0.299
0,289
0.028
0,098
Table 6:
performance evaluation across different datasets when trained with various configurations mentioned the paper.
Sources
FakeXplainer
No-FT
Portial
Training Strieeg
no-bbox
no-caption
label-only
SFT
GRPO
GRPO
GPT-Image-1
Rapidata 2025
0.801
0.421
0.691
0.760
0.774
0.603
0.591
0.788
0.768
FaceForemsics
0.715
0.826
MMER Datase Call ST
0.874
0.593
0.859
0,708
0.843
0.612
0.671
0.794
0.773
At inference time, the end-to-end pipeline that takes an image as input to generate the verdict and grounding (if the image is deemed Al-generated) takes an average of 7.8 seconds on 2x NVIDIA A100 (80G) GPUs.
C ROBUSTNESS AGAINST IMAGE PERTURBATIONS
To evaluate the practical applicability of our approach, we conduct comprehensive robustness eval- uation under common image degradations that are frequently encountered in real-world scenarios. Table [5] presents a comparative performance analysis across three perturbation categories: JPEG compression, random cropping, and downsampling.
Our method demonstrates exceptional resilience to JPEG compression artifacts, achieving low per- formance degradations of merely 0.3% and 0.8% from the uncompressed baseline, significantly outperforming current state-of-the-art methods. All of which experience at least a 3% degrada- tion. Notably, SegFormer and ObjectFormer show more stability than image-only classification models, indicating that grounding enhances robustness, although they still fall short of our method. For downsampling,
scaled the input images to 50% of their original width and height. In ran- dom cropping and downsampling experiments, our approach achieves the accuracy of 96.2% and 98.0%, respectively, indicating robust performance across different resolution scales. Meanwhile, downsampling does not severely affect the loU score, which suggests that our grounded reasoning approach effectively captures semantic-level artifacts that remain detectable even reduced resolu- tions, unlike methods that may rely on pixel-level features more susceptible to resolution changes. Since random cropping modifies the overall image layout, this action can remove certain fake re- gions from an image entirely, leading to lower loU across all methods. Interestingly, we observe a slight increase in loU after downsampling. We hypothesize that this is because our grounding model focuses on the dominant artifact region, which remains visible at lower resolutions, while noisy fine details are suppressed, leading to more precise and focused localization. Overall, the consistent performance across perturbation types demonstrates that model captures underlying semantic artifacts in al-generated content, enabling robust detection even under challenging image conditions.
19

---

Published as a conference paper at ICLR 2026
FakeXplained
FakeXplainer
by human annotators
by fine-tured Queen-25- FL-VBB-Instrret
The upper beak distorted, too long and looks blurry.
The animal has distorted beak
2. The fur has wrong texture and material.
2. The animal has
Legs twisted and deformed, paws partially blurred with bifurcations, overall texture resembling bambuo.
distorted legs
GLIDE
Sample 80.0%
00000
GLIDE
Mountain body partially missing.
Lava flow lacks connection, broken
everywhere, lava flow textire incorrect, layering effect tou pronounced.
1. Volcanic crater shape abnomna..
Lava flow lacks connection.
flow texture incorrect, layering effect too pronounced.
PictrtSigma
Sample 71.4%
PixtrtSigma
1. Text blurry.
2. Screen lines broken and
Text blurry.
disconnected.
The text on buttons are not identifiable.
3. Knob twisted and deformed.
Text blurry.
4. Knob twisted and deformed,
Text blurry.
5. Text blurry.
5, Three knobs of different sizes.
FLUX.I-Schnell
46.2% Sample 3
FLUX.I-Schmelf
1. Socks different in size and color, two socks asymmetrical, socks merged with floor.
2. Sock has different colors both ends.
1. The patch on the socks merge with the wooden floor.
3. Wrinkles in toe area of sock.
4. Crack appearing in the floor.
Sample 100.0%
1. Font blurry.
2. Object twisted and deformed..
1. The keyhole does not match the key insertion position. 2. Key is deformed.
3. Object twisted and deformed,
4. Inner circle width inconsistent between top and bottom.
S. Shape asymmetrical.
BigGAN
Sample 5 85.0%
BigGAN
Has Perspective 2) Artistic sty Unkaown objects: Structare/attrihute errors: Texture Other anomalies.
Tags:
Bocs not apply.
Figure 9: More annotation examples from FakeXplained and model response visualized from Fak- eXplainer. The ratio the center shows the human preference score.
20

---

Published as a conference paper at ICLR 2026
Table Performance analysis of FakeXplainer-32B across different im- age tags. The table shows the percentage of samples containing each tag, and the classification accuracy for images with (Acc on X) and without (Acc non-X) that
9896
9956
68.4
$6.0
97.5
-8.5
56.7
9556
34.1
specific tag.
Aec on X
62%
Asc on mon-X
Tag (X)
With
Without
Samples
Tattee
Anstle
OSa Aromutes
Unnoopiibble
Pengestive
Structue/AAttibuuc Error
98.39%
97.43%
80.52%
Wrong' Texture
93.5318
97.95%
42.65%
Artistic Style
100.00%
97.95%
12.18%
Other Anomalies
94.08%%
98.45%
5.78%
Figure 10: Visual representation detection accuracy
Unrcoognizablc Objects
96.72%
98.25%
3.12%
across different tag categories.
Perspective Errors
100.00%
98.18%
1.16%
D ADDITIONAL ABLATION STUDIES
D.1 OUT-OF- DIT-OF--DISTRIBUTION PERFORMANCE
We evaluate the generalization capabilities of the ablation models in Section 5.2 (Table 2) of the main paper on three OoD datasets: images generated by GPT--mage-1 Rapidata (2025), Forensssssss++ Rössler al. (2019) and MMFR-Datase (eval) proposed by FakeReasoning Gao al. (2025). Ta- ble 6 shows that our complete pipeline achieves accuracies of 80.1, 86.4 and 87.4 respectively, compared 42.1, 51.9 and 59.3 for the base model without fine-tuning.
Among partial data ablations, the label-only configuration performs the worst among all partial data category entries, yielding near no-finetuning performance. This OoD evaluation further confirms that both spatial grounding and textual reasoning are essential for generalization.
The SFT stage alone yields moderate performance (59.1 on GPT-Image-I, 68.0 on FF++, 67.1 on MMFR). Further into the GRPO training, we see a better overall performance. This result is con- sistent with findings discussed our main paper, as the RLHF stages give more performance boost than the SFT stage. The consistent improvements across both datasets suggest our approach learns generalizable features for Al-generated content detection rather than dataset-specific patterns.
D.2 DISABLING LORA
We employ LoRA during training to reduce computational cost and memory usage. While full- parameter fine-tuning is technically possible, our results show that it does not improve accuracy or loU (Accuracy: 98.2% + 97.9%, loU: 36.0% - 35.4%), likely due the limited amount of annotated data. This suggests that LoRA provides a more efficient and suitable training strategy under current data constraints. With significantly more training data, full fine-tuning may yield better results.
E DISCUSSIONS ON TAG-WISE PERFORMANCE
To assess the impact of different artifact types on detection performance, we analyzed the accuracy of FakeXplainer-322 32B on the FakeXplained dataset across the six distinct image-level tags defined in Section Table 7 details the prevalence of each tag within the dataset and compares the model's accuracy on images containing specific tag versus those without it.
Notably, the model achieves 100% accuracy on images tagged with Artistic Style and Perspective Errors. While Perspective Errors are rare (1.16%), the perfect detection rate implies that vanish- ing point inconsistencies are distinct features that the MLLM can easily leverage for classification. Meanwhile, the model exhibits slight performance drop images labeled with the Other Anoma- lies tag. which is typically assigned to images with global inconsistencies or subtle defects that are difficult to categorize into specific localized regions. This decrease in performance suggests that localized flaws are powerful cues to draw the conclusion that an image is Al-generated. This can potentially be improved with global explanation images in the FakeXplained dataset. Similarly,
21

---

Published as a conference paper at ICLR 2026
Unrecognizable Objects (96.72%) presents moderate challenge, likely because defining "recog- nizability" can be subjective and occasionally overlaps with abstract artistic intent.
F LIMITATIONS
Despite promising results, our approach still has limitations. The Qwen-2.5- 2wwn-2255VLL32BB (LL32B-Innstrus model incurs substantial computational costs, which may limit its deployment in resource- resource-constrained en- vironments. Our evaluation does not sufficiently cover domain-specific or real-world image types, such medical, industrial, artistic imagery. As generated media becomes tic and cross-modal, FakeXplainer, designed to detect visible, human-interpretable artifacts that can be localized and explained, may not able produce explainable responses. This scope is inher- ent to all explainable AIGI detectors: grounded, numann-aligned reasoning is only feasible when the underlying cues are perceptible to humans. However, as long as visible artifacts are still present, FakeX Pakiner can effectively detect and explain. Nonetheless, as long as perceptible cues remain, FakeXplainer provides significantly more robust and verifiable predictions than black-box classi- fiers, especially under distribution shift, due to its grounded and human-aligned reasoning process.
and more realis-
G BROADER IMPACT
While our system improves interpretability in detecting Al-generated content, it may also intro- duce risks. The detailed explanations of detection rationale could inadvertently assist malicious adversaries in developing more sophisticated evasion techniques, potentially contributing to ad- versarial "arms race." The deployment such systems without careful consideration could lead to over-censorship of legitimate content, particularly affecting artists and creators who use Al tools ethically. To mitigate these risks, we recommend responsible deployment frameworks, ongoing monitoring for bias and fairness, and collaborative development with stakeholders ensure the technology serves the public interest while preserving legitimate creative expression.
H
THE USE OF LARGE LANGUAGE MODELS
During manuscript preparation, we employed LLMs only for language polishing and grammar re- finement. All research ideas, methods, and results were conceived, implemented, and validated entirely by the authors. Since our work studies MLLMs in the context of forgery detection, we necessarily employed LLMs as research subjects. Specifically, MLLMs were used generate or assist in generating annotations within our dataset and to serve as baseline models in our experi- ments. These usages are intrinsic to the research problem itself and should not be interpreted as LLMss contributing to the ideation or authorship of this paper.
22


Reviews:
[{"review_id": "5yttHtCB7N", "reviewer": "Reviewer_gGWh", "summary": "The paper introduces FakeXplained, a dataset of 8,772 AI-generated images annotated with region-level bounding boxes and short captions (≈5.42 per image), as well as image-level tags, and FakeXplainer, a progressive SFT→RLHF (pGRPO) fine-tuning pipeline for MLLMs that detects, localizes, and explains synthesis artifacts. The method optimizes three rewards—classification, grounding, and format. Experiments demonstrate that the method yields strong results: overall 98.2% accuracy and 36.0% IoU on their splits, with ablations showing large gains over SFT-only and fixed-weight RL.", "questions": "None.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clear problem framing + resources. A well-specified dataset, an expert-guided annotation protocol (23 annotators), and concrete artifacts improve supervision for grounding and reasoning.\n- Strong empirical results. High in-distribution accuracy/IoU, consistent ablation results (e.g., captions/boxes matter), and human-preference-parity trends support the claim of explainable detection.\n- Method integrates structures and learning. The <think>/<tag>/<verdict> output format aligns neatly with reward design, reducing formatting errors and encouraging grounded explanations.", "weaknesses": "- Computational demands. Best results rely on large MLLMs (e.g., Qwen-2.5-VL-32B) with noted deployment costs.\n- Lenient quality-control thresholds may admit noisy labels. The QC accepts region matches at just 20% IoU and image-level tag accuracy of 1/3, which risks training/evaluating on imprecise boxes and weak tag agreements—particularly harmful for grounding-heavy RL.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uUdz0eWKK5", "reviewer": "Reviewer_bATs", "summary": "This paper addresses the critical challenges of interpretability and generalization in AI-generated image detection. The authors argue that existing methods are often \"black boxes\" that fail to generalize to out-of-distribution (OOD) data, while multi-modal large language models (MLLMs) tend to \"hallucinate\" explanations lacking reliable visual grounding. To overcome this, the paper introduces two key contributions: 1) The FakeXplained dataset, a novel collection of AI-generated images meticulously annotated by humans with bounding boxes and descriptive captions that pinpoint specific synthesis artifacts, providing a foundation for human-aligned, grounded reasoning. 2) The FakeXplainer method, a detector developed by fine-tuning an MLLM on this dataset using a progressive training pipeline (SFT + GRPO). Experiments demonstrate that FakeXplainer achieves state-of-the-art performance in both detection accuracy (98.2%) and artifact localization IoU (36.0%), while also showing strong OOD generalization and robustness. Its primary advantage is the ability to generate spatially grounded, human-aligned textual explanations for its verdicts, effectively mitigating the hallucination issues common in MLLMs.", "questions": "None", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The method provides trustworthy, spatially grounded textual explanations for AI detection, mitigating MLLM hallucination.\n\nThe paper introduces the FakeXplained dataset, featuring meticulous human annotations (bounding boxes and captions) essential for training reliable, explainable models.", "weaknesses": "The model shows strong performance on the current dataset. To further validate its generalization and mitigate potential overfitting risks, it would be beneficial to test its zero-shot capabilities on an external dataset, such as Chamelon[1].\n\nThe analysis could be further strengthened by discussing and comparing against other relevant explainable MLLM-based detectors, for instance, FakeVLM[2], So-Fake[3], and FakeScope[4].\n\nTo provide a more comprehensive picture of the method's capabilities, it would be valuable to extend the evaluation to include additional testable benchmarks, such as Fakebench[5] and LOKI[6].\n\n[1] A Sanity Check for AI-generated Image Detection. \n\n[2] Spot the fake: Large multimodal model-based synthetic image detection with artifact explanation. \n\n[3] So-Fake: Benchmarking and Explaining Social Media Image Forgery Detection\n\n[4] Fakescope: Large multimodal expert model for transparent ai-generated image\n\n[5] FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models\n\n[6] Loki: A comprehensive synthetic data detection benchmark using large multimodal models", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cXtdhrRu4z", "reviewer": "Reviewer_iUyJ", "summary": "This paper introduces FakeXplained, a new dataset for explainable AI-generated image (AIGI) detection, featuring human-annotated bounding boxes and captions for synthesis artifacts. The authors use this to train FakeXplainer, a model based on an SFT+RLHF (GRPO) pipeline, which achieves high detection accuracy (98.2%) and provides human-aligned explanations.", "questions": "1. Since the SFT+GRPO pipeline is adapted from existing work, what is the specific, non-trivial novelty introduced in the model structure or training details that makes this method unique for AIGI detection?\n2. The authors should explicitly discuss the limitation mentioned in Weakness #2 in their conclusion. Clarifying that the scope of this work is \"detecting visible, explainable artifacts\" rather than all AIGI, would strengthen the paper's positioning.\n3. To improve reproducibility, the authors should report the specific values for all critical hyperparameters, such as the batch size, reward relaxing constant $\\eta$ and the GRPO sampling number $G$.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Valuable Dataset: The FakeXplained dataset is a strong contribution, enabling the development and benchmarking of explainable AIGI detectors.\n2. Strong Empirical Results: The model achieves SOTA accuracy (98.2%). The human preference study, showing near-parity with human annotators, is a compelling validation of the \"human-alignment\" claim.", "weaknesses": "1. Limited Methodological Novelty: The paper's methodological contribution is somewhat limited. The core SFT+GRPO training pipeline is a successful application and adaptation of existing frameworks (e.g., from DeepSeek-Math) rather than a new algorithm.\n2. Inherent Limitation of the Task Formulation: The paper's core premise, \"Human-Aligned Grounded Reasoning,\" focuses by definition on human-perceptible flaws. This is an inherent challenge for all current work in explainable AIGI detection, as it does not address the separate problem of detecting SOTA images that may be \"fake\" but lack obvious, semantically explainable artifacts.\n3. Potential for Overfitting: There is a notable mismatch between the modest dataset size (8,772 images) and the large models being fine-tuned (32B parameters). This poses a risk of overfitting.\n4. High Computational Cost: The SOTA performance relies on a 32B parameter model, which, as the authors note, incurs substantial computational costs. The reported inference time of 7.8 seconds per image (on 2x A100 GPUs) and 41 hours of training (on 8x A100 GPUs) limits the method's practical applicability in real-time or resource-constrained environments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I2unaMq82M", "reviewer": "Reviewer_N3fb", "summary": "This paper presents FakeXplain, an artifact detection, localization, and explanation method. To train the FakeXplainer, they first curate a synthetic dataset with region and textual annotations for artifacts (FakeXplained). Finetuning MLLMs on the FakeXplained dataset improves the detection and localization accuracy, as well as robustness and generalization on out-of-distribution cases.", "questions": "- For the image generation, how many images are generated by each model? How many images per model for each prompt? What's the filtering criteria and rate?\n- For tag annotation, could you show the distribution of the tags (diversity, and distribution train/val/test)? When dividing data for SFT and RL, do you use the tag to control difficulty? \n- What is the benchmark you use for Table 1? Could you show the correlation between accuracy and tags? In what category does your method perform better, and how do SFT and RL improve the performance distinctively?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper studies an interesting task in artifact detection, localization, and explanation. They notice that the existing methods or datasets face challenges such as hallucination and low human alignment. They aim to build a reliable and interpretable MLLM-based system.\n- They have solid contributions, including curating a dataset with 28 T2I models and using it to achieve a big improvement across different MLLMs.\n- Extensive experiments and analysis are provided.", "weaknesses": "- Some important technical details are unclear, such as the image generation and filtering process. \n- For the Table 1 experiment, it is unclear what benchmark they use. If they are using their own test split, then it is less impressive. Additionally, all the baseline methods have very good accuracy, which makes it seem doubtful how challenging the benchmark is.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)