VLForgery Face Triad: Detection, Localization and Attribution via Multimodal Large Language Models
Authors: Xinan He
Yue Zhou
Bing Fan
Bin Li
Guopu Zhu
Feng Ding
Venue: NeurIPS 2025 poster ()
Keywords: Multimedia Forensics
AI safety
Primary area: applications

Abstract: Faces synthesized by diffusion models (DMs) with high-quality and controllable attributes pose a significant challenge for Deepfake detection. Most state-of-the-art detectors only yield a binary decision, incapable of forgery localization, attribution of forgery methods, and providing analysis on the cause of forgeries. In this work, we integrate Multimodal Large Language Models (MLLMs) within DM-based face forensics, and propose a fine-grained analysis triad framework called VLForgery,
that can 1) predict falsified facial images;
2) locate the falsified face regions subjected to partial synthesis; and 3) attribute the synthesis with specific generators. To achieve the above goals, we introduce VLF (Visual Language Forensics), a novel and diverse synthesis face dataset designed to facilitate rich interactions between `Visual' and `Language' modalities in MLLMs.
Additionally, we propose an extrinsic knowledge-guided description method, termed EkCot, which leverages knowledge from the image generation pipeline to enable MLLMs to quickly capture image content. Furthermore, we introduce a low-level vision comparison pipeline designed to identify differential features between real and fake that MLLMs can inherently understand. These features are then incorporated into EkCot, enhancing its ability to analyze forgeries in a structured manner, following the sequence of detection, localization, and attribution.
Extensive experiments demonstrate that VLForgery outperforms other state-of-the-art forensic approaches in detection accuracy, with additional potential for falsified region localization and attribution analysis.

VLForgery Face Triad: Detection, Localization and Attribution via Multimodal Large Language Models
Xinan He', Yue Zhou²,Bing Fan², Bin Li², Guopu Zhu', Feng Ding" Naachang University shahur@email. .cu. edu. en; fengdingencu. edu.gguucuuuuuu.u.ue..m
"Shenzhen University 2450042008@mmail. sww.e www..uu..uu..ee.oo en; libin@szu. edu. cn
University of North Texas bingfandmy.unt.. unu
Harbin Institute of Technology guopu. cepppiu.uuhhi..oom edu.
Abstract
Faces synthesized by diffusion models (DMs) with high-quality and controllable attributes pose a significant challenge for Deepfake detection. Most state-of-the-ard detectors only yield binary decision, incapable of forgery localization, attribution of forgery methods, and providing analysis on the cause of forgeries. In this work, integrate Multimodal Large Language Models (MLLMs) within DM- based face forensics, and propose fine-grained analysis triad framework called VLForgery, that can predict falsified facial images; 2) locate the falsified face regions subjected to partial synthesis; and 3) attribute the synthesis with specific generators. To achieve the above goals, we introduce VLF (Visual Language Forensics), novel and diverse synthesis face dataset designed to facilitate rich interactions between 'Visual' and 'Language' modalities in MI.LMs. Additionally, we propose extrinsic knowledge-guide which leverages knowledge from the image generation pipeline to enable MILMM quickly capture image content. Furthermore, introduce low-level vision comparison pipeline designed to identify differential features between real and fake that MLLMs can inherently understand. These features are then incorporated into EkCot, enhancing its ability to analyze forgeries in a structured manner, following the sequence of detection, localization, and attribution. Extensive experiments demonstrate that VLForgery outperforms other state-of of-te-of-the-an forensic approaches in detection accuracy, with additional potential for falsified region localization and attribution analysis.
description method, termed EkCot,
Introduction
Al-generated faces have enriched human life by creating realistic virtual characters that enhance our experiences in various creative industries. However, the domain of Al-generated face forensics is encountering unprecedented challenges due to the rapid advancement of generative models [ 55
12 24 55). In particular, diffusion models [8] B5] generate synthetic faces with substantially higher fidelity than earlier technologies such as variational autoencoders (VAEs) [41], rendering increasingly challenging for detectors to identify subtle forgery artifacts. This development has led to a significant surge in the proliferation of forgeries, which present grave threats to personal privacy and security.
Several pioneering studies 17
48 54 have investigated diffusion model -eneerened
image forensics and demonstrated the effectiveness of their proposed models. However, notable
'The author contributed equally to this work. Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

limitation of these approaches is their lack of clear analysis for the underlying causes of the forgeries. Specifically, most existing studies focus solely on detection, neglecting the localization of forgeries or attributing specific generation methods to them.
Recently, multimodal large language models (MLLMS)) have emerged as powerful tools for complex scene characterization. leveraging extensive pre-training on both images and text, MLLMS can interpret visual tasks through natural language, providing more nuanced understanding of forensic analysis [53] [4 However, MLLM-based models face two primary limitations. First, forensic analysis research with MLLMs has largely focused on traditional face-swapping Deepfakes [36 22).
with limited analysis of DM-based synthetic faces. While recent work [53] 14) has shown that automated descriptions generated by MLLMs (e.g., ChatGPT-44 or manual annotations guiding MLLM training, effectively capture artifacts in traditional face-swapping Deepfakes, these approaches still face challenges generating credible forgery descriptions, particularly for high-fidelity DM- based images where subtle artifacts often evade detection. The primary issue lies in their reliance on subjective human-defined judgment biases guide the generation of descriptions. Such descriptions may not align with the forgery knowledge that the models inherently understand. Additionally, MLLMs are typically trained for semantic-level visual alignment, lacking fine-grained forensic perception capabilities, which can lead to hallucinations in the generated descriptions. Second, none of the existing MLLM-based methods attempt fine fine-graine grained DeepFake forensics that simultaneously addresses Detection, Localization, and Atrribttion.
To address these challenges, we propose triad -nnooppasnnnn detection, localization, and atributionn-eesigned for Al-generated face forensics, referred to as VIForgery. In the proposed framework, to address the lack of data specifically tailored for DM-based partial synthesis face, we create diverse prompt repositories and templates to build new multimodal Visual Language Forensic (VLF) dataset, suitable for the triad tasks of detection, localization, and attribution.
Second, to enhance the reliability of MLLMs-generated descriptions, we introduce a low-level vision comparison pipeline that identifies low-level vision discrepancies between real and fake sample sets, guiding MLLMs to generate high- high-confidence descriptions. Building on this, we develop description generation module, where we design an Extrinsic chowledge-guides Chain-of- of-thought (EkCot) method. This method also integrates additional knowledge from the generation pipeline, enabling the model to quickly grasp image content. Third, we develop unified MILMS fine-tuning and inference module, where we aggregate the generated images and corresponding descriptions to fine-tune MILMs.
Specifically, VLForgery incorporates external information relevant to DM-based image generation with the low-level vision discrepancies between real and fake to structure a systematic analytical approach, allowing MLLMs to evaluate the image through the following steps: 1) Detection: identify whether the image real or Aggeenert: 2) Localization: For synthetic images, determine if the forgery is partial (only specific regions are altered) or full (the entire image is synthetic). For partial forgeries, locate the altered regions; 3) Attribution: Determine the likely method or model used to generate the forgery.
Our contribution can be summarized follows:
We investigate the potential of MLLMs in tackling Al-generated face forensics challenges by proposing the framework VLForgery. Additionally, we designed an extrinsic knowledge- guided mhain-of-though method, termed EkCot, that can assist the MLLMs achieving
fine- fine-grained forensics performance.
We introduce the VLF dataset, a new multimodal Visual Language Forensic dataset generated by diffusion models, designed to address the needs of fine- ane-grained forensies tasks.
We present a comprehensive evaluation experiment tailored to assess VLForgery on fine- grained forensic tasks. This evaluation encompasses three task scenarios, and 9 DM-based face types.
Related Work
2.1
DM--enerraed Images
Recent advancements have centered on diffusion models [8, which have demonstrated exceptional performance in generating high-fidelity face images with high quality. Many studies have also
2

---

Real Image
Fako Imago
Contrasting Aupects Fattures
Thee wageerggee se play...
Judgmemt Blas Filterinn
ereguker
Vision Enooder
wuli duftned conteurs
Goopare oviin
noticaable fragmantattinn
The wntggee stepp
pgt adgr
cesaity
VLM
50%
40%
20%
0%
20%
40%
60%
Low--Lever Vision Blas
Figure 1: The comparison pipeline of low-level vision. By extracting the most distinctive low-level vision descriptors that differentiate the real and fake sample sets, we establish judgment biases for the MLLMs.
incorporated attention mechanisms enhance the controllability of generated content [8 23 such
as text-guided 335 30 45 32
50 2 and subject-guided [37 21 [2 16]. These achievements have
led the proposal of many DM-based datasets, such as object-oriented image datasets [56] 39 and face-oriented image datasets ( 24). However, most DM-based partially synthesized face datasets suffer from poor quality and lack sufficient ground truth support more fine-grained forensics tasks, such as forgery localization and attribution.
2.2 Fine-grained Forensics of DM-based Images
For Detecting tasks, traditional works usually attempt find the traces left by diffusion models to expose synthesized images. For example, Chen et al. 3 proposed DRCT, a universal framework to enhance the generalizability of the existing detectors for detecting DM-based images. Recognizing the robust semantic comprehension capabilities of pre-trained vision-language models (e.g.. CLIP (34)), recent studies [1 have demonstrated their promising results in detecting DM- DM-generated images. For Locating tasks, a primary goal is for the detector to provide predictions with a localization map indicating which regions have been manipulated
identify forgery artifacts within DM- based images. Guo et al. [10] proposed a hierarchical fine-grained Image Forgery Detection and Localization (IFDL) framework consisting of three components: a multi-branch feature extractor, localization, and classification modules. Zhang et al. (5I] centers on detecting and segmenting artifact areas that are only perceptible to human observers, rather than identifying the full manipulation region. For Attributing tasks, the objective is to recognize the specific diffusion model that generates the images. Guarnera et al. concentrate on attributing DM MM-ggeneated images through a multi- level hierarchical approach. However, the aforementioned studies address the tasks of detection, localization, and attribute individually, and there a notable absence of research that integrates these tasks into unified framework.
2.3 Multimodal Large Language Models in Forensies
Recently, some studies have commenced utilizing MILMS to investigate analytical ability within the field of forensics. Jia et al. (15) pioneered the exploration of the forensic capabilities of prompt engincering using ChatGPT. Zhang al. (53) proposed incorporating common-sense reasoning to enhance traditional deepfake detection by manually labeling the manipulated regions and extending it develop Visual Question Answering (VQA) framework. Huang et al. 14] recently proposed an automatic method utilizing GPT-40 for annotations, thereby replacing human annotators. These studies primarily focus on traditional Deepfakes and are limited to fngle-forensic scenarios, lacking the capability to locate or attribute forgery areas in synthesized images. In the case of forgery artifacts from high-fidelity faces generated by diffusion models, generating credible forgery analyses may pose greater challenges. Ours, compared to other MLLM-based forensic frameworks, can guide MLLMMs to generate highly credible fine-grained forgery descriptions and is applicable to a broader range of forgery types and more complex forensic scenarios.

---

Facial Attribute bichionsry
Looncollpten dndagraih dhaie
Partal Syntisss
6PT-40
(rrots) for vderi, Irooutant, berkeel, Jewer sonuen, frme
Modil Croak dooomeau (3rrr
Findl Prompt
A persaa
dat
Reedom
Regional Teat Dascripiion
slirr and glosky coded spai featensl, and
Data Acqulsition
Pristine Fack
Landftrkk
Pristing Foce
Idantiy
Aftribane Dictionary
Extractor
Nareeiid duslage diy weee
Synthers
carty
(b)
eyswng
tryle shotel photeempty phite (preto Spender, Drepttart Jusaan) (mars (yene ww_.e...........
Extdent
Final Promte
Seaty pwstgraa photo notaa
GPT-40
(c)
Prompt Worchouse Template
Nathe mrentr.....
Extrinsic Knowledge
which
cunentie Aadiecacyy fur oppoee atgulyy aforrd wit
Desctipton
Generation
Rital Aneaar
MLLMs
Low-LLvsal
(d)
Progar eoo Che Actucta weach woeel Jarraue fusc oieec
MLL Mo Free manning
jend Interence
Fiit tuniigg Stuadarr
<Ouestoox
Gantioal be lake?? What anddeica suspprrte your
Fine-tuning
21 ZiLoowing of this image is genuiio or falot duucs 1o00 you comotssoo??
VLF+ EkCot
MLLMs
MLLMs
Inference
<Visuat>
(e)
Figure 2: Construction of three modules for VLForgery framework: Data Acquisition, Description Generation Module, and MILMS Fine-tuning and Inference Module.
VLForgery Framework
With the development of generative technologies, methods like diffusion models have emerged. Compared with traditional generative AI techniques (eg, VAE), the diffusion models enable partial synthesis guided by masks and text descriptions, well full synthesis through identity insertion or text descriptions. Therefore, our VI_F dataset encompasses both partial and full synthesis types. In the following sections, will demonstrate how acquire their corresponding prompts and/or images. Furthermore, we will elaborate on the specific details of our proposed Description Generation Module and MLLMs Fine-tunning and Inference Module in VLForgery.
3.1 Data Acquisition
This section primarily outlines the pipeline details for collecting data. We synthesized all DM- generated data according to the requirements of the fine- -ree-graied forensics tasks. The details of the generation, involving partial and full synthesis, will be deseribed in the following sections. For more detailed introduction to the dataset, please refer to Sec.
3.1.1 Partial Synthesis
In this part, we define the pipeline for partial synthesis. An exemplar pipeline is illustrated in Fig. a).
To ensure the high quality of the partially synthesized images, we utilized the well-established facial dataset CelebAMask-HHr [19 as data source since it already contains landmarks for various face attributes such as nose, brow, hair, ears, eyes, and teeth. Then, facial attribute dictionary and prompt template. For each image, we randomly combine masks to generate masked faces. Based on the generated masked face, choose and combine templates for the corresponding attributes, then randomly select descriptive keywords from the relevant attribute dictionary populate the templates, thereby generating the final prompt for image partial synthesis. During the image generation phase, we input the generated masked face, corresponding final prompt, and pristine face into the Inpainting Generator to produce inpainted face.
GPT-4o to generate each

---

3.1.2 Full Synthesis
We introduce two types of fully synthesized forgery: generating identity insertion (Fig. 2(b)) and non-existent faces (Fig. ([.) Both generation approaches utilize the same prompt template with slight modifications. In the face identity insertion pipeline, the objective is to preserve the original facial identity while situating it within a new scene. This pipeline initially uses Face Feature Extractor to capture identity-speciffe features. In the generation of non-existent faces, the diffusion model creates images of completely new faces that are unrelated to any existing individual based on a given prompt.
3.2 Description Generation Module
This module leverages the extrinsic information required throughout the low-level vision comparison pipeline and the data acquisition processes guide the construction of the forgery analysis description for each type of forgery sample in the VLF dataset. In this section, we introduce the workflows of the low-level vision comparison pipeline and the construction of EKCot separately. We provide detailed low-level vision analysis along with all the details and data you may need (refer to Sec C).
3.2.1 Low-level Feature Comparison Pipeline
researchers attempted generate explainable descriptions for Deepfakes using Vision Large Language Models. However, hallucinations are commonly observed in VQA tasks. Forensic analysis places more emphasis on the perception of image fine- ane-grrined details compared with general VQA tasks. Therefore, in the absence of ground-truie truth labels to supervise these explainable descriptions, how can we determine their correctness?
Motivation: In previous studies 46
To address this challenge, we aim to seek forensic features that the model itself can understand to discriminate the authenticity of an image, thereby mitigating the model's struggles with explainable description generation. Therefore, we propose a low-level vision comparison pipeline that leverages the visual feature understanding capabilities of low-level vision discrepancies between real and fake sample sets, as illustrated in Fig. 1 First, we randomly select one image from the real sample set and another from the fake sample set, Iı and 1z. Notably, the VLLMs is unaware of which image is real and which is fake. Furthermore, to comprehensively verify the low-level visual discrepancies between the real and fake sample sets, we selected ten distinct aspects and designed corresponding descriptive templates for each, as Teem (refer to Sec. C.TI.
models (VLLMs) identify
Initially, 1 and T2 are processed by the vision encoder E,, which generates the corresponding token representations T Tim-- These tokens, along with the comparison template tokens Trem. are then fed into the V VLLM() () for generating low-level vision discrepancies text O, It is formulated as:
Tmgg T ing I2),
(la)
0 Of = VLLM(TT smg: Tamg: T Trem)-
(Ib)
Judgment Bias filtering: Given 0 O², we first perform tokenization and part-of-speech tagging extract compound noun phrases. Next, we compute the proportion of each extracted compound noun phrase across all low-level vision discrepancy texts. Finally, we identify the top ten phrases with the largest proportion difference between the real and fake sample sets, selecting them as the judgment biases for the MLLMs. These phrases are then used to generate the final usable judgment bias descriptors through a predefined template (refer to Sec. C.3
Furthermore, to avoid potential model interpretation biases, we selected the Llama-3.22 2-1 BB-Vision model as the VLLM for the low-level vision comparison pipeline, ensuring consistency with the models used in subsequent stages.
3.2.2 Construction of EkCot
As shown in Fig. d), we utilized MLLMs the description generation model to construct a chain of thought for forgery analysis, termed EkCot. Initially, we propose General Rule template designed to integrate the extrinsic information required for generating desired data. The extrinsic information includes the prompts used within the pipeline, the corresponding ground truth for the generated image, and the low-level vision judgment bias, which are subsequently combined to serve as prompt inputs for MLLMs. The ground truth encompasses several categories, including real/fake, forgery type,
5

---

forgery region, and forgery method. For pristine images, the forgery type, forgery region and forgery method values are set 'none' by default. Additionally, image original features are provided as visual inputs to the MIL.Ms. Furthermore, we utilize the Llama-3.2-11 11-22-11---1 B-Vission as the description generation model.
3.3 MLLMs Fine-tuning and Inference Module
In this module, we designed the final fine-tuning data structure and the details of MILMs fine-tuning. Final Fine-tuning Data Structure. The final fine-tuning data is composed in the form of triplets:
<visual, question, answer>. For question's format, use ChatGPT-40 to generate range of
question formats, shown in Fig. 2[e), for example: 'Is this image real fake? Can you provide the reasoning behind your judgment?' For each triplet, the question is presented in a randomly generated format. Additionally, each image is linked to its corresponding specific description (i.e., the answer). For the answer's 's format, we designed to address three primary forensic tasks: detection,
localization, and attribution. The answer format follows this pattern: 'Detection Result: real/fake' + Avesuts + FFrgeey Type' 'Falsified Region' + 'Forgery Method", where Aresuit denotes the analysis of detection results. For analysis steps, our objective to guide the model to adopt a chain of thought,
performing each step sequentially follows: 1)Raal Fake Judgment: The authenticity of the image is first assessed. If an image is identified as a forgery, the type of forgery is then classified into one of three categories: partial synthesis, identity insertion, and non-existent faces. The reasons for the detection result are further analyzed. 2) Localization of Falsified Region: If the forgery involves partial synthesis, the specific regions of manipulation (e.g.. nose, brow. hair, ear) are identified. In cases involving multiple edits (e.g., hairAndnose), each forged region is localized. 3) Attribution of Synthesis Face Generator: The final step determines the generator of the synthesis face. Fine-tunning
and Inference Process. We fine-tune the MLLMs the final fine-tuning data through a structured two-step process (Fig. ff). Inspired by LLaVA [25], we adopt their published Llava- 1.5-7B model and training architecture. First, fine-tune projector to align the facial visual features extracted from the frozen CLIP visual encoder with the corresponding question text features. This alignment links the synthetic face's forgery artifacts with corresponding ground truth and detailed forgery descriptions. Second, we employ Low-Rank Adaptation (LoRA) [3] (rank=128, alpha=256) to selectively update the model by adjusting only the LoRA parameters, effectively fine-tuning the pre-trained language model.
During inference, the input is structured tuple <image, question>> For the questions, we continue to use ChatGPT-4 to generate diverse question formats, ensuring input variety. Finally, the model's response is structured, similar to the final fine-tune fine-tuning data's 's answer format.
This approach ensures the model conducts forensic analysis on input faces a structured, stepwise manner according to pre-defined reasoning framework, thereby improving both its fine-grained forgery analysis and forensic accuracy.
Table 1: VLF dataset.
Samples
Type
Source
train
test
IFHHQ
42k
14k
Pristine
FF++ (original)
12k
4.5k
CelebA-HQ
18k
6k
SDXL.
76k
19k
DM--haed Partial Synthesis
SD2
19k
Kandinsky2.2
19k
SDXL
SD2
SD3
24k
DM-based Full Synthesis
Kandinsky2.2
Flux
InstantID
2.5k

---

Lieva-v1.5-78 (0 shot)
Question:
The tumnn lass image natural
man beard mustache evidenee
supoorting conciution ta the presence the mants hat: which as dutinitiee festure aeperrance. The man in wno weering eui, unich hirther indioese that he is neal parson. The image shows the nan ngiing next to doy, and the is als a real arinnt, which furticr supoirts the idoa that the scene is natural one.
uredite mostted? What factors infurnoed your
yidgment?
(Real)
Liava-vI.S-7 (only Image
VLForgey Image vision Becnnn+ECC////// EcCoo/LLow-ooeeeee Vision Judgment Blas Extrinsic Knowledgel)
Viston Extrinsic Knowledge)
Detection (ake
Detectton Result: Forgery
Fargery onalysis:
a lake face gererated by a modis wussstoit midi the fot that faso inapes olton cetdicc rightr smage qualicy Additorally the proppe looks any parpttuc sonarts plois wil Low-Leee
me
with 20t compraciion anstect,
anslacte TIV
C conimaa prac (ale chiges
Knowlledge
evientred the uncatual ippearance teatuve
ooter rpandredired The furgery mothed used generate nis image Mely stid, as isss conmos meeioud used for gensating highquality inaes The
the inopi,
complesety
inces and ruarses dhet are chanctendde mas phonogmepts
sin the culri
imagiyy and quality consutent those produced by Solt.
Falsified Type: D01 pxts--etteetttt haces
Fargery Type: noneeszatent loces Falsified Region: Tull synthesls
full sunthesst
Forgery Mechod: SDXL
Trake,non-exiiteen faces, full synthesis,
Forgery Method! 5D%
(Fake)
Charipyy4.55
Booll on the image you provided, the hur face appears to be natural one. hou's the reasering Pocul Festuros: The hurnnn lase the inage has rec colkock progurtiats such Llc symmetry the eyes, noee, and mouth. There are unseual datortions artiacts that are common Algennmated fazes (e.g. mitemtttedd eyec, ndd son tedu m. unaligned features) Expr ession The fock expresio essisn, combired with the doy's inteaction, loois natural The subjects pose and gaze are cohterett wiihh signe with how a real human would betrave a porbult Ligh ting and Shadowas The lightin on the human face la consatantt, with shadows falling nuturall by sesed on the light sourssa. Pake focss sometimes have urisstrna march the detel comturs Troperyy Denall The den becture hals other delain the mustamte appoor ratumi with dtecltionn or sliniig that soretinns soon in goncrated contert This evidance suggests that the human foce in this image is real rochir than artfically genasstot Homorrs Aither ane yelr uning frrends bocle could ofter add Bonnl vertfcation.
wouse steet lighting where the shadawe
(Real)
Figure 3: Qualitative results. The upper left presents synthesis face from full synthesis. Ours is presented in contrast to the responses of Llava-v1.5-7B (0 shot), Llava-v1.5B-77 (conditional fine-tune), and Chatgpt-4.5, respectively. The lower-right bracket provides the result annotations.
4 Experiment
4.1 Experimental Settings
Datasets. All facial samples in experiments were sourced from the VLE The VLF dataset consists of 12 subsets, with each subset divided into training and test sets in a 4:1 ratio, as illustrated in Table 1
Compared Baselines. We selected four types of models for evaluation: 1) Naive convolutional
neural networks (CNNs): Xception (6), Resnet-50 [T]. 2) Typical frequency detectors: SPSL
26). 33), (2], NPR[40]. 3) Typical Spatial detectors: (ramNet(28), SAFE[20),
CNNspo 43), DRCT B. 4) VLLM-based CLLping [17], Llava-1.5-7B 25], Qwen2.5-VL-
7B 47
Llama3.2- 3.1 IB-IB--AAA [VViioon[[]
Implementation Details. All experiments are based on the PyTorch and trained with 8 NVIDIA RTX L40. For training, we utilized the Adam optimizer with learning rate of 2e-5 and a batch size of 128 for 3 epochs.
Table 2: Evaluation Full Synthesis type. Results on different training and testing subsets using VL.Frgery. Accuracy used for evaluation.
Full Synthesis($$)
Train
Non-cxisicnt
ld-inscttion InstaniLD
SDXL
Kandinsky2.2
SDXL
96.35
98.47
96.02
98.78
98.7
$7.38
96.45
SID2
99.47
99.97
98.38
99.40
98.92
80.67
96.
Kandinsk22.
99.95
99.37
97.95
99.95
99.67
93.75
98.44
Flux
98.01
95.85
98.10
98.25
100.00
48.10
74,89
InstantID
90.38
87.05
60.83
87.40
84.97
92.97
83.94
Table 3: Evaluation on Partial Synthesis type. Results different training and testing subsets using VLForgery. Accuracy used for evaluation.
Partial Synthesis(%)) y
Train
SDXL
SD2
Kandinsky2.7
Avg(5))7
SDXL
80.35
68.78
98.08
82.40
SD2
54.53
61.77
69.77
Kandinsky2.2
42.09
34.20
95.93
57.41
7

---

4.2 Evaluation of Multi-task Forensics
4.2.1
Task1: Detecting the Authenticity of Faces
Table Cross- geseertor dataset evaluation on ACC metric. All methods are trained on SDXL and evaluated on other subsets. * indicates the use of trained models provided by the authors. Note that "PS' means Partial Synthesis, and 'FS' means Full Synthesis.
Type
Mechol
(Oin-SDXX((55))
SD2(%))
Sandincyy2.2455
SD3/S]T
InsamiDD(9]]
PS
PS
Naive
Resnet
Xoeption
11
68.19
97.55
99.03
99.57
88.75
SPSL
31.62
99.58
4.79
96.92
51.42
99:33
96.92
99.28
95.09
75:09
Frequency
F1Na 33
SRM 29
59.21
74.09
99.95
99:01
46.97
12.30
99.45
98.05
81.65
98.34
99 00
99.57
98.41
90:33
09. 00.95
98.17
97.06
99 8484
89.01
84.65
NPR 20
42.67
99
10:22
98.95
70.51
99.57
98. 25
99.75
99.37
79.92
GramNet 23
14:28
96.93
48.37
99.27
74,94
Spatill
$7.05
CNNapod
70.00
DRCT
57.65
99.41
53.50
97.71
99
99.92
65.97
59.00
98.92
81.83
CliPping Ansper L7
36.29
39151
12.13
V4..3
75,044
95:35
58.38
97.35
57.03
81.05
CLiPping Linoar Prakery
77
61.79
99.98
$4,33
97.18
87.33
99.65
97.88
99.03
99.80
88.55
VII M-Ravl
Qwen2.5V1 5VB 37
Llava- 1.5-78 2S
69.92
4.77
99.92
4.32
51.07
3.78
99.97
21.88
31.92
99.56
99 71
6.55
97.66
3.82
99.12
3.00
46.13
99.69
87.07
14.02
Liama 3.2. Vision
08.95
58.42
43,80
63.05
VI Vi.Forger
(ours)
66.32
99.90
99.97
99.57
Table 5: Evaluation of falsified regions localization performance across distinct generators. Accuracy is used for evaluation.
Itan
AugeRat
Angetsat
Angtio1
Augrisat
SDXL
5D2
Kasdandy22
SDX0L
$00
Kasleoke22
(DDXL
S02
Kassinike2.2
SDX0L
502
77.35
SD2
95
40.16
Kaxlinky2.2
58.53
30.47
35.9
64.99
3433
13,67
79.38
42.66
29
24.98
71.61
42.12
14.01
16.11
$0.04
59.73
Intra-type performance with MLLMs. In this task, we first evaluate the Full Synthesis type per- formance with VLForgery. As shown in Table the Full Synthesis type comprises two subtypes: Non-existent and Id-insertion. The non-existent face subtype consists of five distinct subsets, each corresponding to specific generator. For the non-existent subtype, training and testing within each subset surpass 99.35% accuracy, while cross-subset testing achieves an average accuracy of 91.66%. Conversely, cross-subtype performance varies significantly. For instance, training on the Flux subset and testing on InstantID results in classification accuracy drop to 48.1%.
Table presents the performance evaluation of the Partial Synthesis type. This type consists of three subsets, each associated with a specific generator. Training was conducted on each of the three subsets, and testing was subsequently performed across all subsets. As indicated in Table training on partial synthesis samples generated by SDXL-based models yields superior generalization performance.
Based on these observations, detectors trained with full-synthesis images are typically trained based on global features, focusing on the overall structure and content of the image. Also, they primarily focus on analyzing the integrity of the entire image. However, partial symthesis images often involve modifications to small areas of the image, and these details may not be obvious in the global features. Therefore, when these detectors encounter partial synthesis images with only small local area tampered with, they may fail to capture the subtle changes, leading reduced recognition rate for local tampering.
Goss-Generrttrr Faces Classification. We propose evaluating the generalization performance of distinet generators, encompassing both Full Synthesis and Partial Synthesis types. As illustrated in Table 4 all detectors were trained on samples with partial synthesis and full synthesis from the SDXL- baxed model, and subsequently tested on other subsets. For each compared method, we trained for 10 epochs and validated their performance the final epoch. VLForgery exhibits a notable advantage, especially in accurately detecting partially synthesized face images
4.2.2 Task2: Locating the Falsified Regions
We evaluate the localization capabilities VI.Forery for four distinet regions (i.e., nose, brow hair, and ears), as shown in Table 5 The Partial Synthesis type contains edited faces with single-edit and
8

---

Table 6: Evaluation of model's attribution accuracy VLF.
Method
FS. SDXL
FS_SD2
FS FS_SD3
FS_[ID
FS. Kan222
SS_Flux
PS_SDXT.
PS SD2
PS Kan2.2
Ave.
reanet50
99,93
99.53
84.43
98.82
99.23
99.87
11.81
6.09
76.48
75.13
sception
99.98
99.97
87.93
99.57
99.37
99.52
16.87
27.64
91.81
80.30
efficientb4
99.99
99.88
88.93
98.15
98.95
99.90
27.61
27.85
81.43
80.29
Guarnera et
99.78
99.87
86.65
97.72
99.92
99.43
35.75
12.71
90.17
80.22
VLForgery 93.27
92.47
93.24
90.42
89.13
82.43
63.64
60.20
73.45
82.03
Table 7: Ablation Study the VLForgery's forgery description generation component in the detection task. "VF', and 'EkCot' represent visual input features in forgery description generation, and extrinsic mholledge-guides chain-of- tf-hhughtt, respectively
Method
SD2
Desection8337
Kandinnky222
SD3
LostastID
Name
EXCxx
Puntal Symthesis
Full Systhesis
Parial Synthesis
Svuthesis
Partal Syuthesis
Sxnthesis
Vanama
90:92
11.00
96.97
90.09
VarianB
99.97
Table 8: Ablation Study of the VLForgery's 's forgery description generation component in the localiza- tion task. 'VF', and 'EkCot' represent visual input features in forgery description generation, and extrinsic knowledge- guided chain-of- ohain-of-thought, respectively.
Mahhod
Locdionisat
VF
BKCoi
SDXL
SD2
Kandinsky222
SDNL
Kandinsky2.2
SDXL
Kandinsk22
SDXL
Kandinsky222
59.40
38.1
13.00
76
23.56
30
42,3
73.84
72.01
7201
77.35
multi-edit. Additionally, this type is categorized into three subtypes based on distinct generators. Each subtype was trained separately, and tested across subtypes, with validation results tallied across the four facial regions. Note that, if multi-edited face is detected with the result 'earAndnose', but the ground truth is 'carAndhair', the detection of the ear region marked as correct, while the hair region is marked as incorrect.
Why Use Natural Language Instead of Masks for Localization Results? Fig. illustrated a compari- son of localization results between VLForgery and PSCC-Net (IFDL method). For subtle falsified facial regions, such as brows and nose, IFDL methods often struggle to identify them accurately, being easily influenced by extraneous features from the image background, which leads to incor- rect localization. In contrast, for facial localization, where tampered areas are limited, VI.Forgey leverages natural language descriptions instead of mask images to reduce the difficulty of localization. This approach significantly enhances local-
ization accuracy and minimizes susceptibil- ity to interference from background features. Furthermore, quantify the performance gap in localization between ours and the IFDL method, we selected 1,000 samples from the Partial Synthesis test set. We used the IFDL method generate mask images, which were then manually evaluated. A lo- calization deemed correct if any part of the manipulated area was slightly visible in the mask image; otherwise, was considered incorrect. As shown in Table 9 our method demonstrates a clear advantage accurately localizing manipulated regions.
Edit
bais
bzow
bait
bair
Region
Pertal
Synthesis
PSCC-Net
VLIorgger::
(sung)
db-hair
beooww
Andbrow
nose
hhar
Androse
Andbrow
thair
Figure 4: Comparative results of tamper facial local- ization capabilities between IFDL method and VL- Forgery. Both models were trained on partial synthe- sis generated by SDXL, and tested on Kandinsky2.2 samples. Mislocalized areas are marked in red.
Pross-Generator Falsified Region Localiza- tion. As observed Table [ localization
accuracy for hair consistently remains the highest, followed by the nose, brows, and then the ears. For instance, when trained and tested on SDXL. samples, the hair localization accuracy reached 88.99% In contrast, localization accuracy for smaller regions, such the ears, decreases to 59.1699..19 39.16%.

---

Table 9: Evaluation of falsified regions localization performance across distinct generators with Vorgery and IFDL method.
SD2(SOT
Kandinsky2.2(59))
Method
hair
brows
ears
nose
hair
brows
cars
nose
PSCC-Net
53.57
39.47
41.20
29.73
60.71
68.42
63.16
44.44
VLForgery
86.91
53.36
51.99
56.53
92.44
72.61
72.01
91.34
4.2.3 Task3:Attributing Source Models of Synthesis Faces
In this task, we focus on evaluating the VLForgery's capability to capture the synthesis semantic patterns of different generators. We fine-tuned VI.Forgery on the VLF training which includes all subsets, and tested its performance across each generator for the two synthesis types. As shown in Tab. 6 the attribution performance for full synthesis is superior, with average accuracy of 91.71 1.71% %. However, attribution accuracy for partial synthesis falls below 70%. We attribute this disparity to the detection difficulty of partial synthesis.
4.3 Ablation Study
Two main factors affect VLForgery forensics performance: (1) the impact EkCot compared to outputting only forgery detection results, and (2) the influence of additional visual input features forensics performance EkCot. We conducted ablation studies for both detection and localization tasks. Table 7 presents the ablation results for the detection task, demonstrating significant performance improvement in detecting partially synthesized images when using the EkCot. Table displays the ablation results for the localization task, where the model's localization capability significantly decreased without EkCot. And compared to the forgery description generation method without visual assistance, the model performed better on SDXL and SD2 samples, though it showed some limitations on Kandinsky 2.2 generated samples.
4.4 Qualitative Study
The primary advantage of the multimodal large language models (MILMs) their flexible output, allowing them to provide comprehensive analysis of the synthesized images, As shown in Fig. 3 we compared VLForgery with some existing MLLMs, demonstrating that VLForgery enables detailed capabilities, including forgery analysis, localization of falsified regions, and synthesis attribution.
5 Conclusion
In this paper, we introduce VLFFrgery, a framework designed for Al-generated faces in fine-grained forensics scenarios. To address the lack of partially synthesized face datasets, constructed VLF. Additionally, we introduce EkCot, which provides fine- ane-grained analysis forgery artifacts. The satisfying performance of the proposed framework is justified by extensive evaluations.
Acknowledgments and Disclosure of Funding
This work was supported in part by the National Natural Science Foundation of China (Grant U23B2022, U22B2047, 62262041, 62172402), the Shenzhen R&D Program (Grant JCYJ2025060418 121 1016. XSSPG202422111770320004 and the Jiangxi Provincial Natural Science Foundation (Grant 20232BAB202011).
We also sincerely thank Shu Hu, from the Department of Computer and Information Technology, Purdue University, IN, 46202, USA, for providing insightful advice and support during the preparation of this manuscript.
References
[1] Llama-3.2-11b-vsiio.
In
https: hww.gnggggccc....m///// co/meta- llama/ Llama- 3,
2- 2-11B- Vissinn Instruct 2024.
[2] Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392-18402, 2023.
10

---

[3] Baoying Chen, Jishen Zeng, Jianquan Yang, and Rui Yang. DRCT: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images. In Forty-first International Conference on Machine Learning, 2024.
[4] Ruoxin Chen, Junwei Xi, Zhiyuan Yan, Yue Zhang. Shuang Wu, Jingyi Xie, Xu Chen, Lei Xu, Isabel Guan, Taiping Yao, et al. Dual data alignment makes ai ai-generated image detector easier generalizable. arXiv preprint arXiv:2505. 14359, 2025.
[5] Harry Cheng, Yangyang Guo, Tianyi Wang, Liqiang Nie, and Mohan Kankanhalli. Diffusion facial forgery detection. arXiv preprini arXiv:2401.1558 2024.
[6] Francois Chollet. Xception: Deep learning with depthwise separable convolutions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul 2017.
[7] Riccardo Corvi, Davide Cozzolino, Giada Zingarini, Giovanni Poggi, Koki Nagano, and Luisa Verdo- liva. On the detection of synthetic images generated by diffusion models. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023.
[8] Bing Fan, Shu Hu, and Feng Ding. Synthesizing black-box anti-forensies deepfakes with high visual quality. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4545-4549. IEEF, 2024.
[9] Luca Guarnera, Oliver Giudice, and Sebastiano Battiato. Level up the deepfake detection: a method to effectively discriminate images generated by gan architectures and diffusion models. In Intelligent Systems Conference, pages 615-625. Springer, 2024.
[10] Xiao Guo, Xiaohong Liu, Zhiyuan Ren, Steven Grosz, Iacopo Masi, and Xiaoming Liu. Hierarchical fine-grained image forgery detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3155-3165, 2023.
[11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2016.
[12] Xinan He, Yue Zhou, Wei Ye, and Feng Ding. A3:
Ambiguous aberrations captured via astray-learning for facial forgery semantic sublimation. arXiv preprint 1515,3, 2024.
[13] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu. Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2 106.09685 2021.
[14] Zhengchao Huang, Bin Xia, Zicheng Lin, Zhun Mou, and Wenming Yang. Ffaa: Multimodal large language model based explainable open-world face forgery analysis assistant. arXiv preprint arXiv:2408 10072, 2024.
[15] Shan Jia, Reilin Lyu, Kangran Zhao, Yize Chen, Zhiyuan Yan, Yan Ju. Chuanbo Hu, Xin Li, Baoyuan Wu, and Siwei Lyu. Can chatgpt detect deepfakes? a study of using multimodal large language models for media forensics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4324 4333, 2024.
[16] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Trani. Imagie: bastd real image editing with diffusion models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 6007-6017, 2023.
[17] Sohail Ahmed Khan and The- Dang-Nguyen. Clipping the deception: Adapting vision- laiiuge mod- cls for universal deepfake detection. In Proceedings of the 2024 International Conference on Multimedia Retrieval, pages 1006-1015, 2024.
[18] Max Ku, Tianle Li. Kai Zhang, Yujie Lu, Xingyu Fu, Wenwen Zhuang, and Wenhu Chen, Imagenhub: Standardizing the evaluation of conditional image generation models. arXiv preprint 10.01:23 2023.
[19] Cheng- Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image manipulation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[20] Ouxiang Li. Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Fuli Feng. Improving synthetic image detection towards generalization: An image transformation perspective. arXiv preprint arXiv:2408 067411 2024.
11

---

[21] Tianle Li. Max Ku, Cong Wei, and Wenhu Chen. Dreamedit: Subject-driven image editing, urXiv preprint arXiv:2306. 12624, 2023.
[22] Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 3207-3216, 2020.
[23] Li Lin, Necraj Gupta, Yue Zhang, Hainan Ren, Chun-Hao Liu, Feng Ding, Xin Wang, Xin Li, Luisa Verdoliva, and Shu Hu. Detecting multimedia generated by large ai models: A survey. arXiv preprint arXiv:2402.00045 2024.
[24] Li Lin, Xin Wang. Shu Hu, et al. Ai-face: A million-scale demographically annotated ai-generated face dataset and fairness benchmark. arXiv preprint arXiv:2406.0000 00783, 2024.
[25] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
[26] Honggu Liu, Xiaodan Li, Wenbo Zhou, Yuefeng Chen, Yuan He. Hui Xue. Weiming Zhang, and Nenghai Yu. Spatial-phase shallow learning: rethinking face forgery detection in frequency domain. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 772-781, 2021.
[27] Xiaohong Liu, Yaojie Liu, Jun Chen, and Xiaoming Liu. Pscc-net: Progressive spatio-channes correlation network image manipulation detection and localization. IEEE Transactions on Circuits and Systems for Video Technology, 32(1 1) 77775555771777 2022.
[28] Zhengzhe Liu, Xiaojuan Qi, and Philip HS Torr. Global texture enhancement for fake face detection in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8060-8069, 2020.
[29] Yuchen Luo, Yong Zhang, Junchi Yan, and Wei Liu. Generalizing face forgery detection with high- froquency features. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 16317-16326, 2021.
[30] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6038-6047, 2023.
[31] Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards universal fake image detectors that generalize across generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24480-244444 24489, 2023.
[32] Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan La, and Jun-Yan Zhu, Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1-11, 2023.
[33] Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in frequency: Face forgery detection by mining frequency clues. In European conference computer vision, pages 103, Springer, 2020.
[34] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
[35] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10660 10695, 2022.
[36] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Niess- ner. Faceforensics*#:: Learning detect manipulated facial images. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Oct 2019.
[37] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition, pages 22500-22510, 2023.
[38] Jascha Sohl- Sohl-Dickstedd -icckstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256-2265. PMLR, 2015.
12

---

[39] Andreas Stöckl. Evaluating synthetic image dataset generated with stable diffuusion. In International Congress on Information and Communication Technology. pages 805-818. Springer. 2023.
[40] Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei, Rethinking the up- sampling operations in cnn-based generative network for generalizable deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28130-281333 2024.
[41] Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. Advances in neural information processing systems, 33:19663 196667199779 2020.
[42] Qixun Wang. Xu Bai. Haofan Wang, Zekui Qin, Anthony Chen, Huaxia Li, Xu Tang, and Yao Hu. Instantid: Zero-shot identity-preserving generation in seconds, arXiv preprint vrXiv:2401.075/99 2024.
[43] Sheng- Yu Wang. Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot... for now. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8695-8708 8704, 2020.
[44] Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. Dire for diffusion-generaaeed image detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22445 22455, 2023.
[45] Chen Henry Wu and Fernando De la Torre A latent space of stochastic diffrusion models for zero-shot image editing and guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7378-77 777877877,. 2023.
[46] Zhipei Xu, Xuanyu Zhang, Runyi Li. Zecheng Tang. Qing Huang, and Jian Zhang, Fakeshield: Explain- able image forgery detection and localization via multi-modal large language models. arXiv preprint arXiv:241002761.. 2024.
[47] An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li. Dayiheng Liu, Fei Huang. Haoran Wei, et al. Qwen2. technical report. arXiv preprint 4rXiv:24/2.5511, 2024.
[48] Zheng Yang, Ruoxin Chen, Zhiyuan Yan, Yue Zhang, Xinghe Fu, Shuang Wu, Xiujun Shu, Taiping Yao, Shouhong Ding. and Xi Li. All patches matter, more patches better: Enhance ai-generated image detection via panoptic patch learning. arXiv preprint arXiv:2504.01390 2025.
[49] Wei Ye, Xinan He, and Feng Ding. Decoupling forgery semantics for generalizable deepfake detection. arXiv preprint arXiv:2406.0977 097399979999... 2024.
[50] Kai Zhang, Lingbo Mo. Wenhu Chen, Huan Sun, and Yu Su. Magicbrush; A manually annotated dataset for instruction-guiae guided image editing, Advances in Neural Information Processing Systems, 36, 2024.
[51] Lingzhi Zhang, Zhengjie Xu, Connelly Barnes, Yuqian Zhou, Qing Liu, He Zhang. Sohrab Amirghodsi. Zhe Lin, Eli Shechtman, and Jianbo Shi. Perceptual artifacts localization for image synthesis tasks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7579-7590, 2023.
[52] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3847, 2023.
[53] Yue Zhang, Ben Colman, Ali Shahriyari, and Gaurav Bharaj. Common sense reasoning for deep fake detection. arXiv preprint arXiv:2402.00/26 2024.
[54] Yue Zhou, Xinan He, KaiQing Lin, Bin Fan, Feng Ding, and Bin Li. Breaking latent prior bias in detectors for generalizable aige image detection. arXiv preprint arXiv:2506. 008744 2025.
[55] Yue Zhou, Xinan He, Kaiqing Lin, Bing Fan, Feng Ding, Jinhua Zeng, and Bin Li. Brought gun to a knife fight: Modern vfm baselines outgun specialized detectors on in thed ai image detection. arXiv preprint Xrx::25009. 12995, 2025.
[56] Mingjian Zhu, Hanting Chen, Qiangyu Yan, Xudong Huang, Guanyu Lin, Wei Li, Zhijun Tu, Hailin Hu, Jic Hu, and Yunhe Wang. Genimage: million-scale benchmark for detecting ai- ai-generated image. Advances in Neural Information Processing Systems, 36, 2024.
[57] Xiaogang Zhu, Bo Lin, Xinan He, Jianfeng Xu, and Feng Ding. Tmfd: Two-stage meta-learning feature disentanglement framework for deepfake detection. In 2024 IEEE International Joint Conference on Biometrics (IJCB) pages 1-8. IEEF, 2024.
13

---

Structure of Contents
Section[A] Additional Introduction:
Section A.1
Limitations:
Section A.2
Future Works;
Section R Details of VLF Dataset Construction:
Section B
Details of Partial Synthesis Pipeline;
Section B
8.2
Details of Full Synthesis Pipeline;
Section B
3
Details of Attribute Dictionary:
Section B
4
Examples of VLF Dataset:
Section Details of Low-Level Feature Comparison Pipeline:
Seection
Design of Low-Leve Vision Prompt:
-Section
Analysis of Low-Level Vision Description;
Secction
C.3
Analysis of Low-Level Vision Bias;
Section[D] F Full
Experiment Result of VLF:
-Section
D.
Quantitative Analysis Results;
-Section
D.2
Experimental Results Supplement:
Section
Additional Information on VLForgery:
Section Details of the
and Inference Modules;
Section
Comparison of the Inference Result with Other MLLMs.
A
Additional Introduction
A.1
Limitations
Most existing detectors for Al-generated content achieve relatively good generalization on fully synthesized images with minimal training data. However, as discussed in the main text, we introduced additional samples in VLF partial synthesis. These samples incorporate real features from the pristine images to confuse detectors and evade detection. Even when these partially synthesized images included in the training set, many binary classifiers still suffer from performance degradation. Although our VLF framework achieves the best generalization performance, it still falls short of our desired outcomes. Furthermore, the cross-genes genssseenealiiztioon ability between different types of models, such as training on full synthesis and testing on traditional face-swapping deepfakes, is inherently limited due to differences in the generation processes. Enhancing generalization for more unknown generation methods remains field worth exploring.
Moreover, for the VLF dataset, different generators and varying prompts may leave distinct forgery semantics. is necessary to expand the dataset and evaluate the detection and generalization performance across these different forgery semanties.
A.2 Future Works
First, we plan to expand the VLF dataset to encompass broader range of forgery types, generators, and a more diverse and randomized set of semantic attributes. Second, we seek to enhance the model's generalization across varying forgery types by developing more robust model architectures and investigating more reliable techniques for generating forgery descriptions.
B Details of VLF Dataset Construction
B.1 Details of Partial Synthesis Pipeline
In this part, we define the pipeline for partial synthesis, which includes three steps: 1) Mask Preparation; 2) Prompt Design; 3) Image Generation.
Mask Preparation. To ensure the high quality of the partially synthesized images, we utilized the well-established facial dataset BelebAMask-HHO [19 as a data source since already contains landmarks for various face attributes such nose, brow, hair, ears, eyes, and teeth. However, to improve the robustness our later fine- -raingr analysis framework for more highly realistic generated images, only focus nose, brow, hair, and ears landmarks since they can usually be
14

---

generated more realistically. Other attributes such as eyes and teeth, are often generated with more obvious artifacts like misaligned pupils or incorrectly positioned teeth, even with specific prompt design. Therefore, we ignore them during the generation phase but keep their original appearance in the generated image.
Prompt Design. For the selected attributes, we use GPT-4o to generate each facial attribute dictionary and prompt template. For each image, we randomly combine masks generate masked faces. Therefore, each masked face may contain i E 2, 3. 4} masks. Based on the generated masked face, we choose and combine templates for the corresponding attributes, then randomly select descriptive keywords from the relevant attribute dictionary to populate the templates, thereby generating the final prompt for image partial synthesis.
Image Generation. During the image generation phase, we input the generated masked face, corresponding final prompt. and pristine face into the Inpainting Generator to produce an inpainted face. For the Inpainting Generator, several widely used open-source models were selected: SDXL, SD2, and Kandinsky2.2. Furthermore, to better simulate real-world scenarios, certain forgery methods extract modified regions and overlay them onto the corresponding area of the pristine image, preserving the unedited portions of the original. These retained portions then be leveraged to interfere with forensic detectors, thereby facilitating evasion. Consequently, we included this category of partial synthesis in our dataset.
B.2 Details of Full Synthesis Pipeline
Prompt Design. With advancements generative models, particularly text-guided models that now support longer imput prompts to deliver more diverse outputs. In this context, we propose a semantic attribute dictionary, created with guidance from ChatGPT-40, specifically designed to support the generating of realistic faces across diverse real-world scenarios. And, we further devised a corresponding prompt template, (i.e., Prompt Warehouse Template, as shown in Fig. bb, c)). By encompassing diverse real-world scenarios and stylistic elements, this template is structured to simulate realistic facial characteristics across varied photographic conditions.
Image Generation. We introduce two types of fully synthesized forgery: (Fig.2() and non-existent faces (Fig (c)). Both generation approaches utilize the same prompt template with slight modifications. In the face identity insertion pipeline, the objective is to preserve the original facial identity while situating it within a new scene. This pipeline initially uses a Face Feature Extractor to capture identity
generating identity insertion
features. These identity features are then integrated with the final prompt template and input into the ID-preserving Generator to produce the synthesized image. InstantID [42] was chosen the model of choice, with all identities sourced from the He [19] facial datasets. In the generation of non non-existent faces, the diffusion model creates images of completely new faces that are unrelated any existing individual based on a given prompt. In this case, the final prompt is created by randomly selecting keywords from the semantic attribute dictionary and inserting them into the prompt template, which then serves as the sole input to the Text-to- Immee Generator. For the Text-to- Image Generator, we selected several open-source models that support extended text inputs, including Flux, SD3, Kandinsky2.2, SD2, and SDXL.
B.3 Details of Attribute Dictionary
Facial Attribute Dictionary: For fine-grained facial manipulations, the dictionary mainly includes the following attributes:
[size]: Defines the relative size of facial features.
[shape]: Specifies the contour and structure of facial features.
[color]: Describes the color characteristics of facial elements.
[styles]: Indicates the stylistic choices applied to facial features.
Idescription]: Provides a textual description of the face.
inailitioonlfeattreen Lists extra details, such as freckles, scars, wrinkles, or facial expres- sions, that add realism to the modifications.
The data source of the facial attribute dictionary is shown in Fig. B.I (a).
Semantic Attribute Dictionary: For semantic attribute dictionary, including following attributes:
15

---

better ooeer sienna
Selerbbuues seashell
straight
low
purple
Thin
Den eradidds
Smotth
pomato
Walking
comp
leaning
bright
flat
uacksl
cct
bllack
humps
(o Or
thau Blol
sitting
trenddy
Hhairstyle
sandybro
bybrom
sapphirc..
steelblue
pointed
Lon
tregray
high
voured
Skyble
asyyiinetrica
llategr
kod
lighting
pur
springgreen
beige
mmm
red
seagi
leathered
shocking
and
şilver
salmmo
Bgray
sized
Suve and loooed vala
Jacket
Floral
Diai Suibil
natural
scar
wide
narrow
sa
royalblue
pllyy
Mkunkr
eeeclaee
prominent
sung
medium-
blazer
that
stone
siunnding
arched
small
tlvesiin
Fouin
scarlet
eeeat
chic
thislle
rese
sandye
calorful
cannera
Es
Esatebluee
relressonn lusk
ring
pair
betile
arge
rubinehigh. tosnian
Tzachad wedden
saaddcbrown
plaass
snow
Aaguighon
long
striking
angle
casuall
l.
Forehoad callored
(a) Facial Attribute Dictionary
(b) Semantic Attribute Dictionary
Figure B.I: The data source of facial attribute dictionary and semantic attribute dictionary.
pthloo_ Defines the overall stylistic direction of the image. [subject]: Specifies the characteristic attributes of the subject.
[important feature]: fenauutaat_feature::: Important attributes of the subject. [more smnoreddeaais]] Further attributes the subject.
[pose_or action]: Interaction between the subject and the scene. [framing]: Indicates whether body features are included.
[lighting]: Lighting conditions in the scene.
gcamera_angles Camera shooting angle.
[camera propertieo Different parameters produce varying focus effects.
[photographer]: J: Similar to camera parameters, the photographer's choices affect the presen- tation of the image.
The data source of each attribute is shown in Fig. B.1
B.4 Examples of VLF Dataset
We selected a subset of samples from the VLF dataset and organized them by categorizing based on different generators and detailed characteristics.
Fig. B.2 illustrates the partial synthesis generated by different generators guided by the facial attribute dictionary.
Fig. B.3 illustratsss the full synthesis generated by different generators guided by the semantic attribute dictionary.
C Details of Low-Level Vision Comparison Pipeline
In this section, we elaborate on three aspects: low-level vision prompt design, low-level vision description analysis, and low-level vision bias analysis.
C.1 Design of Low-Level Vision Prompt
Due to the high fidelity of existing DM-based images, it is challenging to explore subtle differences between them and real images from a purely visual perspective. However, images generated by different generators still exhibit certain common characteristics that distinguish them from real images. We refer to these characteristics as low-level vision biases.
One key challenge is determining the specific aspects in which these low-level vision biases manifest, such as lighting, noise, or color distribution. To address this, we investigate these biases from the following 10 low-level perspectives:
16

---

SDXL
SD2
kandinsky2.22
single coRod
hair
brow
ear
nose
hair
brow
cdr
nose
hair
brow
car
nose
pristine
muli-ecited
single-edted
hair
brow
ear
nose
hoir
brow
nose
hair
brow
nose
pristin
mult-cclee ecited
Figure B.2: Examples of Partial Synthesis in VLF.
SDXL
SD2
SD3
Kandinsky2.2
Flux
Beauty
Style
Modemist
Style
Asian
Sybject
Indian Sybject
Tousled
Hair Feature
Goatee Feature
Necklace Detail
Kimono Detail
Bounced
Lighting
Edge Lighting
Figure B.3: Examples of Full Synthesis VLE.
17

---

10 Perspectives:
Are the adges sharp. blury,, or tragmented? Are the outlnees complete or imegular? Are there any areas with particularly high or low edge donsity? edges follow specific direction (e.g. horizontal, vertical, diogonalj? Provide specific examples edge and contour characteristics in diferent parts the image.
Analyze the brightness and contrast of the image in detail, Is the image overall too bright, too dark, or well halanced?? Are there any areas with distribution of
the image, Are any signs overexposure underexposure? highights shadows? ? Describe contrast variations different parts image. Provide specific examples brightness
la the image overall sharp or blury? If bluryy what type of biur is present (e motion blur, Gaussian blur, defocus blur?? Are there specilic blurred than others? Compare sharpness different regions the image. Provide specific examples of sharp and blurry
that are areas.
* What are the dominant colors, and how are they distributed across the image? Are the colors evenly distributd, or are there any anomalies like oversaturation, color distortion, color blocks? Analyze the brightness and saturation of colors in different parts of the image. Provide cheracteristice
specific examples
* Are there any geometric distortions, such as stretching, warping, or asymmetry? y Are there any compression artifacts, like blocky petterns jagged edges? Analyze the symmetry of the image. Are there unnatural deformatione? Provide specific examples of distortinns and anomalies different parts of image.
is the image nolsy? it so, what type of noise is present (e. g. Gaussian noise, salt-and-pepper nolse, quantizttio nolee)? Dascrbbe the intensity and dietribuiie ibuiinn of noise across the image. Analyze the texture complexity it uniform or varied? Are there any directional patterns in the texture?? Provide specilic examples of noise and texture cheracteristic in different parts the image.
they appear unnatural? Compare the highlights and shadows in different parts of the image. Are there any lighting anomalies, such as averly bright or dark areas? Provide spocticc examples of lighting and shadow charactensties.
lighting uniform uneven?
the shadows natural and consistent with lighting direction.
well-del rell-defined.coo details are perticularly sharp or blurry? Check for any loss of detail in specilic regions. Provide specilic examples of detail and resolution characteristics
Are details
they lacking? How
clarity of image?
any areas where
'Ara there any compression artifacts, sharpening artifacts, color artifacts? Describe the location and intensity of these artifacts. there any si signs unnatural image processing? Idtenily any abnormal regions in image and describe their charactorisiics. Provide specific
examples of artifacts and anomalies.
dynamic range? the lone mepping netural, does appear unnatural? Check how well the highights shedows retain details. Are there any mapping anomalies, as clipped highlights crushed shadows? Provide specific examplee of dynamic range and tone mapping charactertstics
Does the image have high
Limit
Do not use relative terms like more, less, etc.
Template
Compare low-lovel visual features of imago. and image. 2. Focus an the following aspects for each image: [[Porspatives(iik]] Format your response as follows: image_1: [description) - image.2: [description) But note: (imiif).
The details of the prompt template employed within the low-level vision comparison pipeline for the full synthesis.
Figure C.I:
Edges and contours.
Brightness and contrast.
Blurriness and sharpness.
Color distribution.
Distortions and anomalies.
Noise and texture.
Lighting and shadows.
Level of detail and resolution.
Artifacts and anomalies.
Dynamic range and tone mapping.
Based on these ten perspectives, we designed corresponding prompt templates. The low-level vision comparison prompt for full synthesis is shown in Fig. C.I
Specifically, for partial synthesis, since we only need to explore the low-level differences between the synthesized regions and their corresponding real counterparts, we incorporate region-specific nouns as parameters in the prompt. The low-level vision comparison prompt for partially synthesized images is illustrated in Fig. C.2
C.2 Analysis of LoveLevel Vision Description
To mitigate survivor bias and potential discrepancies in low-level vision bias across different genera- tors, for each category of full synthesis samples, we generated 12,000 differential descriptions from 10 distinct low-level vision perspectives, resulting in a total of 120,000 differential descriptions per full synthesis subset. Subsequently, we selected compound nouns from the 12,000 descriptions for each perspective and calculated their frequency proportions. In addition, it is necessary to generate corresponding real-image low-level vision comparison descriptions based on the synthesis sample categories and subsequently calculate the frequency proportions of compound nouns.
18

---

10 Perspectives:
Describe the edges and contours in the (rogion) of the image detail, Are the edges sharp, blurry, or tragmented? Are the outlines complete edges follow a specific direction (e.g.
edge density the (ragian)? horizontal, vertical, diggnaal?? Provide specific examples of edge and contour characteristics the (regian).
imegular? Are there any areas parriuularly high
Analyze the trightness and contrast in the (region) of the image detail. Is the (rogion) everall too bright, too dark, or well balanced? Are (region),
there any with extreme highighhts
the (region)? Describe overexpooure or underexposure the (regionj? Provide specitic examples brightness and contrast variations the (region).
distribution contrast
any signs
Eveluate the blurrinees and sharpness of the (region) in the image in detail. ls the (region) overall sharp or blury? If biurry, what type of blur le present (e.g. motion blur, Gaussian blur, defocus blur)? Are there specific areas within the (region) that
blurred than others? Compare the sharpness of differant parts the (region). Provide specilic examples sharp and blurry areas in (ragion).
Describe the color distribution in the (region) of the image in dotail What are the dominant colors in the (ragion), and how are they distributed? Are the colors evenly distributed in the (region), or are there any anomalles like oversaturation, color distortion, or color blocks? Analyze the the (region). Provide specific examples color characteristice Check for any diatortions or anomalles in the (region) of the image in detail, Are there any geometric distortions the (region), such as stretching, warping, or asymmetry? Are there any compression artifacts in the fregion), like blocky petterns or jagged edges? ? Analyze the symmetry the (ragion). Are there any unnatural deformations the (ragion)? Provide spectfic examples distorions and anomalies in (region).
brightness and saturation
anomalies the (region):
*
'Anelyze the noise and texture in the (region) of the image in detail le the (region) nciey? If so, what type of noise is present (e. Gaussian noise, salt-and-pepper noise, quantizstion nolse)? Describe the intensity and distriuttion of noise across the (region). Analyze the lexture texture the (region)? Provide specific examples
complexity in (region)-is -ii uniform or varied? Are there any directionll patterss
naise and texture characteristics the (region).
Analyze the lighting and shadows in the (region) of the image detail. Is the lighting in the (region) uniform uneven? Are the shadows in the (region) natural and consistent with lighting direction, or they appear unnatural? Compare the highighhts and shadows cifferent parts the (region): Are there any lightng anomelles in the (region), such as overly bright or derk sress? Provide specific examples of lighting and shadow characteristics in the (region)
Evaluato the loval of detail and resolution the (ragion) of the image detail. Are the details the (region) and wall woll-defindd, lacking? How does resolution affect the clarity of (region)? Are there any areas the (region) where details are particularly sharp or blury? Chesk for any loss of detail in specific parts of the (region). Provide specific examples of detail and resolution characteristice in the (region).
they
Are any compression artifacts, sharpening artifacts, color artifacts the (ragion)? Describe the location and intensity of these artifacts the (region). Are there any signs of unnatural Image processing in the (region)? Identity any abnormal rogions within the (rogion) and describe their charactoristics. Provide specific examples of artifacts and anomalies
Check any artifacts or anomalies
(region)
image
(region).
* Analyze the dynamic range and tone mapping the (region) of the image in detail. Does the (region) have high or low dynamic range? the tone mapping in the (region) natural, or does it appear unnelural? Check how well the highlights and shadows in the (region) retain delails. Are any mapping anomalles in (region), such alipeed highlights crushed shadows? Provide specific examples dynamic range and tone mapping charactoristies the (region).
Limit
Do not relative like more, less, etc.
Template
Compare the low-loval visual features of image_1 I and image. 2. Focus the following aspocts for each image: [[Porsoattios(iik]]. Format your response as follows: image_1: 1: [description). - image_2: [description). But note: (lrmif).
The details of the prompt template employed within the low-level vision comparison pipeline for the partial synthesis.
Figure C.2:
A simple example of the full synthesis samples in the SDXL category is illustrated in Fig. This image illustrate the proportion of compound nouns at the low-level vision perspective for edges and contours, with the upper-left image displaying the noun proportion ranking for synthesis, and the lower-left image showing the noun proportion ranking for real.
To identify the nouns that may serve as judgment biases for MLLMs, we computed the maximum differences in noun proportions between real and synthesis images, with the results presented in the right half of Fig. C.3 It is evident that, compared to the SDXL category of synthesis images, real images exhibit 20% proportion of 'smooth outlines', 'well-defined contours', and 'noticeable fragmentation', while over 70% display 'low edge densities'. In contrast, synthesis images have less than 20% exhibiting 'low edge densities, but a significantly higher proportion of 'high edge densities', ; alongside level visual features, such as 'irregular outlines' and 'lower edge density", not present in real images.
Therefore, for synthesis images in SDXL category, the judgment biases derived from the low-level vision analysis at the edges and contours perspective are as follows: they are likely to exhibit features such as irregular outlines and lower edge density, and, in comparison to real images, they are more prone to display high edge density.
simple example of the partial synthesis samples in the SDXL category is illustrated in Fig. and Fig, C.55 Notably, for partial synthesis images, we generated differential descriptions from 10
distinct low-level vision perspectives for each of four selected regions, as outlined in Sec. B. For
each category of partial synthesis images, a total of 480,000 differential descriptions were generated.
We observed that even for partial synthesis derived from the same real, differences in the modified
region led to slight differences in low-level vision bias. Therefore, in Fig. C.4 and Fig. C.5 we
19

---

Features
top LU reebures with Largest Parcersage Iferences
smooth outlines
Imegular outlines
well dell-defieed contours
noticeable Tregmentation
lower edge density
Real Low-Lovel Features
noticeable blur high edge density
low edge densities
higher edge density
low edge density
200%
Percenttage
Figure C.3: The low-level vision bias obtained after processing full synthesis images through the low-level comparison pipeline.
Top Features with Largest Parcentage Offerences
different parts
noticeable differences left side
light source
realistic look
Real Low Level Features
front-left side
noticeable shadows
significant lighting anomalles
areas-
overty bright
Figure C.4: The low-level vision bias obtained after processing hair-tampered images through the low-level comparison pipeline.
present the ranked data for two types of regions with significant size differences- hair (i.e., Fig. and brows (i.c., Fig. C.5) (low-level vision perspective: lighting and shadows).
C.4
can be observed that even within the same low-level vision aspect, judgment bias varies significantly across different regions. For instance, in Fig. C.4 real images exhibit more pronounced characteristics in lighting and shadows perspective, such as "overly bright areas', 'dark regions', and 'significant lighting anomalies' when compared to manipulated images with hair modifications. However, in Fig. C,5 real images compared to eyebrow-edited fake images display 'higher contrast', a 'more natural appearance' and 'multiple lighting -ngles-feeatue that not observed in the manipulated images.
Low-Lavel Festures
Top Feetures with Largee Percentage Bifernoosss
significant lighting anomalies
multiple engles
consistent appcarance
harsh contrests
natural look
Real Lav-Lavel Features
dark ereas
overly bright
different parts
signiticant
noticeable shedows
Figure C.5: The low-level vision bias obtained after processing brow-tampered images through the low-level comparison pipeline.
20

---

C.3 Analysis of Low-Level Vision Bias
During the noun selection process, issues such as synonymy, ambiguous terms, and minimal propor- tions differences in noun characteristics between real and fake images often arise. Consequently, not every feature with a large proportional difference should be considered a judgment bias. For example, "low edge density" and 'low edge densities" in Fig. C.3 as well 'different parts" and 'left side' Fig. C.4 illustrate such cases. Therefore, manual filtering required identify distinguishing low-level vision features that can effectively differentiate real from fake images.
In Fig. C.6 demonstrate the process of applying the low-level vision comparison pipeline to fake image generated by SDXL. The pipeline extracts low-level vision biases, which are then manually filtered obtain the final judgment bias descriptors.
D Full Experiment Result of VLF
This section provides detailed description of all aspects involved in the experimental phase of the VLForgery framework, including the quantification of textual outputs, error analysis, additional metric evaluations, and extended experiments.
D.1 Quantitative Analysis of Results
Kasttsky1.2%
SD89,
Rack]
PS
1%
P5
P5
P5
P5
Vataaii
0.00
0003
0099
000
000
0.00
0/0)
010
0.00
000
3.52
3,01
1.0
333
Table D.I: Error study.
Result quantification: We employed regular expressions to filter MLLM outputs across multiple tasks. During testing, while the answers seemed correct, they did not always match the groundtruth due to issues like case mismatches (e-g., "True' vs. 'true', 'Fake" vs. 'fake'). Although synonyms did occur (e.g., 'fake' vs. 'false"), such cases were extremely rare. To address this, we created synonym pool and preprocessed the results. the model's output matched an entry the synonym pool, we considered it correct (e.g.. 'false" was treated as 'fake").
Error Analysis: To address the inconsistency in detection results, analyzed two key issues: 1. Variant A: Mismatch between authenticity judgment and forgery type (e.g., detected as 'real' but labeled as 'full synthesis") ). Variant B: Mismatch between forgery type and falsified region (e.g.. labeled as 'full synthesis' but falsified region is 'hair'). )) The results are shown in Tab T.b D.I
D.2 Experimental Results Supplement
Comparative Analysis of Single and Multi-edited Images. Given that multi-edited faces may disturb the detector's s judgment, we divided the test data for each region into single-edited and multi-edited groups. We fine-tuned Vororgyy on SDXL partial synthesis and subsequently tested on both single-edited and multi-edite faces across four regions' test sets. As shown in Fig. D.I we visualized the localization results of these two groups for four regions, respectively.
The results reveal that, although localization difficulty varies across regions, single-edit images are generally easier to locate compared to multi-edit images.
Additional metrics for evaluation. We introduce two additional metrics, Fl and Precision, to comprehensively evaluate the detection performance of different models. Tab. D.2 illustrates the precision performance of different models across various subset test sets, while Tab. D.3 demonstrates their F1 performance on the same subsets.
E
Additional Information on VLForgery
In this section, we present all additional details related to the VL.orrgrr framework that were not covered previous sections. These include the prompt design, question formulation, and inference result examples used in the fine-tuning and inference modules of MLLMs; the full low-level vision
21

---

lower resolution
"Artifacts:"
obvious
artifacts
Most fake images lack visible compression artifacts, sharpening artifacts, color artifacts, while real Images may occasionally show these. Fake images often exhibit higher image quality.
noticesble color artifacts
smouth gredients
sharpening artract
mirimal compression artillcts
clainss sigro.
overall quality
A small number of fake images may show unnatural semantic signs.
visble cumpression artifiets
unnatural Image procossing
200%
00%
49%
30%
20%
40%
100%
Feetu Langest Persenaage
noticeable defecus blu
"Blurriness Sharpness:"
Most real and fake images show little visible blur.
Specific araas
central part
images may still have blurry
areas.
signifeant blrr
images
more detailed skin textures compared to fake
skin texture
mutioaable aruus
few fake images may show unfocused blur.
100%
00%
30%
25%
60%
50%
300%
Fosturea fiferroo
natural light
"Brightness and Contrast:"
balanced brightness
Most real images have balanced
noticeable repossuee
brightness and contrast, while some fake images may show imbalances.
diferent parts
exposure
Real images usually
neutral
overal tone
overall tones, while some fake images may
sigificant
varied
moderate st brightness
Most images have balanced exposure, but some fake images show uneven exposure.
high leve
highlights
100%
59%
60%
45%
20%
40%
60%
55%
100%
Fouturea
Difenrness
visblc anomaliss
Both real and take images generally have uniform distribution, fake images may be slightly
"Color Distribution:"
glow
dominant
uniform.
sigrificc it co distortions
-A very small number take images may have a dominant earthy tone, which is rare in real images.
overseturation
earthy tones
neutral
Real which show soft fake images.
around
signilicant color blocks
warm tones
Real images are more likely have oversaturated colors or natural tones compared to take images.
anomalles
100%
48%
20%
0%
20%
40%
oth
80%
200%
"entar with Percoraage Offersnces
black aal---ndd-aa white photograph
"Distortions Anomalies:"* Most real images are natural close-up portraits under natural light.
distortions
hlerky patters
overall darlty
lighting
unnatural meformations
Real images often have features captured at low resolution, which rare in fake images.
resolution
close-up photograph
facial featunes
portralt
100% 60%
65%
40%
20%
0%
20%
100%
Figure C.6: The process of manually extracting final judgment bias descriptors from low-level vision bias
kandinskk22.2
SD2
SDXL
Accuracy
64.77
K11
hair
brow
ear
nose
hair
brow
ear
nose
hair
brow
ear
nose
single-edited
Figure D.1: Visualization of model's localization accuracy single-edited and multi-edited faces. The Large ACC area means better localization capability.
22

---

Train Train-SDXL SDXL1
SD21
Kandinsky2..2
Method
SD31
Fluxt
InstantID
Avgt
PS
FS
PS
FS
PS
FS
Resnet-50
90.77
89.82
76.88
89.43
94.24
89.6
88.9
89.27
77.71
87.4
Xception
79.83
64.69
69.94
64.14
84.9
64.62
64.17
64.47
43.19
66.66
SPSL
90.1
90.08
57.95
89.8
93.67
90.07
89.8
90.02
78.26
85.53
DRCT
86.7
78.02
86.87
77.73
91.83
78.11
70.2
67.84
59.55
77.38
NPR
65.52
58.43
31.27
58.19
75.84
58.33
58,01
58.38
36.8
55.64
Ciipping Adapter
53.45
49.96
50.42
48.54
70.37
49.58
49.63
49.47
28.78
50.02
CLiPping LP
66.4
50.25
63.48
49.54
73.64
50.16
49.72
50.01
29.58
$3.64
SRM
69.34
54.66
31.96
54.33
78.97
54.56
54.28
54.21
32.78
53,9
F3Net
91.40
81.91
87.08
81.84
92.14
81.91
81.82
81.92
65.34
82.81
GramNet
87.42
86.52
74.38
86.16
90.77
86.48
85.71
86.44
70.29
83.80
SAFE
61.92
42.73
60.89
42.58
61.85
41.8
44.57
34.15
21.3
45.75
CNNspot
91.4
92.59
78.5
90.3
91.59
90.43
83.59
89.93
79.64
87.55
VLForgery
91.41
81.04
89.98
81.03
93.12
81.03
81.04
81.03
63.94
82.62
Table D.2: Precision.
Train-
SD21
Kandinsky2.1
Method
SD31
Fluxt
Instant
Avgt
PS
FS
PS
FS
PS
FS
Resnet-50
50.61
94.52
20.56
92.45
72.11
93.32
89.73
91.59
85.32
76.69
Xception
73.55
78.56
50.98
77.4
90.5
78.41
77.47
78.1
60.25
73.91
SPSL
46.81
94.77
8.85
93.22
66.39
94.7
93.22
94.42
85.86
75.36
DRCT
69.25
87.43
69.93
86.58
95.43
87.68
68.02
63.15
74.34
77.98
NPR
51.68
73.76
15.41
73.29
73.08
73.57
72.95
73.65
53.71
62.34
CLIPping Adapter
43.23
66.61
39.25
64.1
72.63
65.93
66.02
65.73
44.4
58.66
CLIPping L.
64.01
66.88
58.55
65.62
79.9
66.73
65.94
66.46
45.63
64.41
SRM
63.87
70.67
17.76
70.07
87.6
70.5
69.97
69.85
49.01
63.26
F3Net
81.84
90.02
61.02
89.79
86.58
90.01
89.73
90.05
78.99
84.23
GramNet
49.13
92.77
23.96
91.23
63.11
89.4
92.41
78.33
74.77
SAFE
59.07
56.21
57.28
55.94
58.96
$4.66
59.27
42.73
32.81
$2.99
CNNspot
40.85
95.18
16.21
80.73
41.67
81.47
54.06
78.72
76.5
62.82
VLForgery
84.53
89.52
76.36
89.51
96.42
89.51
89.52
89.51
77.87
86.97
Table D.3: F1.
bias results from the low-level vision comparison pipeline; as well comparative examples of inference results from other multimodal language models.
E.1 Details of the Fine-Tuning and Inference Modules
This section describes the fine-tuning and inference module and the various templates required for EkCot generation.
Fig. E. outlines the specific contents of the general rule template involved in the description generation module. The templates incorporate additional knowledge, such as ground truth and low-level vision bias, represented by specific placeholders. For example, 'detection result' indicates whether the image is real or fake, 'text prompt' represents the prompt used during the image generation process, and judgmentbiass denotes the low-level vision judgment bias knowledge required for analysis.
Fig. E.2 illustrates all the question formats randomly generated by GPT-4o for the final fine-tuning dataset.
E.2
Comparison of the Inference Result with Other MILMS
We randomly selected three representative samples of forged images, each corresponding to distinct type of forgery, and demonstrated the reasoning outcomes generated by our framework in comparison to those produced by other open-source Multimodal Large Language Models (MLLMs). Fig. E.3 presents the analysis results generated by different reasoning models for a synthetic image created through identity insertion. Fig. E.4 presents the analysis results generated by different reasoning models for synthetic image created through sdxl model. Fig. E.5 presents the analysis results
23

---

You expert in detecting and analyzing face forgeries.
categorize detection tasks into three primary aspects:
1. Detecting the authenticity of faces.
...deniffling falsified regions.
Tracing the generation process of forgery faces.
We also categorize forgery datasets into five types of forgerys
1.Facial fine-grained semantic editing.
I.Identity transformation.
3. Non-exissent faces.
You will be provided with the following informations:
A text prompt used during the generation process of the image: (text_promp))
The detection result of the image: (detection (detection_result) result)
The forgery type of the image:
The falsified region of the image: [ebsified_region
The method used to generate the forgery: {forgery_method)
need your help to determine whether an image is AT-generated or real. have provide you an image and tell you all the groundtruth and informations about this image, but am unsure what specific features make it fake or real Based my research, here some key differences between real and Al-generated Images that you should analyze: (judgmentbias)
**Output format:**
1. 1***etection Result**: real/fake.
**Foggery analysis**; Based on the key differences have provided you. analyze the image provide and explain why might be fake or real. Focus on identifying specific features or anomalies that align with the characteristics of Al-generated images as described above. Provide detailed explanation of your findings.
Fo**Fyy Type**; facial fine-grained semantic editing//dentity transformatio/noo--essttttt faces/none.
4. Falsiiiified Region**: hairr/ar//oss/brww/....f.flll synttssssnn......snnnnnnee.. Forogeyyy Method**:
**Rules**;
1. Do bold for any titles in the output.
2. Generate answers strictly within the word limit.
Figure E.I: Specific details of the General Rule Template used in the forgery description module.
generated by different reasoning models for partial synthetic image created through sdxl-inpainting model.
24

---

Does this image show genuina artifcially created human face? What clues help docide?
supports your Do you believe this face in the image is natural synthelically created? What leeds you to that conclusion?
depicted
image
generated? What
Looking this face, ls an authentic human face of digita imiaaion?? What observations influence your judgment?
Can you the
image unmodilied manipuisted? Explain your reasoning.
details you consider?
this true representation of human facs maripulatod
Would you sey this image conteins a real human face or an artificial one? What factors helped you determine that?
This image festures human face. it authentic digitally created? What makes you think so?
image ls genuine doctored? Is the human face in this image unedited or attered? What details support your conclusion?
you think the
you find?
Does this image present real human face computer generated one? What observations led you to that assessment?
human Would you classify the face in this image as real or synthetic? What evidence do you base your answer on?
this image,
natural modilied? What makes you that concuuiio?
la this an actual humen face or digitally constructed one? Describe the signs that influenced your evaluation.
Does this face look like netural one, does eppear be digitaly altered? What incicators made you think thal?
Can you determine the human face Is the human face shown ganuine, or does have signs of digital oditing? What observations led you to your conclusion?
authentic or fabricated?
details you analyzs?
Does this
feature image is original you think the
human face manipulated one? What evidence supports your interpretation?
been digitally
your observations.
Is the human face depicted here real or product of digital manipulatio?? What clues indicste your cholcs?
The image contains human face. Would you it's unedited or moditied? What factors influenced your judgment?
this faithful dopiction
face, does show signs of atteration? What datails influenced your view?
Is the face this image authentic or tampered wit?? What observations led you to your conclusion?
Does this appeer to be genuine human face or one that's digitaly fabricated? What cues influenced your perspective?
Would you this
represents netural human face, does seem artificia?? What details you consider?
Is human face in image untouched or altered? What leads you your judgment?
The image depicts a human face. Do you bellove it's authentic or dighially ahtord? Explain your reasoning.
this real human face, has been digitally altered? What makes What
think that?
Looking
image. genuine Do you believe this face looks authentic manipulated? What ressoning backs up your belief?
you to your concusion?
Doss the image depict real human face, does show evidence of editing? What clues suggest this to you?
Can you idantily whether
naturally occurring generated? What you Doss this image show en unmodilled human face or one that's digitally constructed? What observations support your claim?
decision?
Is the face this mmge genuine, or does appear to be digitally produced? What makes you think so?
you this natural humen lace synthelic one? What evidence backe your impression?
Does the human face in this image seem real, or does give signs digita creation? What led you to conclude that?
Would you consider this human lace real or maripulated? What details in the image suggest this to you?
Does the human face look natural altored? What observations support your view?
you believe the image presents
does have features digital editing?
support this?
This a image of human face. Can you determine il it's real or artificial? What supports your answer?
Does the image contain natural human face ane that is artificially generated? What loads believe that?
this digitaly
What avidence you rely on?
image appear genuine manipulated? Can you determine it this human face is real or digitally mocilled? What indicators did you examine?
Does face shown
factors infuence your decision?
A human face appears in this image. Do you think it's real altered? What makes you so?
There's human
or This image displays human face. Ia genuine or manipulated? What leads you to that dedision?
What evidence supports your conclusion?
A human face present ths image. is authentic or doctored? Provide your reasoning.
image shows human Do you
forgery? observatione support your enswer?
Figure E.2: Randomly generated question descriptions using ChatGPT-4o.
25

---

There's human face in this image.
natural ?ahe? evidence supports your
The image natural but mannequin head. The evidence supports conciusion includes following:
The skin on the face appears smooth and even, which not typical of human skin. Human sián has natural wrinides, pones, and impertactions that are absent this image.
Unnatural sián texturo:
Lack of facial hair:
head typically have
have facial including eyebrows, eyelashes, especially as people age.
inconsistent eye shape: The eyes the mennequin head are perfectly round and symmetrical, which is not characteen claracterittcc humen eyes. Human eyes are often siihtly asymmetrical and have a more irregular
shape.
unffom which eyes mannequin head shades, including any expression, green, human face. Human faces ofton display rango of amations through facial expressions
Unnatural
eyes. eyee manrequin
rerge
expression:
unusual
0 Llama
The foco in the image appears to be combination natural and artifccal elemants. Hare are some placos evidence that support this concusion:
Facial Features: The lace realetic facial festures such texture, wrnkees, a naturaHHokking mustache. Thesa teatures typically
lighting shadows on are consistent with natural lighting, suggeste that the face is not entirely computer-gene rerated. 3. Fieflections in Grasees: The glasees reflect the lighte the background, which is common festure in real- life photography.
Lighting Shadows:
Qwen
The human the image is natural, as it a close-up of a man with a beard and glesses. The evidence supporting this conclusion the presence the man's faciel festures, such his beerd, glasses. the
eyes mouth. features typical are not indicative a fale or artificial representation.
LLaVA
Detection Result: fake
Forgery Analysis: The image exhibits several characteristics that common in Al- generated images, such as a lack visible compression artifacts, sharpening artifacts, color artifacts, and higher image quality compared real images. The image has blurred background, which is common feature in fake images. The lighting in the image also unnatural, bright
typical real-life photography. Furthermore, the image has uniform color distribution, which is a characteristic of Al- generated images. Overall, image appears forgery created using method as InstantID.
Illumination
Forgery Type: identity insertion Falsifled Region: full synthesis
Forgery Method: InstantID
VLFrorgory
Figure E.3: Comparison of responses to the full synthesis between other widely used MLLMs and the VI.Forgery framework. The sample shown is an identity insertion face generated by the Instantll) model.
26

---

There's human this image. Is natural or lake ?What evidence supports your conclusion?
The human face the image appears be natural, but actually tattoo. The evidence supporting this conclusion the presence of a tattoo on the forehead, which is common location tattoos. Additionally, overall appearance of the face and surrounding skin tone suggest that it is a natural face, rather than fake one
O Llama
The human face image appears be natural. following evidence supports this conclusion'rinn Skin Texture and Lighting: The skin texture looks realistic, with natural lighting that highlights the facial features without any obvious signs of artificial enhancement. 2. Facial Features: The facial features, such the eyes, nose, and lips, appear be well- defined and consistent with natural human anatomy.
3. and Makeup:
makeup look natural and well- eillleeecuted, without any exaggerated or artificial elements.
4. "Overall Appearance": The overall appearance of the face does not exhibit any signs of digital manipulation, such
Qwen
The face the image natural, as woman's face with her eyes, and mouth clearly visible. The evidence supporting this conclusion the presence of the woman's facial features, recognizable.
image not any signs being manipulated altered, such as an unnatural skin texture or overly smooth appearance, The woman's face appears be genuine representation human face, without any indication being artificial digitally manipulated.
LLavA
Detection Result:fake
Forgery Analysis: The image common in Al-generated images. Firstly, the image has a high level of detail and sharpness, which
to be fake due to several characteristics that
for images. The texture uniform appearance that not typically seen in real people. Additionally, the lighting in the image overly dramatic, with strong highlights and shadows that are not consistent with natural lighting. The overall color palette ls also somewhat muted, which images, Furthermore, the image has a slightly blurred background, which is not consistent with the sharp focus the subject's face. Finally, image has slightly unnatural composition, with the subject's head tilted at unusual angle and the camera angle that is not typical of real photographs. These features, combined with the overall aesthetic the image, suggest that is likely take image generated using
unnatural, with
and
typical of real
Forgery Type: non-existent faces
Falsifled Reglon: synthesis
Forgery Method: flux
VLFrorgery
Figure E.4: Comparison of responses to the full synthesis between other widely used MLLMs and the Vorgeryy framework. The sample shown is non-existent face generated by the flux model.
27

---

There's human image. Is natural or PWhat evidence supports your conclusion?
The image depicts human face, specilically women's with a amile. The woman's lace appears to be natural, and there evidence suggest that it fake. The following reasons support this conclusion: mouth, all
festures: The woman's facial features, present and appear to be in proportion one another. There is indication of any artificial enhancements or alterations.2. Skin texture: The skin texture the woman's face appears smooth natural, hair is also natural, with no visible signs of artificial coloring or styling.4. Eyes: The woman's visible signs
eyes,
visible signs artificial aging wrinkles.3. Hair: woman's
eyes are woman's is smile appears to be genuine and natural, with no visible signs of artificial teeth dental work. Overall, based on the evidence presented the image, appears that natural not fake
enhancement alteration.5 5. Smile:
0 Llama
'The face in the image appears to be natural. The lighting, skin texture, and overall appearance consistent human There visible signs digital manipulation artificial features that would suggest otherwise.
Qwen
human face image is natural, evidence supporting this conclusion is the woman's facial features, such ass her eyes, nose, and mouth, which are clearly visible and appear to in propertion to her face. Additionally, the woman's expression and overall appearance convey genuine, natural smile, which consistent with a real human face.
woman smiling posing picture.
Delection The image appears to be fake due to several anomalies. Firstly, the hair Forgery Analysis:
color too uniform and lacks natural varistions, which common trait in Al-generated images. Additionally,
lacks natural imperfections found in real hair. Furthermore, the lighting and shadows on the hair are not consistent with real-world lighling conditions, suggesting appearance of the image, including the hair, is too perfect and lacks the subtle nuances found in real people's faces.
overly
image
manipulated.
facial Forgery Type: facial fine-grained semantic editing Falsified Region: hair
Forgery Method: SDXL
VLFrorgary
Figure E.5: Comparison of responses to the partial synthesis hetween other widely used MILMss and the VorForyy framework. The sample shown is hair-edited face generated by the sdxl model.
28

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Refer to Sec. A.1
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [TODO]

---

Justification: [TODO]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Refer Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is a new model (e.g., large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
30

---

Answer: [TODO]
Justification: [TODO]
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cpipp/.ccppullc//uuuii wwwtii/gguie///eeeeceoocc CoodeSubeiiie ssi wsionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Refer to Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Refer to Sec.
D
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
31

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Refer Sec. 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httpp::///euripsss wwwpeeuww.....uuu..eeee httppppccccpppp.ctttttccccctteed
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Refer Sec.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
32

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [TODO]
Justification: [TODO]
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee whw.ree.wthhhe....c.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
33

---

Answer: [Yes]
Justification: Refer to Sec.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [TODO]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [TODO]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [TODO]
34

---

Justification: [TODO]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy htteas////eerrpppoccc///o/rreers for what should or should not be described.
35


Reviews:
[{"review_id": "hVXZh9RT3T", "reviewer": "Reviewer_ZgwX", "summary": "The paper introduces VLForgery, a new triad framework for AI-generated face forensics that integrates Multimodal Large Language Models (MLLMs) to perform three key tasks: detection, localization, and attribution of facial forgeries. \nUnlike traditional binary detectors, VLForgery provides fine-grained analysis by identifying manipulated regions and attributing them to specific diffusion models. Central to this framework, authors introduce the VLF dataset, a large, diverse collection of partially and fully synthesized facial images generated using various diffusion models. The authors also propose EkCot (Extrinsic knowledge-guided Chain-of-thought), a method that enhances MLLMs' reasoning by incorporating low-level visual discrepancies and generation metadata. \nExtensive experiments demonstrate that VLForgery outperforms state-of-the-art methods in detection accuracy, localization precision, and attribution reliability, especially in challenging partial synthesis scenarios.", "questions": "The paper demonstrates strong performance on the VLF dataset, but how well does VLForgery generalize to real-world or previously unseen forgery methods, especially traditional face-swapping Deepfakes?\nYou can expand a bit more your future work by providing additional experiments or discussion on cross-domain generalization—e.g., training on VLF and testing on external datasets like Celeb-DF or FaceForensics++. This would help assess the robustness of the model beyond the curated VLF dataset. It could be a demonstration of generalization to real-world or unseen forgery types would significantly strengthenyour work.\n\nAttribution accuracy for partial synthesis is notably lower than for full synthesis (e.g., <70%). What are the main causes of this gap, and how might the model be improved to address it?\nConsider providing a breakdown of attribution errors by region or generator, and discuss whether the low performance is due to insufficient signal, dataset imbalance, or model limitations. Suggestions for architectural or training improvements would be valuable.", "limitations": "Based on the limitations discussed in Section A.1 of the paper, \nWhat strategies can be developed to enhance cross-model generalization, particularly for detecting forgeries generated by previously unseen or novel diffusion models and traditional face-swapping techniques?\nHow can the VLF dataset be expanded or diversified to better capture the semantic variability introduced by different generators and prompts, and how does this variability affect detection and attribution performance?\nIt could be nice to read some lines concerning these points.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Here is a list of identified strengths:\n- The proposed integration of MLLMs with structured reasoning (EkCot).\n- The VLF dataset is comprehensive, covering multiple synthesis types and generators.\n- The framework achieves state-of-the-art performance across multiple forensic tasks.\n- The use of natural language for localization enhances interpretability and usability.\n- The paper includes thorough ablation studies and qualitative comparisons.\nsome weaknesses are:\n- Generalization to unseen generators and real-world forgeries remains an open opportunity, especially for partial synthesis.\n- The reliance on handcrafted low-level vision biases and manual filtering introduces subjectivity.\n- The localization accuracy for small regions (e.g., ears) is still relatively low.\n- The framework’s performance on traditional Deepfakes is not evaluated; it could limit its broader applicability.\n- The computational cost of training and inference with large MLLMs is not deeply discussed, it is an open area.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your recognition of our work and the paper’s formatting quality. We appreciate your positive feedback.\n\n## Response to Weakness 1: Generalization to Unseen Generators and Real-World Forgeries\n\nWe appreciate the reviewer’s observation. Indeed, our motivation in constructing the VLF dataset was driven by the limitations we observed in existing face forgery datasets at the time, particularly with respect to partial synthesis using diffusion models. Most available datasets had limited partial forgeries and were often of **low visual quality**, largely due to the lack of carefully crafted prompts tailored to face synthesis.\n\nFor instance, we examined the dataset from [1] and found that many generated faces exhibited significant distortion and artifacts. In response, we developed a set of long, diverse, and high-quality prompt templates—partially inspired by observations from a YouTube channel (unfortunately, we are unable to disclose this due to policy restrictions). As shown in Section B.4, our generated face images achieve notably higher visual fidelity.\n\n- [1] Diffusion Facial Forgery Detection\n\nWe acknowledge that generalization to unseen generators and real-world forgeriesis an important open direction. Given the emergence of newer datasets and generators in recent months, we plan to conduct broader generalization studies in future work. However, due to limited computational resources and time constraints, a full-scale evaluation on all unseen generators and real-world samples could not be completed for this submission.\n\n\n## Response to Weakness 2: Subjectivity in Handcrafted Low-Level Vision Biases\n\nWe acknowledge that this approach inevitably introduces a degree of subjectivity, especially in the filtering and bias selection process.\n\nInspired by prior work [2], which explores specialized models for low-level vision comparison (e.g., image quality assessment or structural similarity), we believe a promising future direction would be to incorporate multi-expert models or objective evaluation networks to generate more reliable and less subjective low-level vision biases.\n\n- [2] Towards Open-ended Visual Quality Comparison\n\nWe recognize that this would be a significantly more complex engineering effort, but we see it as a valuable path forward, and we are committed to further exploring this direction in future work.\n\n## Response to Weakness 3: Low Localization Accuracy for Small Regions\n\nTo investigate why localization accuracy remains low for small regions (e.g., ears), we conducted a detailed analysis on the partially manipulated samples in our dataset. As shown in Figure, we calculated the average proportion of each manipulated region in both the training and testing sets. The distributions are nearly identical, indicating no significant bias between them. For example, **hair** has the largest average proportion (34.50%), while **eyebrows** are the least represented (0.86%).\n\n| Region | Brow | Ear | Hair | Nose |\n|--------|-----|-------|------|------|\n| Train Set (%) | 0.86 | 1.88 | 34.50 | 2.03 |\n| Test Set (%) | 0.92 | 1.93 | 31.74 | 2.09 |\n\nWe further compared a variety of state-of-the-art manipulation localization methods and their performances across different regions, especially focusing on smaller areas like brow. Here are selected results for hair/brow ACC values:\n\n| Type| Method | SD1.5(%)↑ train-hair |  SD1.5(%)↑ brow | FLUX(%)↑ train-hair |  FLUX(%)↑ brow |\n|-------------|----------------------|----------------------|----------------|----------------|---------------------|\n| Naive       | Resnet-50 | 0.95            | 0.66      | 0.99           | 0.36     |\n| Naive       | Xception | 0.98   | 0.63 | 0.99    |  0.17    |\n| Naive       | PSCC-Net | 0.82        |  0.70    | 1.00         | 0.98    |\n| Frequency   | SPSL| 0.98 | 0.77  | 0.98     | 0.35   |\n| Frequency   | F3Net | 0.98            | 0.62  | 0.99           | 0.28    |\n| Frequency   | SRM | 1.00        | 0.99  | 1.00       |  0.86    |\n| Frequency   | NPR | 0.97       | 0.84  | 0.90           | 0.81    |\n| Frequency   | MVSS-Net | 0.89           | 0.88    | 0.96       | 0.95     |\n| Frequency   | MMFusion | 0.99     | 0.94  | 0.99         | 0.12  |\n| VLM-Guided  | UniFD| 0.78    | 0.61   | 0.65           |  0.57   |\n\nFrom these results, we observe that models integrating **frequency-domain knowledge** (e.g., F3Net, SRM) tend to achieve better generalization to small manipulated regions. This suggests that augmenting MLLM-based explanation models with **frequency-aware reasoning** may help compensate for their current shortcomings in fine-grained localization.\n\nWe believe this is a promising direction, and we will continue to explore it in future work.\n\n## Response to Weakness 4 and Question 1: Lack of Evaluation on Traditional Deepfakes\n\nWe acknowledge that the current version of our framework has not extensively evaluated performance on traditional Deepfake datasets, which may limit the generalizability of our conclusions.\n\nDue to limited time and computational resources, we were unable to construct a separate training and testing pipeline on large-scale Deepfake datasets. However, as a preliminary effort, we evaluated our model on the face-swapping subset of FaceForensics++ . \n\n| Train | DF    | F2F   | FS    | FST   | NT    | Avg(%)↑ |\n|-------|-------|-------|-------|-------|-------|---------|\n| DF    | 96.11 | 96.34 | 95.74 | 95.98 | 95.90 | 96.01   |\n| F2F   | 98.79 | 99.09 | 98.63 | 98.38 | 98.47 | 98.67   |\n| FS    | 99.05 | 99.20 | 99.05 | 98.99 | 99.05 | 99.07   |\n| FST   | 97.09 | 97.30 | 96.57 | 97.01 | 97.01 | 96.99   |\n| NT    | 99.72 | 99.70 | 99.56 | 99.60 | 99.62 | 99.56 |\n\n*Table: Evaluation on Traditional Face-swapping Deepfakes type. Results on different training and testing subsets using VLForgery. Accuracy is used for evaluation.*\n\nWe agree that a more comprehensive study involving diverse Deepfake sources (e.g., face reenactment, lip-syncing, expression manipulation) would be valuable.\n\n\n## Response to Weakness 5: Computational Cost of Large MLLMs\n\nThank you for pointing this out — we acknowledge that we did not provide sufficient details regarding the computational cost.\n\nIn our current setup, training the model for one epoch using 8× NVIDIA RTX L40 GPUs takes approximately 6–8 hours. During inference, generating 100 tokens for a single image takes about 3–5 seconds on a single RTX L40.\n\n## Response to Question 2: Attribution Accuracy Gap Between Partial and Full Synthesis\n\nWe believe this issue is closely related to *Weakness 3*, which concerns low localization accuracy for small modified regions.\n\nOur hypothesis is that attribution fundamentally relies on detecting *consistent semantic patterns* across an image. In the case of **full synthesis**, all regions of the image are generated by the same model and thus share a **unified generative signature or semantic paradigm**. However, for partial synthesis, only a small localized region follows the generative model's semantic pattern, while the rest remains real. This **heterogeneity** makes the attribution task more challenging.\n\nAs mentioned in our response to Weakness 3, incorporating additional frequency-aware features may help enhance the model’s ability to detect and attribute subtle manipulations. These features have shown promise in improving generalization to small forgery regions.\n\nWe also acknowledge the reviewer's point regarding possible dataset imbalance and insufficient signal. While we have balanced the number of examples per generator type, there is an intrinsic imbalance in region size and semantic coverage between full and partial synthesis samples, which could degrade attribution performance.\n\nSince few prior works directly study **attribution for partial synthesis**, our current findings represent a preliminary step. \n\nWe appreciate the reviewer’s valuable suggestions and will consider these directions in future work."}, {"review_id": "eyAz6P1gar", "reviewer": "Reviewer_bT45", "summary": "This paper introduces a single framework for three fake detection problems: detection, localization and attribution. The paper introduces a dataset, called VLF, which contains partially-synthesized images using attribute masks from MM-CelebA-HQ and fully-synthesized images from prompts generated by GPT-4o. The detection model is based on multimodal LLMs, using a prompting template for fake detection, called EkCot, which combines chain-of-thought with extrinsic knowledge in the form of few-shot examples. The fake detection MLLM was finetuned on the VLF dataset with EkCot prompts and tested on a range of datasets and generative models.", "questions": "- Please clarify the methodological novelty of EkCot. How is it different from standard Chain-of-Thought prompting and few-shot prompting in LLMs?\n\n- In multiple steps of the proposed method (e.g., Lines 138, 201, 225), the authors used GPT to generate data and prompts. Can the authors provide more motivation and justification? What is the benefit of using GPT-4o here? How does GPT-4o-generated prompts compare to manually-designed prompts and the prompts generated by other LLMs? In addition, how is GPT-4o prompted in each of these steps?\n\n- In Line 201: \"For question’s format, we use ChatGPT-4o to generate a range of question formats, as shown in Fig. 2(e), for example: ‘Is this image real or fake? Can you provide the reasoning behind your judgment?’\" Can the authors explain the motivation for generating a range of question formats for the same task (fake detection)? Have the authors compared using a range of question formats vs. a single unified question in terms of detection performance?\n\n- Can the authors provide definitions for the undefined technical terms? For example\n  - Lines 178-183: Can the authors clearly define \"judgement bias\" and \"judgement bias descriptors\" in this paragraph?\n  - Line 189: \"Initially, we propose a General Rule template designed to integrate the extrinsic information required for generating desired data.\" Can the authors clarify what the \"General Rule template\" is?\n  - Can the authors define the acronyms in the Tables (e.g., \"FS_SDXL\", \"PS_Kan2.2\", etc. in Table 6)?\n\n- Can the authors consider improving the clarity of Figures 2 and 3? The figure can be less clustered and better organized. In figure 3, the font sizes are disproportionate and some text is obscured by annotations.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The proposed method tackles three challenges, i.e. detection, localization and attribution of generative models, in one unified model, which addresses important problems and is relatively novel compared to conventional detection-only approaches.\n- The paper conducts an extensive evaluation across multiple generators (SD, Flux and Kandinsky) and forging types.\n\nWeaknesses:\n- The technical novelty is limited. The paper primarily combines existing components (e.g. GPT-4o, vision-language models, CLIP, chain-of-thought, prompt engineering, and MLLMs) into a heavily engineered system, rather than proposing a fundamentally new algorithm or model. The reasoning behind many methodology choices are not clear.\n- Although many experimental results are presented, insights and new findings are lacking.\n- The proposed framework relies heavily on closed-source models (e.g., GPT-4o) and substantial prompt engineering, raising questions on reproduction and generalizability of the ideas.\n- The writing quality can be significantly improved. Many technical terms and acronyms are undefined. The clarity of figures can also be improved.", "was_revised": true, "final_justification": "After discussion, my concerns on the novelty and contributions of the paper were partially addressed. However, significant improvements in writing are still required. Based on these considerations, I increase my rating from 2 to 3.", "rebuttal": "I’d like to share some of our thoughts here in hopes of continuing this discussion and eventually reaching a common understanding.\n\n## Response to Weakness 1 and Question 1: Limited Novelty and Differences from Existing CoT/Few-shot Methods\n\nWe humbly disagree with the claim that our work simply combines existing components into a heavily engineered system.Instead, we propose a novel framework that **fine-tunes the MLLM by injecting EkCot knowledge**, enabling it to perform **detection, localization, and attribution** of AI-generated forgeries in a unified manner, rather than relying on auxiliary classifiers or handcrafted prompts.\n\nTo compensate for the limitations of existing datasets, we **curated a new dataset from scratch**, focusing on a diverse range of forgery types. Importantly, **GPT-4o was used solely as an auxiliary tool during the dataset creation process**, helping to generate descriptions. It was **not involved at all in our model architecture, inference pipeline, or explanation generation**.\n\n### As for EkCot, its motivation and mechanism are quite different from previous chain-of-thought or few-shot prompting strategies:\n\n1. Many previous works [1-5] in explainable AIGC forensics suffer from either **(1)** subjective, ambiguous annotations (often generated by ChatGPT or human annotators), or **(2)** many works use ChatGPT to generate explanatory captions, and then fine-tune another model (e.g., LLaVA or LLaMA) with them. However, these models may have different internal representations and biases, and forcing one model to learn another model’s reasoning maybe harms the forensic performance.\n.\n- [1] DDVQA\n- [2] FFAA \n- [3] x2-DFD \n- [4] FakeShield\n- [5] ForgerySleuth \n\n2.These methods typically input only a single image when generating explanations. This is problematic because：\n\n- Most open-source MLLMs are primarily tuned for high-level semantic understanding rather than forensic reasoning.\n- AIGC images today are of extremely high visual quality, making superficial or semantic-level explanations uninformative.\n- More importantly, relying on a single image forces the model to hallucinate plausible explanations without a grounded basis, since current MLLMs lack inherent forensic capabilities. In such cases, the model may fabricate reasons for why an image is fake, especially when no real reference is provided.\n\n### Our Approach\n\nTo address this — how to generate trustworthy explanations.\n\nOur key insight: If we want the MLLM to generate explanatory information that is both beneficial for the model's own training optimization and maintains a certain degree of credibility, we must **respect the model’s own judgment bias** — i.e., what it believes are the most distinguishing features between real and fake images.\n\nSo, we designed the Description Generation Module (Sec. 3.2) with the following principles:\n\n- We **use the same MLLM** for explanation generation and fine-tuning.\n- Instead of using a single image (which leads to randomness), we **input a pair of real and fake images**, so the model can compare and find meaningful differences.\n- We **aggregate** the outputs across multiple pairs to extract dominant **low-level differences**, which we define as the model’s judgment bias.\n\nThis avoids subjective labels, respects model reasoning, and grounds explanations in consistent behavior — distinguishing our method from prior work.\n\n---\n\n## Response to Weakness 2: Lack of Insights and New Findings\n\nWe would like to clarify that several insights and novel contributions are discussed throughout our other responses:\n\n1. **A newly constructed explainable AIGC forensic dataset**  \n   - Face-focused.\n   - Contains partial and fully synthetic images\n\n   - It contains long prompts and template-based facial prompts, designed to generate diverse and realistic content.\n   - It is annotated with explanation-level labels.\n\n2. **A unique strategy to extract judgment bias from MLLMs**  \n   We propose a Low-Level Feature Comparison Pipeline to mine the model’s internal judgment cues. This aligns the explanation process with what the model itself perceives, rather than relying on external interpretation.\n\n3. **Exploration of MLLMs' ability to localize manipulated regions through language**  \n   We provide qualitative and quantitative analyses showing that MLLMs can not only make binary real/fake decisions but also generate fine-grained textual descriptions pointing to manipulated areas. This opens a new direction for text-based localization in multimodal forensics.\n\n---\n\n## Response to Weakness 3 :Reliance on Closed-Source Models and Substantial prompt engineering.\n\nIn our framework, GPT-4o is not involved in the model’s training or inference pipeline.\n\nAs for the concern regarding prompt engineering: it’s important to understand that **prompt design is a central challenge in explainable multimodal forensics**. Generating reliable and faithful explanations from MLLMs inherently requires thoughtful prompt strategies.\n\nIn fact, all recent works [1-6] in this area rely heavily on handcrafted or dataset-specific prompts. For example, [6] introduces an additional classifier to improve detection performance, yet still falls back on traditional prompt-based explanations without fundamental changes. Without meaningful innovation in prompt formulation or explanation mechanisms, the field of MLLM-based forensics cannot truly progress.\n\n- [6] KFD\n\n---\n\n## Response to Weakness 4: Comment on Writing Quality, Terminology, and Figures\n\nThank you for your feedback. We will revise the paper to explicitly define all previously undefined technical terms and abbreviations.\n\n---\n\n## Response to Question2: Use of GPT in Multiple Steps (Lines 138, 201, 225)\n\n### Why Use GPT-4o in Dataset Construction?\n\nIn our dataset, some partially manipulated images are generated by modifying specific regions of real images. In such cases, the visual quality and structure of the base image are preserved, and prompting quality has limited impact on visual realism. Using GPT-4o to generate descriptive prompts in this step allows for:\n\n- Increased diversity in the types of image modifications.\n- Semantic richness, since GPT-4o can create more varied and natural attribute combinations.\n- Better coverage of edge cases, making the dataset more comprehensive and robust.\n\n### Why Use Manually Designed Prompts for Fully Synthetic Images?\n\nIn contrast, fully synthetic images (especially human faces) are extremely sensitive to prompt design. Randomly generated prompts from GPT-4o often lead to unstable or low-quality results due to lack of control over critical attributes.\n\nTherefore, we chose to use hand-crafted prompts specifically tailored to generate high-quality synthetic human faces. These prompts were inspired by real-world image generation use cases (popular prompt styles from YouTube creator communities).  \nWe designed our prompts carefully to ensure high-quality and diverse image outputs, as supported by visual results in our paper.\n\n### Comparison with Other LLMs and Manual Prompts\n\nWhile we used GPT-4o in our implementation, we would like to emphasize that other LLMs (e.g., GPT-3.5, Claude, Gemini, etc.) could also be used for generating image-related prompts, as they all have access to rich linguistic and world knowledge. We simply chose GPT-4o based on convenience.\n\n## Response to Question3:Use of GPT-4o on Line 201, 225\n\n### Motivation for Using a Range of Question Formats\n\nInspired by [7] — which showed that different prompt wordings can lead to noticeable performance differences when using LLMs (e.g., ChatGPT) for real vs. fake image detection. Inspired by this, we followed a similar strategy and used ChatGPT-4o to generate multiple semantically consistent but syntactically diverse question formats.\n\n- [7] Can ChatGPT Detect DeepFakes?\n\nOur goal was to improve the overall robustness and generalization of the MLLM's forensic reasoning ability.\n\nHowever, in our experiments, we found that while using a range of formats does bring slight variation, the impact of question phrasing was relatively minor, especially when compared to other components in the pipeline. Even using a single unified question led to comparable detection performance. Therefore, we decided not to emphasize this part in the main text.\n\n---\n\n## Response to Question4: Undefined Technical Terms\n\n### Judgment Bias and Judgment Bias Description (Lines 178–183)\n\n- Judgment Bias refers to the model’s internal reasoning cu* when determining whether an image is real or fake.\n  \n  For example, if we ask ChatGPT: *“This image is fake — can you tell me why?”* and it responds: *“Because it looks blurry,”*\n  \n  then the keyword \"blurry\" is ChatGPT’s judgment bias.\n  \n- Judgment Bias Description refers to the generated explanatory statement that incorporates one or more judgment biases in natural language form.\n  \n  That is, the judgment bias is only the core evidence (e.g., \"blurry\"), but the description is the full sentence expressing it in an interpretable way.\n\nWe also want to clarify a writing ambiguity in our current draft:\n\nThe \"specific template\" mentioned in Line 183 is the same as the \"general rule template\" mentioned in Line 189. \n\n### Acronyms in Table 6 (e.g., \"FS\\_SDXL\", \"PS\\_Kan2.2\")\n\nThese acronyms follow the same naming rule as described in the caption of Table 4:\n\n- FS\\_SDXL - fully synthetic images generated by the Stable Diffusion XL (SDXL) model.\n- PS\\_Kan2.2 - partially manipulated images generated using the Kandinsky 2.2 model.\n\n## Response to Question 5: Clarity Issues in Figure 2 and Figure 3\n\nUnfortunately, due to NeurIPS rebuttal policy restrictions, we are unable to include revised figures or provide external links at this stage. We appreciate your understanding. We will update these figures in a future version of the paper."}, {"review_id": "lmtRQQJYSG", "reviewer": "Reviewer_9dbu", "summary": "This paper introduces a framework for forgery face detection, localization, and attribution based on MLLM, with the ability to perform fine-grained analysis. Its primary goal is to tackle the challenges of detecting Deepfakes created by high-fidelity human faces generated through diffusion models. Given that existing public datasets lack detailed labels, the authors present the VLF dataset and employ GPT4o alongside various diffusion models to generate forged face data. Additionally, the paper introduces the EkCot (Extrinsic knowledge-guided Chain-of-thought) method, which combines external knowledge from the image generation pipeline with low-level visual differences to guide MLLMs in performing credible forgery analysis. Through extensive experiments, the authors demonstrate that VLForgery outperforms current methods in detection accuracy and shows promising results in localization and attribution tasks.", "questions": "1. Robustness to MLLM hallucinations: Although EkCot aims to reduce the hallucination problem in MLLMs, how well does this mechanism hold up when faced with entirely new or very subtle forgery traces? The paper mentions that MLLMs lack fine-grained forensic awareness, which can lead to hallucinations—how does EkCot fundamentally address this issue?\n2. The results in Table 6 suggest that VLForgery has much lower attribution accuracy for most forgery methods in the Full Synthesis category, as well as for Partial Synthesis type Kan2.2, compared to traditional methods. It only outperforms in three other forgery methods. Could you provide more detailed analysis and explanation of the underlying reasons for this?\n3. Additionally, has the proposed method been tested for robustness against Unseen Perturbations (such as Gaussian blur, JPEG compression, etc.)?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "### Strengths\n1. Most existing forgery face/deepfake detection techniques are either classification models or a combination of classification and segmentation models. This work stands out by integrating all three tasks into a single MLLM-based framework, validating its effectiveness through experiments. This provides a more comprehensive tool for analyzing high-fidelity AI-generated content and its forgeries.\n\n2. The VLF dataset, which is carefully crafted and designed, is a notable contribution to the field. It includes diverse types of forgeries (both partial and full synthesis) and various generators (SDXL, SD2, Kandinsky, SD3, Flux, InstantID), significantly filling the gap left by current datasets that lack fine-grained support for tasks like localization and attribution. This dataset provides valuable resources for future research.\n\n### Weaknesses\n1. Generalization issues for fully synthesized images across subtypes: As shown in Table 2, within the fully synthesized category, the model experiences a significant drop in accuracy under certain cross-type test scenarios (e.g., training on SD3 or Flux but testing on InstantID, where accuracy drops to 21.93% and 48.1%, respectively). This suggests that the model may struggle to generalize when confronted with generators that differ substantially, even within the fully synthesized category, and may fail to capture universal generator fingerprints.\n\n2. While the VLF dataset, along with the low-level visual comparison pipeline and EkCot, enhances the model's perception of fine-grained features, the experiments reveal that the model still faces performance bottlenecks when dealing with local synthesis, especially in smaller regions or cross-generator cases (as shown in Tables 5 and 6). This could indicate that the current low-level visual feature extraction or EkCot method may need further refinement to better capture and utilize these subtle local differences within MLLM.", "was_revised": true, "final_justification": "The authors have fully explained and addressed the concerns I raised in the rebuttal. This work provides valuable insights for future research on deepfake detection using MLLMs. I maintain my original rating.", "rebuttal": "## Response to Weaknesses 1: Generalization Issues Across Fully Synthesized Subtypes (e.g., InstantID)\n\nWe also noticed that **performance consistently drops when InstantID is used as the test generator**, regardless of the training source.\n\nUpon closer examination, we believe a major reason lies in the unique nature of InstantID. As an identity-preserving generation method, InstantID retains facial identity characteristics from the reference (pristine) image, resulting in a feature distribution that closely resembles that of pristine images. This makes it inherently more difficult to distinguish from authentic images.\n\nFurthermore, we observed that some training generators, such as **Kandinsky**, appear more capable of capturing and generalizing to these identity-preserving characteristics, while others like SD3 and Flux are more prone to confusion when encountering InstantID outputs.\n\n## Response to Weakness 2: Low Accuracy for Small Regions\n \nTo investigate why accuracy remains low for small regions (e.g., ears), we conducted a detailed analysis on the partially manipulated samples in our dataset. As shown in Table, we calculated the average proportion of each manipulated region in both the training and testing sets. The distributions are nearly identical, indicating no significant bias between them. For example, **hair** has the largest average proportion (34.50%), while **eyebrows** are the least represented (0.86%).\n \n| Region | Brow | Ear | Hair | Nose |\n|--------|-----|-------|------|------|\n| Train Set (%) | 0.86 | 1.88 | 34.50 | 2.03 |\n| Test Set (%) | 0.92 | 1.93 | 31.74 | 2.09 |\n \nWe further compared a variety of state-of-the-art manipulation localization methods and their performances across different regions, especially focusing on smaller areas like brow. Here are selected results for hair/brow ACC values:\n \n| Type| Method | SD1.5(%)↑ train-hair |  SD1.5(%)↑ brow | FLUX(%)↑ train-hair |  FLUX(%)↑ brow |\n|-------------|----------------------|----------------------|----------------|----------------|---------------------|\n| Naive       | Resnet-50 | 0.95            | 0.66      | 0.99           | 0.36     |\n| Naive       | Xception | 0.98   | 0.63 | 0.99    |  0.17    |\n| Naive       | PSCC-Net | 0.82        |  0.70    | 1.00         | 0.98    |\n| Frequency   | SPSL| 0.98 | 0.77  | 0.98     | 0.35   |\n| Frequency   | F3Net | 0.98            | 0.62  | 0.99           | 0.28    |\n| Frequency   | SRM | 1.00        | 0.99  | 1.00       |  0.86    |\n| Frequency   | NPR | 0.97       | 0.84  | 0.90           | 0.81    |\n| Frequency   | MVSS-Net | 0.89           | 0.88    | 0.96       | 0.95     |\n| Frequency   | MMFusion | 0.99     | 0.94  | 0.99         | 0.12  |\n| VLM-Guided  | UniFD| 0.78    | 0.61   | 0.65           |  0.57   |\n \nFrom these results, we observe that models integrating **frequency-domain knowledge** (e.g., F3Net, SRM) tend to achieve better generalization to small manipulated regions. This suggests that augmenting MLLM-based explanation models with **frequency-aware reasoning** may help compensate for their current shortcomings in fine-grained localization.\n \nWe believe this is a promising direction, and we will continue to explore it in future work.\n\n## Response to Question 1: Robustness to MLLM Hallucinations and Effectiveness of EkCot\n\n### Our Core Insight\n\nIf we want the MLLM to produce explanations that are both credible and beneficial for model optimization, we must first respect the model’s own judgment bias — i.e., what the MLLM *internally considers* to be the most distinguishing features between real and fake images. This insight directly aligns with our objective (Line 165) of **seek forensic features that the model itself can understand to discriminate the authenticity of an image**, instead of imposing externally defined features.\n\n### How EkCot Addresses Hallucination\n\nTo tackle hallucination and guide the model toward more grounded reasoning, we designed the **Description Generation Module (Section 3.2)** based on the following principles:\n\n- We use the **same MLLM** for both explanation generation and fine-tuning, ensuring consistency in the model's knowledge and representation space.\n- Instead of using a single image (which leads to randomness), we **input a pair of real and fake images**, so the model can compare and find meaningful differences.\n- We **aggregate explanations across multiple image pairs** to extract recurring, low-level forensic attributes — which we define as the model’s **judgment bias**.\n\nThis approach avoids handcrafted or subjective features, encourages the model to stay within its own representational bounds, and grounds explanation in behaviorally consistent visual evidence. These qualities distinguish EkCot from prior approaches that rely on static or external labels.\n\n### Empirical Evidence of Robustness\n\nTo evaluate whether the extracted **judgment bias** is effective — especially when facing **subtle forgery traces** like in partially manipulated images — we conducted ablation studies. We isolated the judgment bias from the EkCot pipeline and tested its impact on various subsets.\n\n| Method       | VF  | EKCot (w/o JuB) | EKCot (JuB) | SDXL Partial Synthesis | SDXL Full Synthesis | SD2 Partial Synthesis | SD2 Full Synthesis | Kandinsky2.2 Partial Synthesis | Kandinsky2.2 Full Synthesis | SD3     | Flus   | InstantID |\n|--------------|-----|-------|------|------------------|---------------------|------------------------|--------------------|-------------------------------|----------------------------|---------|--------|-----------|\n| VariantA     |     |       | |69.92                  | 99.92               | 51.07                  | 99.97              | 99.56                         | 99.74                      | 97.66   | 99.12  | 99.69     |\n| VariantB     |     |      |✓| 68.00                  | **99.99**           | 47.64                  | **99.98**          | 99.18                         | **99.99**                  | 99.98   | 99.98  | **99.84** |\n| VariantC     | ✓   | ✓  | | 74.32              | 99.97               | 64.21              | 99.97              | 99.94                     | 99.97                      | **100.00** | **99.98** | 99.41 |\n| **Ours**     | ✓   |      | ✓|**78.62**              | 99.98               | **66.32**              | 99.97              | **99.96**                     | 99.97                      | 99.98 | 99.97 | 99.57 |\n\n\n*Table: Ablation Study of the VLForgery's forgery description generation component in the detection task. 'VF' , 'EKCot' and 'Jub' represent visual input features in forgery description generation,  extrinsic knowledge-guided chain-of-thought, and Judgment Bias, respectively.*\n\nThe results (provided in above Table) show that incorporating judgment bias improves detection performance on partially synthesized images, which are known to exhibit more subtle manipulations.\n\nWe appreciate the reviewer’s suggestion and will include additional results and analyses to further highlight these findings in the final version.\n\n## Response to Question 2: Lower Attribution Accuracy in Full Synthesis and Partial Synthesis Methods (Table 6)\n\n### 1. Architectural Differences: Convolutional Models vs. Vision Foundation Models (VFMS)\n\nIn our study, traditional models, which are based on conventional convolutional architectures, tend to show very high attribution accuracy (around 98–99%) for Full Synthesis (FS) samples. However, their performance significantly drops when dealing with Partial Synthesis (PS) samples, with accuracy as low as 6%. In contrast, our method, which is built on vision foundation models (VFMS), exhibits more **balanced performance** across both FS and PS categories.\n\n### 2. Explaining the Drop in Accuracy\n\nWe believe this drop in accuracy can be interpreted with insights from the paper [1], which discusses the asymmetric detection phenomenon in AIGC forensics. Traditional convolutional models are prone to overfitting to specific forgery patterns in the training data, especially in the FS case, where large-scale synthetic artifacts are easier to capture. However, these models struggle with generalization when encountering subtle or unseen manipulations (e.g., PS samples such as PS_SDXL).\n\n- [1] Effort: Efficient Orthogonal Modeling for Generalizable AI-Generated Image\nDetection\n\nIn contrast, VFMS-based methods can better incorporate semantic-level information (e.g., scene context, object structure, identity consistency), enabling the model to form a higher-dimensional decision space that fuses both semantic content and forgery cues. This reduces overreliance on any single artifact or forgery pattern and leads to more robust performance across forgery types.\n\n## Response to Question 3: Robustness Against Unseen Perturbations\n\nDue to the NeurIPS 2025 rebuttal policy that limits the response to 10,000 characters, we have addressed this concern **under our reply to Reviewer fi2j's Weakness 4**, as the two questions are closely related in nature. We sincerely appreciate your understanding."}, {"review_id": "uIUd0cxd1l", "reviewer": "Reviewer_fi2j", "summary": "Images generated by the diffusion models pose a significant challenge for Deepfake detection. This paper proposed a Multimodal Large Language Model (MLLM) based deepfake detection framework named VLForgery. This framework identifies manipulated regions and distinguishes different synthetic generators, thereby effectively detecting forgeries. To this end, the paper first collected the VLF (Visual Language Forensics) dataset with various generators. Then, the Extrinsic knowledge-guided Chain-of-thought (EkCot) is constructed to guide the MLLM to detect forgeries. Finally, the paper proposes a low-level vision comparison pipeline to explore the significant differential features between real and fake images. Extensive experiments demonstrate the superior performance of the proposed method.", "questions": "Please refer to the weakness.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. The Extrinsic knowledge-guided Chain-of-thought (EkCot) improved the interpretability of large models for forgery detection. The method provides new insight into the explainability of deepfake detection based on multimodal large language models.\n2. The paper conducts a comprehensive evaluation experiment to demonstrate the superior performance of the proposed method.\n\n**Weaknesses**\n\n1. In this paper, the VLF dataset generates fake images with published manipulation methods. The data collection pipeline is similar to previous work [1]. It seems that the paper should discuss the difference between VLF and other manipulation detection datasets to demonstrate the contribution of the proposed benchmark.\n2. In Table 4, the vanilla binary classifier based on Xception achieved high forgery detection accuracy on cross-dataset evaluation. Such experimental results indicate a strong correlation among forgeries generated by different manipulation methods within the VLF dataset. The paper should conduct cross-dataset evaluation experiments on more challenging benchmarks, such as DFDC [2] and FF++ [3].\n3. In this paper, the VLForgery is only compared with a few vanilla binary classifiers. More recent works on deepfake detection, such as [4, 5, 6], should be considered in the experiments.\n4. Robustness toward different kinds of perturbations is also an important part of DeepFake detection. The paper should conduct such experiments to indicate the robustness of the proposed method.\n\n[1] Diffusion Facial Forgery Detection\n\n[2] The deepfake detection challenge dataset. \n\n[3] Faceforensics++: Learning to detect manipulated facial images.\n\n[4] Face X-Ray for More General Face Forgery Detection\n\n[5] Detecting Deepfakes With Self-Blended Images\n\n[6] Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization", "was_revised": true, "final_justification": "Thank you to the authors for their response and the additional experiments. It addresses my concern. I maintain my recommendation of borderline accept.", "rebuttal": "## Response to Weakness 1: Differences Between VLF and Prior Manipulation Detection Datasets\n\nThank you for this important comment. At the early stage of our work, we indeed conducted a careful survey of existing datasets such as [1] and others. However, we found that they could not fully meet the needs of our task. Below, we outline key differences and contributions of the proposed VLF benchmark:\n\n1. **Prompt Design for Face Generation**:  \n   Existing AIGC datasets (e.g., [1], [2]) often use short and simplistic prompts, which may not produce low-quality images, but result in less detailed and diverse facial features.  \n   In contrast, we design long-format prompts that are*specifically tailored for face generation, inspired by dedicated YouTube channels studying facial synthesis (details in Section B.3). Our prompts include semantically rich tokens such as `[framing]`, `[camera_angle]`, `[lighting]`, etc., which significantly enhance facial realism and diversity.\n\n2. **Generator Quality and Coverage**:  \n   [1] often rely on privately trained or lower-quality open-source models, which may not match the realism of widely-used diffusion models like Flux, Kandinsky, etc.  \n\n3. **Detailed Annotations for Partial Synthesis**:  \n   Our task focuses on fine-grained forgery localization and attribution, which requires precise metadata. Existing datasets rarely provide such details.  \n   In VLF, we include annotations on:\n   - Whether edits involve single-region or multi-region manipulation;\n   - The exact name and location of modified regions;\n   - Whether the image was synthesized from scratch or by editing real images.\n\n- [1] Diffusion Facial Forgery Detection\n- [2] GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image\n\n## Response to weakness 2: Cross-Dataset Evaluation on Challenging Benchmarks (e.g., DFDC, FF++)\n\nWe appreciate the reviewer’s suggestion regarding broader cross-dataset evaluation. We agree that the current version of our framework has not been extensively evaluated on traditional Deepfake datasets, which could limit the generalizability of our conclusions. \n\nDue to time and computational resource constraints, we were unable to build and run a dedicated training/testing pipeline on large-scale Deepfake datasets like DFDC. However, as an initial step, we conducted evaluation on the face-swapping subset of FaceForensics++. \n| Train | DF    | F2F   | FS    | FST   | NT    | Avg(%)↑ |\n|-------|-------|-------|-------|-------|-------|---------|\n| DF    | 96.11 | 96.34 | 95.74 | 95.98 | 95.90 | 96.01   |\n| F2F   | 98.79 | 99.09 | 98.63 | 98.38 | 98.47 | 98.67   |\n| FS    | 99.05 | 99.20 | 99.05 | 98.99 | 99.05 | 99.07   |\n| FST   | 97.09 | 97.30 | 96.57 | 97.01 | 97.01 | 96.99   |\n| NT    | 99.72 | 99.70 | 99.56 | 99.60 | 99.62 | 99.56 |\n\n*Table: Evaluation on Traditional Face-swapping Deepfakes type. Results on different training and testing subsets using VLForgery. Accuracy is used for evaluation.*\n\nWe will include these results in the revised version of the paper and plan to expand to additional benchmarks in future work.\n\n## Response to Weakness 3: Limited Baselines – Consideration of Recent Deepfake Detection Methods [4, 5, 6]\n\nThank you for this valuable comment. We understand your concern regarding the comparison with only a few baseline methods. Below, we clarify our rationale:\n\nAlthough both traditional Deepfake datasets (e.g., [FF++]) and AIGC-based datasets (e.g., [1]) focus on face manipulation, **the manipulation mechanisms differ fundamentally**. Traditional Deepfakes are typically generated via **face-swapping or reenactment techniques** such as StarGAN, FaceAPP, or FaceSwap, whereas our dataset, VLForgery, is based on **diffusion models** and **prompt-driven generation**.\n\nMethods like [4, 5, 6] enhance forgery detection by exploiting **artifacts specific to face-swapping** (e.g., warping, boundary inconsistencies). While these are effective for traditional Deepfakes, such artifacts **do not exist** in AIGC-based forgeries. Instead, recent works for AIGC forensics (e.g., DRCT using self-reconstruction, SAFE and CNNspot using traditional image augmentation, and SRM, SPSL, NPR employing frequency-based features) are more suitable.\n\nTherefore, we believe our inclusion of these **AIGC-relevant forensics methods**—particularly those utilizing frequency priors—is more aligned with the nature of our benchmark. That said, we acknowledge that our current baselines might appear limited, and we are open to incorporating more recent models in future versions of the paper. We sincerely welcome further discussion with the reviewer to reach a consensus.\n\n## Response to Weakness 4: Robustness Against Perturbations\n\nWe agree that robustness toward different types of perturbations is a crucial aspect of DeepFake detection. To investigate this, we conducted robustness experiments using five commonly used perturbation types: \n\n- **GB**: Gaussian Blur  \n- **BW**: Block-wise Masking  \n- **CC**: Color Contrast  \n- **CS**: Color Saturation  \n- **JC**: JPEG Compression  \n\nWe selected representative detectors from three categories, including our proposed VLForgery. For simplicity and to reduce resource cost, we applied a single fixed severity level for each perturbation type. The detailed results are shown in the table below.\n\n|Robustness Type|||||||\n|--|--|--|--|--|--|--|\n|**Train-SDXL（PS)**|f3net|gramnet|srm|CLIPing_linear|VLF|\n|GB|69.63|37.12|25.17|55.68|68.23\n|BW|72.89|32.86|60.74|52.19|69.32\n|CC|72.91|37.89|52.91|58.33|75.83\n|CS|71.92|32.79|52.17|48.81|58.64\n|JC|72.30|25.10|3.47|29.11|59.91\n|**Trani-SDXL(FS)**|\n|GB|43.00|99.92|99.35|99.98|99.67\n|BW|37.68|99.95|99.98|99.98|99.87\n|CC|41.90|97.93|87.97|99.98|99.54\n|CS|41.22|99.73|99.43|99.98|99.98\n|JC|42.28|99.65|72.42|99.22|99.31\n|**SD2(PS)**|\n|GB|45.12|32.03|0.11|51.57|67.34\n|BW|45.47|23.79|14.98|50.42|68.19\n|CC|44.29|28.61|32.63|47.63|65.43\n|CS|43.00|20.37|26.86|40.12|63.24\n|JC|43.03|7.54|0.15|45.43|60.17\n|**SD2(FS)**|\n|GB|32.67|98.48|96.30|99.75|98.73\n|BW|25.83|96.30|99.02|97.33|98.62\n|CC|32.20|93.47|89.00|96.08|94.31\n|CS|31.18|95.83|97.50|96.70|97.62\n|JC|31.82|83.17|85.73|93.93|92.89\n|**Kan(PS)**|\n|GB|65.20|56.07|2.34|79.73|87.35\n|BW|72.31|56.10|98.87|70.23|89.25\n|CC|75.78|52.27|77.74|75.99|87.31\n|CS|74.06|49.17|81.91|71.36|70.94\n|JC|73.79|33.08|1.64|32.57|65.45\n|**Kan(FS)**|\n|GB|38.95|99.65|89.15|99.98|99.92\n|BW|32.32|99.53|99.62|99.58|99.87\n|CC|38.87|98.53|87.05|99.60|98.64\n|CS|36.93|98.85|98.98|99.50|99.17\n|JC|38.18|98.43|85.97|99.23|99.33\n|**SD3**|\n|GB|29.96|95.17|93.45|99.52|98.35\n|BW|27.97|91.82|98.35|98.05|97.83\n|CC|28.62|90.57|84.12|98.17|99.23\n|CS|27.75|90.50|96.32|95.48|93.27\n|JC|29.02|88.92|64.98|72.72|89.13\n|**Flux**|\n|GB|35.50|99.13|94.02|99.85|99.82\n|BW|32.28|99.10|98.30|99.13|99.09\n|CC|36.85|98.08|85.72|99.25|99.14\n|CS|33.23|98.63|95.55|98.42|99.03\n|JC|34.05|98.18|74.77|87.53|88.27\n|**InstantID**|\n|GB|63.34|98.66|90.33|100.00|99.48\n|BW|67.74|98.94|97.41|99.76|99.39\n|CC|72.81|97.45|81.93|99.61|99.41\n|CS|69.43|98.59|92.73|99.69|98.39\n|JC|68.06|97.80|49.39|96.66|97.82\n\n\nKey observations include:\n\n- **JPEG compression (JC)** tends to significantly affect most detectors.\n- There is noticeable variation in robustness across detectors and test sets.\n- Our method demonstrates a relatively balanced robustness profile — unlike SRM, which fails entirely under JC, or F3Net, which collapses in performance on fully synthesized samples."}]

Author rebuttal:

Meta-review: The paper introduces VLForgery, a triad framework for AI-generated face forensics that integrates MLLMs to perform three key tasks: detection, localization, and attribution of facial forgeries. Compared to traditional binary detectors, VLForgery provides fine-grained analysis by identifying manipulated regions and attributing it to specific diffusion models. The authors also present the VLF dataset by employing GPT4o and various diffusion models to generate forged face images. Additionally, the paper introduces the EkCot (Extrinsic knowledge-guided Chain-of-thought) method, which combines external knowledge from the image generation pipeline with low-level visual differences to guide MLLMs in performing credible forgery analysis. 

Overall speaking, the proposed work is well-motivated, with moderate novelty, technically sound, with extensive experiments, and reporting superior results. More importantly, the work advances the field from mostly binary classification to detailed analysis with some degree of explainability. The new dataset is also good for the field to shift to new tasks. 

There are some negative concerns mainly on generalizability to real-world unseen forgery methods, relatively low accuracy on forgery localization, heavy reliance on closed-source models (GPT-4o), lack of evaluations on some perturbations, and writing quality issue. During the rebuttal, most of the concerns have been solved by the authors. One reviewer is not satisfied mainly due to the writing problem. While indeed there are many writing problems, they do not downgrade the contribution of the work essentially. Therefore, the AC suggests an Accept. In the final version, the authors should improve the writing very carefully by considering the reviewer’s suggestions.
Decision: Accept (poster)