$\mathcal{X}^2$-DFD: A framework for e$\mathcal{X}$plainable and e$\mathcal{X}$tendable Deepfake Detection

Authors: Yize Chen, Zhiyuan Yan, Siwei Lyu, Baoyuan Wu
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: Deepfake Detection; Multimodal Large Language Models; Media Forensics

Abstract:
Detecting deepfakes (*i.e.*, AI-generated content with malicious intent) has become an important task. Most existing detection methods provide only real/fake predictions without offering human-comprehensible explanations. Recent studies leveraging multimodal large-language models (MLLMs) for deepfake detection have shown improvements in explainability. However, the performance of pre-trained MLLMs (*e.g.*, LLaVA) remains limited due to a lack of understanding of their capabilities for this task and strategies to enhance them. In this work, we empirically assess the strengths and weaknesses of MLLMs specifically in deepfake detection via forgery-related feature analysis. Building on these assessments, we propose a novel framework called $\mathcal{X}^2$-DFD, consisting of three core modules. 
The first module, *Model Feature Assessment (MFA)*, measures the detection capabilities of forgery-related features intrinsic to MLLMs, and gives a descending ranking of these features. 
The second module, *Strong Feature Strengthening (SFS)*, enhances the detection and explanation capabilities by fine-tuning the MLLM on a dataset constructed based on the top-ranked features. 
The third module, *Weak Feature Supplementing (WFS)*, improves the fine-tuned MLLM's capabilities on lower-ranked features by integrating external dedicated deepfake detectors. 
To verify the effectiveness of this framework, we further present a practical implementation, where an automated forger-related feature generation, evaluation, and ranking procedure is designed for *MFA* module; an automated generation procedure of the fine-tuning dataset containing real and fake images with explanations based on top-ranked features is developed for *SFS* model; an external conventional deepfake detector focusing on blending artifact, which corresponds to a low detection capability in the pre-trained MLLM, is integrated for *WFS* module. 
Experimental results show that the proposed implementation enhances overall detection performance compared to pre-trained MLLMs, while providing more convincing explanations. 
More encouragingly, our framework is designed to be plug-and-play, allowing it to seamlessly integrate with more advanced MLLMs and external detectors, leading to continual improvement and extension to face the challenges of rapidly evolving deepfake technologies.

Decision: reject
Reviewer scores: 5, 6, 8, 3

Reviews and discussion:
<thread>
creation date:
1730421350773

thread_type:
official_review

<thread_object>
creator: reviewer_frf6

<content>
summary:
The authors propose a novel framework called X2-DFD, consisting of three core modules (i.e., Model Feature Assessment (MFA), Strong Feature Strengthening (SFS), and Weak Feature Supplementing (WFS)) for explainable deepfake detection. The results on several benchmark datasets demonstrate its effectiveness. However, the overall story and solution is interesting but with too much human workload and tricks while without enough technical contribution.

strengths:
This paper adopts MLLMs for explainable deepfake detection. The cross-dataset performance seems good. The structure of the paper is clear.

weaknesses:
1. The cross-manipulation performance presented in Table 2 is suboptimal despite using a trainable MLLM and a non-trainable GPT-4. Given the extensive parameters and pre-trained data leveraged, the proposed method cannot achieve SOTA performance. The capability of handling unseen attacks limits its applications in real-world scenarios.

2. X2-DFD’s explainablity is unsatisfactory for some samples, such as the left bottom sample in Figure 1, which outputs vague statements like “...with an unusual layout and unnatural skin tone...”. This description is oversimplified. It cannot provide sufficient information to users and offers less explanatory power than traditional methods like Grad-CAM (which can provide heatmaps). Is this a common issue across samples, and how is explainablity assessed (e.g., through user studies)?

3. The authors ignore prior work using LLMs for deepfake detection and over-claim their contribution (Lines 100-103)(r1,r2). The summary on Lines 067-075 misrepresents previous work [r1] by claiming“...fail to provide intuitive and convincing explanations behind predictions.”

[r1] FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models
[r2]Common Sense Reasoning for Deep Fake Detection

4. The authors do not present experiments on low-quality images or post-processed images.

5. The ablation study is insufficient. What would happen if WFS were applied to Strong Features or SFS to Weak Features? Or if both Strong and Weak Features used the same module?

6. The idea of using an external model (EDD in this paper) to support LLMs is not novel.

[r3] LISA: Large Language Instructed Segmentation Assistant
[r4] AnomalyGPT: Detecting Industrial Anomalies using Large Vision-Language Models

7. The reason for choosing LLaVA & GPT-4 is unclear. What would happen with different MLLMs, or different parameter configurations (e.g., LLaVA-13B/7B or GPT-3)? X2-DFD’s sensitivity to various MLLMs remains unexplored.

8. Writing issues:
(1) Citation inconsistencies, e.g., the reference in L136 is missing. ArXiv references, especially in L598-601 and L619-621, follow inconsistent formats.
(2) Missing percentage symbols in metrics reported on L175-180.
(3) Metric explanations occupy excessive space.
(4) Table 4 lacks clarity regarding which metrics are presented.


9 .During the reasoning process, the paper utilizes an external reasoning engine for analysis. However, if the results from this engine are subpar, it could significantly impact the model's performance. Unfortunately, the paper does not discuss this in detail（Especially, the training details of external expert models are not mentioned）. Additionally, in Table 3's ablation study, the minimal performance difference between the EDD only and final models suggests limited gains. Furthermore, using an untrained MLLM as a baseline for comparison seems unreasonable.

10. In the strong feature data selection phase, the paper considers only one metric, and similarly, it evaluates performance using just that metric. This lack of additional metrics related to accuracy (ACC) or natural language processing (NLP) interpretability may limit the comprehensiveness of the results.

11.The paper lacks a discussion on how common large models would perform in comparative experiments after fine-tuning, such as the effects of switching to different LLM base models, which is worth exploring further.

questions:
see weakness for details.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730534605328

thread_type:
official_review

<thread_object>
creator: reviewer_umwy

<content>
summary:
The paper proposes X2-DFD, a novel framework that utilizes Multimodal Large Language Models (MLLMs) for explainable and extendable DeepFake Detection. The basic idea of the paper is quite interesting. And the paper seems to the first to systematically assess the inherent capabilities of MLLMs specifically in deepfake detection, which is valuable.

strengths:
1.The paper first systematically assesses the inherent capabilities of MLLMs specifically in deepfake detection, and has found that MLLMs have varying discriminating capabilities on different forgery features.
2.It proposes a novel approach to fine-tune the MLLM to make it better adaptive with the deepfake detection task. 
3.Besides MLLM, it integrates external dedicated detectors (EDDs) to fill the gap where MLLMs show limitations.

weaknesses:
Although interesting to see MLLM being applied in improving Deepfake detection performance, I still wonder if the interpretability brought by  MLLM is reliable. Therefore I have two major concerns:
1.	The explainability of large language models (LLMs) like GPT, BERT, and similar models is a complex and evolving area. Generally, LLMs are not inherently transparent or easily interpretable due to their massive size and the intricacy of their neural architectures. So, how do you ensure the explainability brought by LLMs is “real” explainability instead of merely providing some texts describing why an image is fake or not. 
2.	Based on my first point, how would you evaluate the explainability of the proposed method? There seems to be a lack of metric or ground-truth involved in your research. I wonder if human evaluation is fairly enough to evaluate the explainability since human subjects involved in the experiments also have no real knowledge about how the fake data was created and why it is fake. By the way, the details of the human experiment are not provided. 
In my opinion, it will be more promising to integrate the knowledge about the deepfake creation and the true difference between real and fake data into the detection stage in order to improve the detector’s explainability. Nevertheless, I still think it is a good paper which provides a good trial on solving deepfake detection with MLLMs.

questions:
The extendability of the proposed method should be further explained. Most efforts of the paper are on the explainability. I do not clearly see what you mean by “extendable”.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730652615820

thread_type:
official_review

<thread_object>
creator: reviewer_37wm

<content>
summary:
In this paper, the authors use a multi-modal large model to enhance the interpretability of face forgery detection. To construct an effective fine-tuning dataset, they propose first ranking various forgery features and then building a corresponding VQA dataset based on the top-ranked features. Additionally, the detection results from smaller models are fed into the LLM as extra guidance.

strengths:
1. The recognition effect has been greatly improved, especially for cross-dataset scenarios.
2. It is the first exploration to design a novel MLLMs-based framework for explainable forgery detection.
3. The experiments are thorough and the comparisons are comprehensive, effectively demonstrating the validity of the proposed method.
4. The paper is well-organized and easy to understand.

weaknesses:
1. It is hard to believe the simple fine-tuning of multi-modal large models typically can achieve such satisfactory results. The public source code can help prove its reproducibility.
2. Directly using the prediction probabilities of the smaller model as text input for the LLM seems unreasonable, as LLMs lack numerical reasoning capabilities.
3. Since Pretrained LLMs lack fundamental knowledge about true/false classification, ranking based on relevance generated by Pretrained LLMs may not be meaningful. It may be that random selection of other questions has a similar effect. Can the author provide further analysis?
4. Authors only select the specific LLM and MLLM models in the framework. How to ensure the generality and reasonability of the conclusion?

questions:
1. Only the LLaVa model is chosen as the typical MLLM. Do most MLLMs contain similar properties? The author should point this out.
2. How can the commonality of the definition of forgery features be ensured? Authors utilized the GPT-4o to generate a questions list, how about other LLM models? Different LLM models may generate different question lists to extract forgery-related features.
3. In TABLE 2. Why the proposed method is inferior to comparison methods with fsgan and inswap generation models? Please give a deeper theoretical analysis.
4. In TABLE 3, the GCS concept was not explained clearly, and the ablation study settings are somewhat confusing.
5. To ensure the integrity of the experiment, authors also could provide in-domain results in the FF++ dataset.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730699823279

thread_type:
official_review

<thread_object>
creator: reviewer_nfd6

<content>
summary:
This paper proposes a new method for deepfake detection based on MLLMs. The method enhances the deepfake detection capabilities of MLLMs by ranking forgery-related features, strengthening strong features and augmenting weak features. In addition, the paper raises the explainability of MLLMs' inference through fine-tuning. Evaluations on multiple datasets prove the effectiveness of the proposed method.

strengths:
1.	The paper proposes Model Feature Assessment(MFA), Strong Feature Strengthening (SFS) and Weak Feature Supplementing (WFS) to enhance the detection capabilities of MLLMs, contributing to MLLM-based algorithms on deepfake detection.

2.	The paper points out the limitations of pretrained MLLMs and introduces a pipeline for constructing datasets based on more effective forgery features.

3.	The experiments provide different protocols and a wide range of evaluation to prove the effectiveness of the method.

weaknesses:
1.	The authors claim that human-annotated VQA data (zhang et al, 2024) may not be ideal as standard answers for fine-tuning MLLMs. Instead, they propose using MLLM-generated data to fine-tune MLLMs. This raises an important question: if all answers are expressed in human's natural language, how can MLLM-generated data establish a more 'standard' answer? Even in the rebuttal response, the author claims their collection method is efficient instead of being a "standard" answer. 

2.	The generated questions lack reliability due to the absence of a human verification process to ensure the accuracy of these fake features. Additionally, the authors use data generated by MLLMs to evaluate the MLLMs themselves, which raises concerns about the validity of this approach. This circular logic is confusing and calls into question the robustness of the evaluation.

3.	The authors claim to study the intrinsic capabilities of MLLMs on detection. However, Strong Feature Strengthening (SFS) and Weak Feature Supplementing (WFS) seem to be a process of dataset construction based on more effective features rated by the ranking in Sec. 3.3, thus becoming less pervasive in how it improves the explainability of MLLMs. Although the rebuttal response emphasizes its effectiveness, I am still not convinced by it. For example, in Fig. 5, what if the blending score is wrong? what if the face is not generated by the blending process so the blending score is low while the face is still fake?

4.	The explanation ability of X^2-DFD should also be quantitatively evaluated in the main paper. The authors mention conducting a human study to assess explanation performance in Fig. 7, but they only verbally claim that their model is superior and provide a single qualitative example in the appendix. This lacks reliability and ignores the main contribution of explainability.

5.  A new question after the rebuttal: Line 176, LLaVA and XceptionNet achieve really unreliable detection performance, e.g.., 63.7% and 75.8%; this means both of them are almost equally incapable in the detection task, and analyzing them is less meaningful.

questions:
1.	In Figure 1, I wonder why X^2-DFD predicts a lower fake probability on the second picture compared with a pre-trained MLLM. Meanwhile, the response of X^2-DFD considers the image completely fake, which is contrary to the predicted probability.

2.	Figure 2 is too small; the numbers are barely visible.

3.	There are undefined and inconsistent expressions. In Sec. 4.2.3 and Figure 5, The abbreviation of WCS is undefined in the aforementioned method. In Sec. 5.4 and Tab. 3, GCS has never been seen before.

4.	More details on the implementation of fine-tuning MLLMs should be provided. Which part of the parameters are trained? The dataset information should be detailed.

[Summary]: I appreciate the author's efforts in the rebuttal, while I still would like to maintain my first-round score and suggest this paper revised for the next submission.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a MLLM-based method to detect face forgery. All reviewers gave positive recognitions to adopting MLLMs for explainable deepfake detection. However, part of the reviewers still had concerns about experiments, opinions and writing. I have reviewed this revised paper and had some concerns that were consistent with reviewers.

Some necessary experiments or analyses are missing to support the authors' views or conclusions. For example, the performances on the low-quality images are not evaluated (e.g., using the model trained on the RAW version of FF++ to test the FF++ with the C23 or C40 version), the discussion on whether LLM would misunderstand normal post-processing traces (e.g., blur or contrast) as forgery traces is not provided, specific experimental results are lacking to demonstrate that applying SFS to weak features is ineffective and that the fine-tuning results are similar to the No SFS baseline, sufficient quantitative results are missing to support the claimed enhancements to explanation of the pre-trained MLLM.

In addition, given that the large gap between the Phi-3-vision and the LLaVa-7B model, the stated view that the framework does not rely on a specific MLLM is not sufficiently convincing. According to this view, the proposed method should be able to achieve similar detection performance under different MLLMs. Besides, some obvious writing issues remain in the revised version, e.g., missing citations in L916 and L1115. Based on the above considerations, I do not recommend accepting this paper.