Title: X2-DFD: A framework for explainable and extendable Deepfake Detection
Venue: NeurIPS 2025
Accepted: yes
Keywords: Deepfake Detection

Forgery Detection

Fake Image Detection
Abstract: This paper proposes **$\mathcal{X}^2$-DFD**, an **e$\mathcal{X}$plainable** and **e$\mathcal{X}$tendable** framework based on multimodal large-language models (MLLMs) for deepfake detection, consisting of three key stages. 
The first stage, *Model Feature Assessment*, systematically evaluates the detectability of forgery-related features for the MLLM, generating a prioritized ranking of features based on their intrinsic importance to the model.
The second stage, *Explainable Dataset Construction*, consists of two key modules: *Strong Feature Strengthening*, which is designed to enhance the model’s existing detection and explanation capabilities by reinforcing its well-learned features, and *Weak Feature Supplementing*, which addresses gaps by integrating specific feature detectors (e.g., low-level artifact analyzers) to compensate for the MLLM’s limitations.
The third stage, Fine-tuning and Inference, involves fine-tuning the MLLM on the constructed dataset and deploying it for final detection and explanation.
By integrating these three stages, our approach enhances the MLLM's strengths while supplementing its weaknesses, ultimately improving both the detectability and explainability.
Extensive experiments and ablations, followed by a comprehensive human study, validate the improved performance of our approach compared to the original MLLMs.
More encouragingly, our framework is designed to be plug-and-play, allowing it to seamlessly integrate with future more advanced MLLMs and specific feature detectors, leading to continual improvement and extension to face the challenges of rapidly evolving deepfakes. Code can be found on https://github.com/chenyize111/X2DFD.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the problem of deepfake detection with the help of an MLLM by funetuning it with a self-constructed new dataset. The dataset constructing process consists of 2 stages. To construct the dataset, for a given sample, it first evaluates and ranks forgery-related features (as strong and weak features)  based on their detectability. Based on the ranking results, they further suggest directly using the strong feature as classification prompts, while for the weak features, it includes an additional prompts related to its blending factor. After finetuning the MLLM on the newly contructed dataset via LoRA, this method shows leading performance in different datasets.\n\nSTRENGTHS AND WEAKNESSES:\nPros:\n \n1. This method focuses on generating reliable explanations by prioritizing features the model can reliably detect, which is relatively new in the field;\n\n2. The manuscript is overall well-written, the pipeline figures clear and easy to understand.\n\nCons:\n\n1. The context don't elaborate on how to group features as strong or weak, is there a selected threshold for the computed balance accuracy? If yes, there should be ablation study regarding this matter.\n\n2. The current weak feature supplementing module relies on the blending factor as additional classification clue, why specifically use this? There are also other low-level clues in previous works that can be used for classification (lightening or shadow as shown in Fig. 2). Would it be better to use others or combine them? More justifications are required in here.\n\n3. How about the results of using MLLMs finetuning only with the original dataset for comparison?\n\n4. It is suggested to put the zero-shot acc. of the MLLM in the manuscript instead of the supplementary material.\n\nQUESTIONS:\nSee weakness\n\nLIMITATIONS:\nThe dataset construction process in the proposed framework relies on human verification to ensure the reliability of identified weak features, which can be labor-intensive and time-consuming for large datasets. This dependency on human intervention may introduce scalability challenges and could potentially limit the framework’s performance due to human subjectivity or errors in feature validation.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces X2-DFD, a novel framework for explainable and extendable deepfake detection that leverages Multimodal Large-Language Models (MLLMs). The core idea is to strategically utilize MLLM strengths while compensating for their inherent weaknesses, significantly enhancing both the detectability and explainability of deepfake images. The framework's modular design allows for seamless integration with future, more advanced MLLMs and Specific Feature Detectors (SFDs), ensuring its continuous improvement and scalability in the face of rapidly evolving deepfake challenges.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n(1)Seamless SFD Integration: X2-DFD's modular design allows for the seamless integration of Specific Feature Detectors (SFDs). This modularity is crucial in the rapidly developing field of generative AI, ensuring the framework's continuous improvement and adaptability to new deepfake techniques.\n\n(2)Comprehensive Experiments and Clear Explanations: The paper presents comprehensive experiments covering both deepfake detection and explanation.\n\n(3)Novel and potential Core Idea: The methodology behind X2-DFD is highly reasonable and innovative. The approach of leveraging MLLM strengths while compensating for their weaknesses offers significant potential for future framework expansion and maybe a fundamental solution for both deepfake detection and explanation.\n \nWeaknesses\n\n(1)Limited Scope for Forgeries: The current framework primarily demonstrates strong performance in face-swapping. However, the exploration of more advanced techniques like Entire Face Synthesis (EFS) forgeries, particularly those generated by diffusion models, is relatively limited. Expanding the training dataset to include these latest forgery types would be beneficial.\n\n(2)Underexplored Simultaneous Use of Multiple SFDs: While the authors comprehensively discuss SFD integration, the paper does not explore or discuss scenarios involving the simultaneous use of multiple SFDs. Integrating multiple \"expert\" SFDs concurrently could offer a more robust solution for detecting and explaining diverse types of fakes.\n\n(3)Minor Writing Issue: In Section 5, \"Ablation Study and Analysis,\" the phrasing of the question \"How to ensure WFS module\" should be rephrased for clarity.\n\nQUESTIONS:\n（1）How will the training dataset be expanded to include newer deepfake types (e.g., from diffusion models) to test the framework's real-world applicability? (Refers to Weakness 1)\n\n（2）What are the plans for simultaneously integrating multiple SFDs, and what are the expected benefits or challenges? (Refers to Weakness 2)\n\nIf both questions are addressed well, I will consider increasing the score.\n\nLIMITATIONS:\nX2-DFD mainly focuses on face-swapping deepfakes; its effectiveness with full image generation (e.g., diffusion models) is less explored. Like the author mentioned implementation focuses solely on static image detection, the framework could expand to other data types like audio or audio-visual forgery. However, for face forgery images discussed in paper, the framework is very comprehensive.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a multi-stage approach for leveraging MLLMs for deefpake detection.\n \nFirst, a frozen MLLM is asked to generate a list of questions for assessing whether some deepfake characteristics are present in an image (ie skin color consistency, mouth blurriness...). Then each of these questions is passed to a frozen along with deepfake data MLLM to observe whether the MLLMs is capable to detect the characteristics described in these questions (blurriness, consistency... the authors call them artifacts or sometimes features, which confusing with respect to actual features as in a feature space).\nThis assessment helps defining what questions/artifacts the MLLM is capable of detecting in a zeroshot way. The questions then are ranked and a top-k are selected to stage 2.\nStage 2 consists in using the questions of stage 1 to generate textual prompts (real and fake) to augment some deepfake dataset.\nThis helps building a VQA-style dataset that can be used to finetune an MLLM and making better what it already is. The intuition of finetuning is since the MLLMs is already good in zeroshot on these question, finetuning, would make it better.\nSince the MLLM cannot pick up all possible artifact, the authors incorporate the predictions of a blending-based detector (the fake-it-till-you-make-it one) to compensate for the lack of performance of the MLLM for some questions/artifacts. The output of this specialized detector is also used as an annotation.\n The final model is finetuned using the VQA style built dataset which contains the initial textual prompts from stage 2 and the score of the specialized detecto\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Extensive implementation details.\n- Several datasets are considered, including DF40\n\nWeaknesses:\n- Unclear added scientific value/novelty, although the work introduces a sophisticated pipeline for explainable forgery detection, it remains incremental and mostly seem like an engineering solution rather than a research one\n- In the related works, the limitations of existing benchmarks on forgery detection using VQA i.e. [33,19,29] are unclear.\n- Several typos: Fig 1 caption: \"improvsed\", L38 \"relies\", L82 \"there\", L170, L173, \\mathcal{M} is undefined.\n- The writing style is often vague, for examples the fact of referring to the questions/artifacts learned by the model as \"features\", or statements like \"learning to learn\" etc, so, overall the writing could benefit from more clarity.\n- The reliance on a blending-based detector as the dedicated detectors assumes that MLLMs are only bad in blending-based forgeries, whereas they can be bad in on other types of artifacts.\n- The performance on CDF, DFD and DFDC is extremely close LAA-Net which is a simpler blending-based approach. This makes the proposed very complex in comparison to dedicated detector, which only requires generating self-blended images on the go.\n- Too many acronyms making the paper hard to read\n- It is unclear whether the generated question are reliable, MLLMs are can often give different answers making the annotations and mostly random. Was there some process that ensured that the questions being ranked are actually reliable, like some sort of asking the same question 100 times and average over the answers?\n- The claims on extendability are not convincing and seem overstated, if by being extendable the authors mean using off-the-shelf existing approaches to generate pseudo-labels, build a dataset and train an existing architecture, then previous methods such as Face-Xray and sbi are also extendable.\n- The complexity of the proposed approach is unclear.\n\nQUESTIONS:\nOther/questions:\nIt would be nice if Tab2 indicated an arrow regarding the metrics to indicated whether higher/lower is better \nWhat would be the results if the model used LAA-Net as a dedicated detector (although this still remains within the assumption that MLLMs are only performing badly on blending-based artifacts)\nWhy is \"Photoshop traces\" irrelevant aren't Adobe using AI in their software nowadays?\nWhy is the title including \\chi as X, it sounds phonetically wrong.\nWhy is the work does not generalize to other types of deepfakes, doesn't this mean that the generated questions are not actually indicative enough of deepfake artifacts, or that maybe they are biased towards faceswap?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a detailed exploration of the potential of MLLM for deepfake detection. It focuses on one of the main challenges in this area — how to build a good image-text dataset. The author raise two central questions — whether MLLM can interpret forgery clue descriptions generated by humans or GPT, and which types of clues are more effective — and provide preliminary experimental validation. They further propose enhancing the model's strong features while supplementing weak ones using external deepfake detectors. A concrete framework is introduced and validated through extensive experiments.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n1. The paper suggests a good way to generate text descriptions of forgery features with little manual effort, which helps solve a key problem when using MLLM for deepfake detection.\n2. It presents a well-designed integration framework that leverages the complementary strengths of MLLM and Specific Feature Detector.\n3. The paper conducts thorough ablation studies to demonstrate the significance and effectiveness of feature selection and other individual components.\n4. The paper is well written, easy to follow, and the ideas are clearly explained.\n\n### Weaknesses\n1. The overall performance boost comes from using the SFD module. It’s not very clear if MLLM alone can really beat traditional deepfake detection models.\n2. The method works well on datasets with clear fake artifacts, but it may not be strong enough for high-quality fakes, like those made by diffusion models. This is understandable, but still a limitation.\n\nQUESTIONS:\n1. Why is fine-tuning MLLM only with strong features more effective than using all available features? The paper demonstrates this via experiments, but there’s not much explanation behind it.\n2. As noted, the proposed method still heavily relies on the capabilities of specific detectors. Could the authors analyze the deeper reason behind this? Is it because the visual encoder of the MLLM fails to capture fine-grained forgery cues?\n3. For deepfake images/videos without obvious artifacts, how can one obtain accurate and reliable textual descriptions to effectively utilize the capabilities of MLLM?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-02"}