Paper: AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors (ICLR-2024)

Reviewer comment (Novelty):
The contributions are somewhat limited. The prompt tuning technique employed in this paper is not a highly original advanced method, and it lacks sufficient adaptation and analysis in the downstream task, i.e., fake image detection. In other words, the proposed method could be applied to various other visual tasks, raising doubts about its specific contribution to fake image detection.

Author response:
**1. [The contributions are somewhat limited.]**
- This comment differs from all the other reviewers, who agree on our contributions are "novel" (Reviewer NiiK), “innovative” (Reviewer eMV8).
- First, we think that there may be a misunderstanding of our goal. Our goal is to find a more general solution to deepfake detection problem, and then chose VLM as the backbone model. Hence, the review "the proposed method could be applied to various other visual tasks" is out of our original motivation, making it not a strong reason for claiming limited contribution.
- It is true that prompt tuning technique is not a highly original advanced method. However, we pioneer to employ prompt tuning technique on VLM, and propose a simple yet effective solution to deepfake detection, which is a "novel attempt in this area" (NiiK). In Abstract section in the revised submission, our method demonstrates better performance on held-out datasets comparing to the baselines (61.79 to 92.72) with less learnable parameters (~4M vs. 4.96K), highlighting the resource efficiency and effectiveness of our method. This is a non-trivial and significant contribution on its own.
- We respectfully disagree about the comment of "it lacks sufficient adaptation and analysis in the downstream task". As mentioned in Section 3 in the original submission, we carefully formulate the deepfake detection into a visual question answering problem with clear definition and equations. Also, in Section 4.3 in the original submission, we conducted and analyzed several ablation studies (e.g. position of pseudo-word, prompt tuning for Qformer or LLM, number of training images), demostatrating our extensive experiments and complete analysis.

Reviewer comment (Experiments):
The experiments lack comparison with SOTA baselines, such as [1], which also focuses on developing a general diffusion detector. Furthermore, there are numerous methods in the field of deepfake detection that specialize in generalization across various forgery types, and these should also be considered in this paper.

Author response:
**2. [The experiments lack comparison with SOTA baselines]**
- Thank you for your comment. We sincerely hope that you can provide the reference of [1] so that we can compare it with our method as soon as possible.
- We chose Wang 2020 and DE-FAKE as the baselines due to their highlight in generalizability and their representativeness in this field. Furthermore, following the comments by reviewer eMV8, we have introduced the results of other deepfake detections methods in Table 1 in the **revised submission**.

Reviewer comment (Experiments):
Despite the authors creating their own dataset, it is advisable to validate their method on other benchmark datasets, such as the one introduced by De-fake. Additionally, the fifth category in the dataset constructed in this paper, namely "Deeperforensics," should be accurately labeled as "Deepfakes" or "Face Swap."

Author response:
**3. [Validate our method on other benchmark datasets]**
- Thank you for the advice. As the exact testing dataset of DE-FAKE is not open-sourced, in our original submission we have done our best to follow their experimental setting to adopt COCO, Flickr, LD, SD, and DALLE in our testing dataset except for GLIDE. In order to better fulfill the reviewer's suggestion, we then further introduce GLIDE to our testing dataset with providing the corresponding in the Table 1 of the **revised submission**. 
- What's more, both DE-FAKE and our method are trained on COCO+SD2, while our method is tested on more difficult and more diverse types of datasets (e.g. the held-out testing datasets from image inpainting, super resolution, image attacks, etc), which make our superior experimental results more conclusive and convincing.

Reviewer comment (Novelty):
Enriching the experiments can enhance the quality of this paper. However, as a detection method, the innovativeness of the approach must be a crucial consideration.

Author response:
**1. [Lack of significant technical contribution.]**

We understand this is a subjective judgement, and the debate would never end. We would rather demonstrate our contributions by proving the novelty of the proposed method using Prof. Michael Black’s interpretation about novelty from [Novelty in Science](https://medium.com/@black_51980/novelty-in-science-8f1fd1a0a143). We quote some:

- About the novelty of our **connection between two established fields, visual question answering and deepfake detection**, Prof. Michael Black says 
    - “The novelty arose from the fact that nobody had put these ideas together before.”
    -  “To see the connections for the first time, before others saw them, was like breathing for the first time.” 
    - “The resulting paper embodies the translation of the idea into code, experiments, and text. In this translation, the beauty of the spark may be only dimly glimpsed. My request of reviewers is to try to imagine the darkness before the spark.”
- We pioneer to formulate the deepfake detection problem as a visual question answering problem, in which no prior work has put these two ideas together, making the proposed method an impressing "spark" in this field.
- About the novelty of our **simple and reasonable reuse of VLM and prompt tuning** on deepfake detection problem, Prof. Michael Black says 
    - “I value simplicity over unnecessary complexity; the simpler the better. Taking an existing network and replacing one thing is better science than concocting a whole new network just to make it look more complex.” 
    - “If a paper has a simple idea that works better than the state of the art, then it is most likely not trivial. The authors are onto something and the field will be interested.” 
    - “The inventive novelty was to have the idea in the first place. If it is easy to explain and obvious in hindsight, this in no way diminishes the creativity (and novelty) of the idea.”
- The proposed method simply adds a pseudo-word and optimizes the word embedding, and then reaches the SOTA performance on various types of generated images, making it not a trivial idea, while also demonstrating the simplicity and effectiveness of the idea.
- Additionally, the proposed idea is **“straightforward”** and **“effective”** (by Reviewer 4Y5R), which supports the value of the proposed method as well.

We therefore appeal the reviewer’s re-evaluation about the “contribution” rating and final rating.

Reviewer comment (Evaluation):
The authors should delve deeper into the analysis of the characteristics of forged images to guide the judgment of VLM.

Author response:
**2. [Lack of in-depth analysis of deepfake characteristics.]**

- Thank you for your comment. Actually, we have conducted some analysis on the prompt-tuned text (pseudo-word). We first used t-SNE to reduce the dimensionality of every word embeddings into 2-dimension representations. Then, we found some nearest neighbors of the pseudo-word embedding using Euclidean distance metric, trying to figure out the closest meaning of the pseudo-word. However, most of the neighbors were meaningless symbols or punctuation marks such as "-", "of", "and" or even unrelated words in other languages.
- We also conducted some analysis on the visual features. We have used GradCAM to analyze the important region in the images. Although we did observe AntifakePrompt focused on certain region in some samples to consider them as fake, we have not found any significant or consistent visual characteristic throughout all GradCAM results of our samples to draw a concrete conclusion.
- What's more, since soft prompt tuning technique aims to tune the continuous embedding for the simplicity of optimizing, it is hard and meaningless to find a corresponding human-readable word of the tuned embedding.
- Therefore, following [1][2], we demonstrate the effectiveness of the proposed method by analyzing the outstanding experimental results instead of the meaning of the pseudo-word.

[1] Chen, Lichang, et al. "InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models." arXiv preprint arXiv:2306.03082 (2023).

[2] Zou, Andy, et al. "Universal and transferable adversarial attacks on aligned language models." arXiv preprint arXiv:2307.15043 (2023).

Reviewer comment (Experiments):
Deepfake detection is a subject of extensive research with many related papers available. Research works that address the performance degradation on the Diffusion models, and across various cross-datasets, are not incorporated [1,2,3,4,5]. There are also studies that deal with the detection of low-quality, low-resolution deepfakes [6]. There is a need to consider those and compare analysis with other studies.

Author response:
**1. [Compare analysis with other studies]**
- Thank you for suggesting these related works. The reason why we chose the two baselines (Wang2020 & DE-FAKE) is that both works highlighted their generalizability, and they are well-known or SOTA method in this field before the submission.
- We investigated the listed related works: [1] proposes a detection method using reverse and denoising computation error of intermediate steps; [2] proposes a method via language-guided contrastive learning; [3] proposes a method related to diffusion reconstruction error at the initial timestep; [4] proposes a method using their local intrinsic dimensionality; [5] conducts extensive experiments on many different datasets; [6] proposes a collaborative learning framework to conduct detection of different quality of deepfakes.
- We follow reviewer's suggestion to include the comparison with the methods of [2,3,5,6], where the experimental results and the corresponding analysis are added to Table 1 and Section 4.2 respectively in the **revised submission**. We did not do experiments on [1,4] since they are regrettably not open-sourced, but we will introduce the experimental results into our final version as long as their codes are published.

Reviewer comment (Experiments):
The test dataset used in the paper is biased towards diffusion model-generated data. It would be great to evaluate the performance with well-known other deepfake datasets such as DFC[7], DFDC[8], and FF++[9].

Author response:
**2. [Test dataset is biased towards diffusion model-generated data.]**
- Thank you for pointing this out. Since diffusion-based models are known for their outstanding ability of generating more genuine and more photorealistic images than those from GAN-based models, we naturally incoporated more diffusion-based models to generate various types of fake images to conduct more persuasive evaluations of our method and other baseline methods. Moreover, we chose DeeperForensics instead of other deepfake datasets because it is a newer and thus more difficult dataset than DFDC and FF++ datasets, which makes our performance more convincing. Out of similar reasons, we selected SGXL to represent GAN-based models. The rationale is that SGXL can generate images with lower FID and higher IS score than those generated by other GAN-based methods.
- Also, we have followed your suggestion to test the methods on DFDC and FF++. The results are presented in Table 1 in the **revised submission**.
- The reason why we did not test on DFC is that the link of DFC directs us to the dataset of DFDC, so we considered DFC and DFDC as the same dataset. If there is any misunderstanding, please let us know.

Reviewer comment (Experiments):
The authors compared the proposed method with existing methods (i.e. Wang 2020 and DE-FAKE), the pretrained vision-language model without finetuning, and the pretrained model finetuned by LoRA. However, since Wang 2020 is trained on different datasets, the comparison is not fair.

Author response:
**1. [Unfair to compare the two baselines.]**
- The main goal of our work is to propose a solution to deepfake detection with stronger generality, and thus we mainly focus on the performance on the held-out datasets. Therefore, we chose the two baselines (Wang 2020 and DEFAKE) due to their highlighted generalizability. When testing, those held-out datasets are unseen data for both our method and the baselines, so we think it is a fair comparison for our method and the two baselines.
- We respectfully disagree with the idea of changing backbones. Wang 2020 and DE-FAKE use ResNet as their backbone model, while we choose InstructBLIP (VLM) as our backbone model. Because of the difference of input and output between ResNet and InstructBLIP, we think the backbone is not transferable.
- Still, we really appreciated your suggestion about fairness, so we have trained the model of Wang 2020 and our model with a subset (due to the insufficient time) of training data (COCO+SD2, not including SD2IP). We did not train DE-FAKE as its training code is not open-sourced, however please note that official DE-FAKE was actually trained on COCO+SD2 which is the same as our training dataset. The results are shown below:


|                | COCO  | Flickr | SD2   | SDXL  | IF    | DALLE2 | SGXL  |
|:-------------- |:----- |:------ |:----- | ----- | ----- | ------ |:----- |
| Wang 2020      | 99.83 | 99.97  | 66.10 | 35.30 | 30.43 | 5.83   | 12.03 |
| DE-FAKE        | 85.97 | 90.67  | 97.10 | 90.50 | 99.20 | 68.97  | 56.90 |
| AntifakePrompt | 97.73 | 95.20  | 98.33 | 98.13 | 96.57 | 88.30  | 88.00 |

|                | GLIDE | ControlNet | DeeperForensics | DFDC  | FF++  | LaMa  | SD2(IP) |
|:-------------- |:----- |:---------- |:--------------- | ----- |:----- |:----- |:------- |
| Wang 2020      | 10.83 | 6.97       | 0.37            | 0.60  | 0.63  | 0.10  | 1.27    |
| DE-FAKE        | 76.50 | 63.97      | 86.97           | 56.13 | 78.90 | 13.03 | 16.00   |
| AntifakePrompt | 75.57 | 80.50      | 97.13           | 95.10 | 94.47 | 6.37  | 28.73   |

|                | LTE   | SD2(SR) | AdverAtk | BackdoorAtk | DataPoisonAtk | Avg. Acc  |
|:-------------- |:----- |:------- |:-------- | ----------- |:------------- |:--------- |
| Wang 2020      | 0.63  | 4.30    | 0.27     | 0.40        | 1.90          | **19.88** |
| DE-FAKE        | 9.97  | 29.70   | 60.40    | 22.23       | 55.87         | **61.00** |
| AntifakePrompt | 23.80 | 60.47   | 54.10    | 44.07       | 55.07         | **72.51** |


- Following the suggestion of reviewer **eMV8**, we test the methods on more fake datasets (DFDC, FF++). The results demostrate that Wang 2020 suffers from insufficient generalizability, leading to poor performance on held-out datasets (datasets other than COCO & SD2). We think the reason behind is that Wang 2020 chooses ResNet as the backbone, which may result in overfitting to the training due to the large number of trainable parameters (instead, noting that our proposed method does not finetune the entire VLM model but only optimizes for the pseudo-word $S_\ast$).

Reviewer comment (Experiments):
It would be necessary to give the results of Wang 2020 and DE-FAKE with the same training data and backbone as the proposed method.

Author response:
**1. [Unfair to compare the two baselines.]**
- The main goal of our work is to propose a solution to deepfake detection with stronger generality, and thus we mainly focus on the performance on the held-out datasets. Therefore, we chose the two baselines (Wang 2020 and DEFAKE) due to their highlighted generalizability. When testing, those held-out datasets are unseen data for both our method and the baselines, so we think it is a fair comparison for our method and the two baselines.
- We respectfully disagree with the idea of changing backbones. Wang 2020 and DE-FAKE use ResNet as their backbone model, while we choose InstructBLIP (VLM) as our backbone model. Because of the difference of input and output between ResNet and InstructBLIP, we think the backbone is not transferable.
- Still, we really appreciated your suggestion about fairness, so we have trained the model of Wang 2020 and our model with a subset (due to the insufficient time) of training data (COCO+SD2, not including SD2IP). We did not train DE-FAKE as its training code is not open-sourced, however please note that official DE-FAKE was actually trained on COCO+SD2 which is the same as our training dataset. The results are shown below:


|                | COCO  | Flickr | SD2   | SDXL  | IF    | DALLE2 | SGXL  |
|:-------------- |:----- |:------ |:----- | ----- | ----- | ------ |:----- |
| Wang 2020      | 99.83 | 99.97  | 66.10 | 35.30 | 30.43 | 5.83   | 12.03 |
| DE-FAKE        | 85.97 | 90.67  | 97.10 | 90.50 | 99.20 | 68.97  | 56.90 |
| AntifakePrompt | 97.73 | 95.20  | 98.33 | 98.13 | 96.57 | 88.30  | 88.00 |

|                | GLIDE | ControlNet | DeeperForensics | DFDC  | FF++  | LaMa  | SD2(IP) |
|:-------------- |:----- |:---------- |:--------------- | ----- |:----- |:----- |:------- |
| Wang 2020      | 10.83 | 6.97       | 0.37            | 0.60  | 0.63  | 0.10  | 1.27    |
| DE-FAKE        | 76.50 | 63.97      | 86.97           | 56.13 | 78.90 | 13.03 | 16.00   |
| AntifakePrompt | 75.57 | 80.50      | 97.13           | 95.10 | 94.47 | 6.37  | 28.73   |

|                | LTE   | SD2(SR) | AdverAtk | BackdoorAtk | DataPoisonAtk | Avg. Acc  |
|:-------------- |:----- |:------- |:-------- | ----------- |:------------- |:--------- |
| Wang 2020      | 0.63  | 4.30    | 0.27     | 0.40        | 1.90          | **19.88** |
| DE-FAKE        | 9.97  | 29.70   | 60.40    | 22.23       | 55.87         | **61.00** |
| AntifakePrompt | 23.80 | 60.47   | 54.10    | 44.07       | 55.07         | **72.51** |


- Following the suggestion of reviewer **eMV8**, we test the methods on more fake datasets (DFDC, FF++). The results demostrate that Wang 2020 suffers from insufficient generalizability, leading to poor performance on held-out datasets (datasets other than COCO & SD2). We think the reason behind is that Wang 2020 chooses ResNet as the backbone, which may result in overfitting to the training due to the large number of trainable parameters (instead, noting that our proposed method does not finetune the entire VLM model but only optimizes for the pseudo-word $S_\ast$).

Reviewer comment (Evaluation):
It would be nice to see more discussions about the possible reasons for the good performance of the proposed method. For example, where does the performance gain come from — the pretrained image encoder, the training algorithm, or some other possible factors?

Author response:
**2. [Reasons for our good performance]** 
- Thank you for your comment. As mentioned in Section 2.3 in the original submission, VLM helps multimodal comprehension through utilizing the strong generality of LLMs. Specifically, the strong generality of LLM comes from the large scale of its training data (e.g. corpus), enabling the LLM to answer the questions that it has never seen before. Therefore, VLM can also achieve strong generality with the help of its LLM component. This detailed discussion has been added in **red** to Paragraph 5 of Section 1 and Paragraph 4 of Section 4.2 in the **revised submission**.

Reviewer comment (Experiments):
As shown in Table 1, except for the three attack tasks, the LoRA-finetuned model achieves better performance than the proposed method. What’s the possible reason for this phenomenon? Is it possible to obtain a better model by just replacing LoRA with some other finetuning algorithms?

Author response:
**4. [Is it possible to obtain a better model by just replacing LoRA with some other finetuning algorithms?]**
- We choose LoRA as one of our baseline as it is one of the most popular, and the most widely-used parameter-efficient finetuning methods of LLM.
- Nevertheless, we are happy to follow your suggestion by replacing LoRA with another finetuning algorithm, AdaLoRA[1]. The results are shown below:

|                | COCO  | Flickr | SD2   | SDXL  | IF    | DALLE2 | SGXL  |
|:-------------- |:----- |:------ |:----- | ----- | ----- | ------ |:----- |
| LoRA           | 95.73 | 91.83  | 98.03 | 96.33 | 86.60 | 99.57  | 97.67 |
| AdaLoRA        | 96.10 | 88.73  | 97.00 | 95.23 | 88.67 | 99.10  | 98.93 |
| AntifakePrompt | 95.37 | 91.00  | 97.83 | 97.27 | 89.73 | 99.57  | 99.97 |

|                | GLIDE | ControlNet | DeeperForensics | DFDC  | FF++  | LaMa  | SD2(IP) |
| -------------- |:----- |:---------- |:--------------- | ----- |:----- | ----- |:------- |
| LoRA           | 95.9  | 92.87      | 98.80           | 90.03 | 94.70 | 59.50 | 93.03   |
| AdaLoRA        | 96.27 | 90.63      | 98.53           | 99.60 | 95.90 | 38.23 | 84.40   |
| AntifakePrompt | 99.17 | 91.47      | 97.90           | 100   | 97.43 | 39.03 | 85.20   |

|                | LTE   | SD2(SR) | AdverAtk | BackdoorAtk | DataPoisonAtk | Avg. Acc |
| -------------- |:----- |:------- | -------- | ----------- |:------------- | -------- |
| LoRA           | 99.53 | 99.97   | 64.30    | 53.40       | 50.87         | 87.30    |
| AdaLoRA        | 99.33 | 99.83   | 70.53    | 78.30       | 76.57         | 89.05    |
| AntifakePrompt | 99.90 | 99.93   | 96.70    | 93.00       | 91.57         | 92.74    |


- As being observable from the results, AdaLoRA-finetuned model also suffers (as LoRA does) from accuracy drop when applied to three attack tasks, which underscores the generalizability of our proposed AntifakePrompt again.

Reviewer comment (Evaluation):
It seems that directly using average accuracy over different datasets and tasks might not be a proper metric, since different tasks have different numbers of test images and different tasks might have different importance in real-world applications. It would be better if the authors could give some results in other metrics, e.g. weighted average accuracy, AUC, etc.

Author response:
**5. [Using average accuracy over different datasets and tasks might not be a proper metric.]**
- We respectfully disagree. First, it has to be clarified that the numbers of images used in every task are actually the same, so the accuracies can be averaged without any weights. Secondly, the importance of each task can not be formally defined, so we consider every task to have the same importance. Lastly, since we take VLM as our backbone, which directly outputs a text instead of a probability distribution of each class, so the AUC metric may not be suitable in our case.
- Still, we really appreciate your comment upon the relation between the importance of tasks and the weighted accuracy. Therefore, we provide the weighted accuracy, including "Real Accuracy" (average of all real datasets), "Fake Accuracy" (average of all fake datasets), and "Weighted Accuracy" (average of Real Accuracy and Fake Accuracy), considering the same importance of real datasets and fake datasets.

|                | Real Acc | Fake Acc | Weighted Acc |
|:-------------- |:-------- |:-------- |:------------ |
| Wang 2020      | 99.95    | 11.02    | 55.49        |
| DE-FAKE        | 88.32    | 57.78    | 73.05        |
| AntifakePrompt | 93.19    | 92.69    | 92.94        |

- No matter using what kind of metric, our method shows superior performance than the baselines.

Reviewer comment (Experiments):
This is just for discussion: it seems that the proposed method is not limited to the preset question (i.e. Is this photo real) adopted in this manuscript. If we provide more specific information in the question (e.g. Is this photo real or generated by deep learning models), is it possible that the model can attend to more task-specific visual features and achieve better performance under this circumstance? Or is it possible that simpler finetuning techniques might achieve similar performance as the proposed one in this situation?

Author response:
**6. [Other preset questions may achieve better performance?]**
- The influence of the preset question is quite small. Actually, we have tried different preset question in Section 4.3 in the original paper, namely "prefix" ("$S_*$ *Is this photo real?*"), "postfix" ("*Is this photo real* $S_*$?"), and "replace" ("*Is this photo* $S_*$?"). The experimental results of "postfix" and "replace", which have different preset questions, show that the accuracy (88.94 / 91.59) is not sensitive to the preset question or the position of the pseudo-word.
- Still, we also conduct more experiments on other preset questions during rebuttal, which we set the question to be "*Is this photo real or generated by deep learning models* $S_*$?" (denoted as "Extended" in the results below) and "$S_*$" (denoted as "One word" in the results below), and here are the results: 

|                | COCO  | Flickr | SD2   | SDXL  | IF    | DALLE2 | SGXL  |
|:-------------- |:----- |:------ |:----- | ----- | ----- | ------ |:----- |
| Extended       | 95.30 | 91.50  | 98.10 | 97.73 | 92.13 | 98.70  | 99.43 |
| One word       | 95.77 | 89.70  | 97.10 | 96.40 | 89.13 | 99.40  | 99.40 |
| AntifakePrompt | 95.37 | 91.00  | 97.83 | 97.27 | 89.73 | 99.57  | 99.97 |

|                | GLIDE | ControlNet | DeeperForensics | DFDC   | FF++  | LaMa  | SD2(IP) |
|:-------------- |:----- |:---------- |:--------------- |:------ |:----- | ----- |:------- |
| Exetended      | 99.17 | 92.53      | 99.83           | 99.97  | 99.40 | 38.80 | 84.47   |
| One word       | 98.73 | 89.47      | 100.00          | 100.00 | 98.30 | 36.07 | 80.03   |
| AntifakePrompt | 99.17 | 91.47      | 97.90           | 100.00 | 97.43 | 39.03 | 85.20   |

|                | LTE   | SD2(SR) | AdverAtk | BackdoorAtk | DataPoisonAtk | Avg. Acc  |
|:-------------- |:----- |:------- | -------- | ----------- |:------------- |:--------- |
| Extended       | 99.97 | 99.93   | 85.60    | 84.33       | 74.73         | **91.14** |
| One word       | 99.97 | 99.80   | 96.67    | 86.67       | 83.70         | **91.45** |
| AntifakePrompt | 99.90 | 99.93   | 96.70    | 93.00       | 91.57         | **92.74** |

- The difference between the three average accuracies is less than 2%, which implies that the performance is not sensitive to the preset question.