Paper: An Image Is Worth 1000 Lies: Transferability of Adversarial Images across Prompts on Vision-Language Models (ICLR-2024)

Reviewer comment (Experiments):
The paper assumes the white-box access for VLMs, which enables the framework to have backward gradients to update in multimodal input space. However, this assumption may not be generalized to all mainstream VLMs as they are fast scaling up. For instance, the authors evaluated the effectiveness of OpenFlamingo-9B and InstructBLIP, whose LLM component is not of large-scale. The applicability of CroPA on large-scale VLMs (e.g., those embedded with LLaMA-65B) or black-box VLMs (e.g., GPT4-V) remains challenging.

Author response:
**Q1.1: The method may not be generalized to all mainstream VLMs as they are fast scaling up.**

**A:** This is a good question. We discuss the challenges brought by scaling up the models in the following two perspectives.
1. Accurate gradient: With larger VLMs, it can be more challenging to obtain accurate gradient information. In our community, several approaches have also been proposed to address this problem such as EOT [1]. These approaches can be combined to effectively address the gradient accuracy challenge in large VLMs. In this work, we applied the standard attack method. 
2.  The memory issue caused by large VLMs: our community has actively proposed methods to allow for the usage of increasingly large models. Hugging Face, for instance, allows users to load models by distributing model shards across multiple graphics cards.

**Q1.2:   The applicability of CroPA under the black-box setting**

**A:** Achieving strong transferability across models is indeed challenging. We also studied the performance under the black-box setting. Our experimental results in Table 7 in Appendix C indicate an ASR of close to 10% for transferability across models, despite differences in architecture and size, such as between OPT-2.7b and vicuna-7b. We expect the cross-model transferability to be stronger if the language model components are highly similar (e.g. one model is the instruction fine-tuned variant of the source model). Our study is expected to have broad applications as many models are built based on prevalent open-source vision models (e.g. OpenCLIP) and language models (e.g. LlaMA).

Reviewer comment (Evaluation):
The framework learns adversarial prompts to enhance the attack's effectiveness. However, very few text prompt instances are shown in the paper, and the quality of the result text prompts is not sufficiently evaluated.

Author response:
**Q2: The framework learns adversarial prompts to enhance the attack's effectiveness. However, very few text prompt instances are shown in the paper, and the quality of the result text prompts is not sufficiently evaluated.**

**A:** Thank the reviewer for pointing this out. The adversarial prompts are learnable embeddings instead of tokens. The following are more detailed clarifications regarding this question: 
1. In optimization, we update the embedding of the prompt perturbations instead of updating the discrete tokens of the prompts. That's the reason why we are not able to show the adversarial prompts directly.
2. We also decoded the perturbed prompt embeddings by approximating the tokens in terms of cosine similarity. The approximate tokens of the perturbed prompt embeddings are the same as the original prompts, which can be found in Section 4.6.

Reviewer comment (Experiments):
Existing adversarial attacks on VLM [1] have adopted query-based techniques to enhance the attack effectiveness. What's more, this baseline only assumes the black-box access to the VLMs, which more are generalizable. The paper should show sufficient validity or advantages (e.g., ASR, Convergence, etc.) of CroPA over existing baselines.

Author response:
**Q3: Discussion about the difference with another work [2] under the black-box settings.**

**A:** The study [2] under the black box settings indeed provides meaningful insights into the cross-model transferability, but our work focuses on a new perspective transferability across. Our works differ in the following aspects.

1. The goals of these two studies are different. The study [2] explores the adversarial robustness of VLMs from an evaluation perspective. It cannot be trivially applied to solve the cross-prompt transferability, which is the focus of this paper.

2. The evaluation methods in the two papers are different as well. The ASR (Attack Success Rate) discussed in our paper adheres to a strict standard: an attack is considered successful only if the model is deceived into generating the exact target text. The evaluation metric used in [2] is CLIP-score, which does not require the exact match of the generated text and the target text.

Moreover, our method makes sense in both black-box and white-box. The black-box attacks on VLMs can be combined with our CroPA method to enable our methods to work in a black-box fashion.

Reviewer comment (Writing):
Please fix the typo "leanable" in the introduction (page 2).

Author response:
**Q4: Typo "leanable" in the introduction (page 2)**

**A:** Thanks for pointing this out, and we have corrected the typo.

Reviewer comment (Evaluation):
The primary concern for this paper is whether the proposed attack can indeed raise significant security issues. For instance, when considering the exemplary target prompts in Table 1, they have the potential to disrupt the functionality of VLMs, but it remains unclear how they may pose security risks in real-world applications. This security concern would be more valid if the attack successfully tricks VLMs into producing contradictory predictions or harmful instructions. Otherwise, the objective of this study appears to be more like a pioneering exploration of VLMs' adversarial robustness.

Author response:
**Q1: Whether the proposed attack can indeed raise significant security issues？ This security concern would be more valid if the attack successfully tricks VLMs into producing contradictory predictions or harmful instructions.**

**A:** We agree that tricking the model into generating harmful instructions is indeed a security concern, but our goal is different. The goal of this paper is to study the adversarial transferability across prompts of the VLMs. 

There is some connection between the two topics. When the target output is set to harmful content, such as  “suicide”, the model can output inappropriate sentences for images.

The detailed data are shown as follows, which is also presented in Table 9 in Appendix C. 

| Target Texts | Method | VQA_general | VQA_specific | Classification | Captioning | Overall |
|--------------|--------|-------------|--------------|----------------|------------|---------|
| suicide      | Multi-P| 0.69        | 0.83         | 0.75           | 0.61       | 0.72    |
|              | CroPA  | **0.84**    | **0.91**     | **0.91**       | **0.78**   | **0.86**|
| kidnap       | Multi-P| 0.87        | 0.92         | 0.69           | 0.73       | 0.80    |
|              | CroPA  | **0.94**    | **0.96**     | **0.88**       | **0.81**   | **0.90**|
| bomb         | Multi-P| 0.68        | 0.82         | 0.90           | 0.60       | 0.75    |
|              | CroPA  | **0.80**    | **0.90**     | **0.94**       | **0.70**   | **0.84**|



Therefore, setting the target sentence to harmful words can pose threats to the applications of large VLMs. We would like to point out again that harmful content generation is not the focus of this work. But, we agree it is interesting to discuss the relationship between them.

Reviewer comment (Experiments):
The author mentions another potential use case for the proposed framework, which is to "prevent malicious usage of large VLMs for unauthorized extraction of sensitive information from personal images." If this is indeed the case, the proposed method should also be assessed in a black-box setting, as users' VLMs cannot be known in advance. However, this set of experiments is absent from the paper. Previous jailbreaking attacks aimed at emerging large language models have shown substantial transferability across models [1]. I am curious whether this holds true for various VLMs as well.

Author response:
**Q2: Questions about the statement "prevent malicious usage of large VLMs for unauthorized extraction of sensitive information from personal images" and comparisons with the transferability across models in the context of jailbreak.**

**A:** Thank you for highlighting this aspect.
1. The white-box setting also makes sense for this statement.  For example, staff in technology companies may have access to users’ images, and there is potential for them to maliciously use their company’s models to extract users' information. Our method can serve as a privacy protection method if the images are pre-processed by adding invisible perturbations designed for their company’s models.

2. Our setting is much more stringent than the jailbreak scenarios in Large Language Models (LLMs). In this work, we assume that only images can be manipulated while the jailbreak-related study in LLMs allows for modifying prompts directly. Manipulating VLM outputs through images is challenging due to the difficulty in aligning visual information with text, as detailed in [1].  It is also not entirely clear how to achieve jailbreaking by modifying images only. However, it is a really good question to explore in our future work.

Reviewer comment (Experiments):
Is the improvement achieved by the proposed Cross-Prompt Attack over Multi-P due to the increased number of textual prompts generated through perturbations during minimax optimization? One approach to achieve a similar effect is to utilize another language model to rephrase the text prompts into new ones, thus serving as data augmentation.

Author response:
**Q4: Is the improvement achieved by CroPA due to the increased number of textual prompts generated through perturbations? One approach to achieve a similar effect is to utilize another language model to rephrase the text prompts into new ones, thus serving as data augmentation.**

**A:** This is a very good question. 
1. Firstly, as shown in Figure 2 in the paper, the improvement from adding more prompts quickly saturates. Beyond the point where the number of prompts reaches ten, the increase in cross-prompt transferability becomes marginal, especially for Multi-P. Therefore, using more prompts does not necessarily mean a stronger performance.
2. Given the same number of prompts and the same number of optimization iterations as shown in Figure 3 in the paper, the proposed CroPA method clearly outperforms the Multi-P method.

Reviewer comment (Evaluation):
The method does not achieve transferability across models or images in addition to cross-prompt transferability. This might limit its practical applicability.

Author response:
**A:** Thank you for your insightful observation regarding the transferability in other aspects. We appreciate the opportunity to clarify and expand upon these points.
1. The transferability across different images is orthogonal to the transferability across different prompts. Transferability across prompts can be combined with transferability across images by utilizing more images during the optimization stage. As stated in the paper [1], the perturbation computed over a larger dataset can increase the cross-image transferability. 

2. The transferability across different models depends on the similarities between the model architecture. Our study is expected to have broad application as many models are built based on common open-source vision models (e.g. OpenCLIP) and language models (e.g. LlaMA).

Reviewer comment (Reproducibility):
Clarifications are required in some parts of the paper. Certain statements about the “textual prompts” are misleading. For example, in the abstract, the method is described as follows: “This proposed method updates the visual adversarial perturbation with learnable textual prompts”. This might suggest that the method directly modifies the textual prompts or discrete prompt tokens whereas it is actually modifying the embedding of these textual prompts. Moreover, the authors should explicitly state the underlying optimization algorithm (is it projected gradient descent?), the norm (is it L-infinity?), and the selected value of the perturbation size for easier comparison with related work.

Author response:
**A:** Thanks for pointing this out. We agree with the reviewer about the potential misunderstanding. Following the suggestion, we replaced "learnable textual prompts" with "learnable prompts" across the paper. 

**A:** We follow the standard setting, and adopt the PGD as our optimization algorithm with L-infinity norm. The perturbation size is fixed to 16/255. We have updated our paper accordingly. We added the ablation experiments with different perturbation sizes as shown in Table 7 in Appendix C.
| Perturbation size | Method | VQA_general | VQA_specific | Classification | Captioning | Overall |
|-------------------|--------|-------------|--------------|----------------|------------|---------|
| 8/255             | Multi-P| 0.37        | 0.59         | **0.56**       | **0.06**   | 0.39    |
|                   | CroPA  | **0.53**    | **0.75**     | 0.48           | 0.03       | **0.45**|
| 16/255            | Multi-P| 0.67        | 0.86         | 0.64           | 0.31       | 0.62    |
|                   | CroPA  | **0.92**    | **0.98**     | **0.70**       | **0.39**   | **0.75**|
| 32/255            | Multi-P| 0.85        | 0.95         | 0.47           | 0.32       | 0.64    |
|                   | CroPA  | **0.98**    | **0.99**     | **0.59**       | **0.42**   | **0.75**|

Reviewer comment (Writing):
There are some minor issues in the paper such as typos (for example, in the conclusion “baseline approaches only archive” → “achieve”).

Author response:
**A:** We have fixed this typo, thanks for the correction.

Reviewer comment (Experiments):
What is the ASR of Multi-P when transferred across models or images?

Author response:
**A:** The ASRs of the Multi-P are all zeros when transferred across images. The ASRs of Multi-P across different images are around 3% and the detailed data can be found in Table 8 in Appendix C. As the table shows, the Multi-P underperforms CroPA, even though the cross-model transferability performance of CroPA is limited. Please note that the techniques for enhancing cross-image transferability are orthogonal to our study, and they can be combined with our method.