An Image Is Worth 1000 Lies: Transferability of Adversarial Images across Prompts on Vision-Language Models
Authors: Haochen Luo
Jindong Gu
Fengyuan Liu
Philip Torr
Venue: ICLR 2024 spotlight (2024)
Keywords: Vision Language Model
Adversarial Transferability
Prompt Tuning
Primary area: societal considerations including fairness, safety, privacy

Abstract: Different from traditional task-specific vision models, recent large VLMs can readily adapt to different vision tasks by simply using different textual instructions, i.e., prompts. However, a well-known concern about traditional task-specific vision models is that they can be misled by imperceptible adversarial perturbations. Furthermore, the concern is exacerbated by the phenomenon that the same adversarial perturbations can fool different task-specific models. Given that VLMs rely on prompts to adapt to different tasks, an intriguing question emerges: Can a single adversarial image mislead all predictions of VLMs when a thousand different prompts are given? This question essentially introduces a novel perspective on adversarial transferability: cross-prompt adversarial transferability. In this work, we propose the Cross-Prompt Attack (CroPA). This proposed method updates the visual adversarial perturbation with learnable textual prompts, which are designed to counteract the misleading effects of the adversarial image. By doing this, CroPA significantly improves the transferability of adversarial examples across prompts. Extensive experiments are conducted to verify the strong cross-prompt adversarial transferability of CroPA with prevalent VLMs including Flamingo, BLIP-2, and InstructBLIP in various different tasks.

Published as a conference paper at ICLR 2024
AN IMAGE Is WORTH 1000 LIES: ADVERSARIAL TRANSFERABILITY ACROSS PROMPTS ON VISION- LANGUAGE MODELS
Haochen Luo", Jindong Gu'f, Fengyuan Liu, Philip Torr Torr Vision Group, University of Oxford
Parks Road, Oxford OXI 3PJ, UK haochen. tuo@outlood com, jindong .wrtoonggguuuuue out look com
ABSTRACT
Different from traditional task-specific vision models, recent large VLMs readily adapt to different vision tasks by simply using different textual instruc- io.,, prompts. However, a well-known concern about traditional task-specific vision models is that they can be misled by imperceptible adversarial pertur- bations. Furthermore, the concern exacerbated by the phenomenon that the same adversarial perturbations can fool different task-specific models. Given that VLMs rely on prompts to adapt to different tasks, an intriguing question emerges: Can single adversarial image mislead all predictions of VLMs when thou- sand different prompts are given? This question essentially introduces a novel perspective adversarial transferability: cross- pross-prompt adversarial transferabil- ity. In this work, we propose the Cross-Prompt Attack (CroPA). This proposed method updates the visual adversarial perturbation with learnable prompts, which are designed to counteract the mislcading effects of the ad adversarial image. By doing this, CroPA significantly improves the transferability of adversarial exam- ples across prompts. Extensive experiments are conducted to verify the strong cross-prompe adversarial transferability of CroPA with prevalent VLMs including Flamingo, BLIP-2, and InstructBLIP in various different tasks. Our source code is available ht/httss///gthha com www.oooooooeeeeeoooooooooocceeee Luo/CroPA
INTRODUCTION
Previously task-specific vision models have demonstrated remarkable capabilities in various visual tasks such as image classification (He et al., 2016) and image captioning (Yao et al.] 2018 Yang (et al. 2019). These models are designed to extract specific information which is pre-defined during the construction phase of the model. Recently, large Vision-Language Models (VLMs) (Gu et al. 2023a Li al. 2022; Alayrac et al. 2022 Li et al. 2023 Zhu et al. 2023) have emerged, providing more unified approach to addressing computer vision tasks. Instead of relying merely the image input, VLMMs integrate information from both images and associated textual prompts, enabling them to perform varied vision-related tasks by utilizing appropriate prompts. This versatility paves the way for exploring various visual tasks.
Those task-specific models are known to be vulnerable to adversarial examples (Goodfellow al. 2014 Szegedy et al. 20133; Wu et al. 2022 Gu et al., (002).. These examples are developed by adding subtle perturbations, typically invisible humans, original samples. These perturba- tions can significantly degrade the performance of such models significantly (Madry et al. 2017). Compounding the mentioned vulnerabilities, the adversarial examples further exhibit transferability across models (Gu et al.) 2023b). This implies that these adversarial examples can attack target
models beyond those for which they were specifically crafted (Liu et al. 2016 Tramer al. 2017).
Given that adversarial examples have transferability across tasks (Salzmann et al. 2021 Gu et al.) 2023b), an interesting question emerges: an adversarial example transferable across prompts? We
"qqual contribution, +Corresponding author

---

Published as a conference paper at ICLR 2024
dub it pross-prompt adversarial transferabilityy, which means that regardless of the prompts provided, the model output can consistently be misled by an adversarial example. For example, if the target text is set to "unknown", model, deceived by an adversarial example exhibiting cross prompt transferabilityy, will always predict the word "unknown" regardless of the prompts. In this case, VLMM are incapable of extracting information from the image, even when different textual prompts are presented.
Exploring cross-prompt adversarial examples is crucial to revealing the prompt-related vulnerability of VLM and protecting image information. From the perspective of an attacker, adversarial examples with high cross-prompt transferability can mislead large VLMs to generate malicious outputs even when queried with various benign prompt questions. From the defensive perspective, the potential to obfuscate image information through human-imperceptible perturbations can cause the model to uniformly output a predefined target text. This can prevent malicious usage of large VLMS for unauthorized extraction of sensitive information from personal images.
Our experiments have shown the gross-promps transferability created with single prompt is highly limited. An intuitive approach to increase the cross-prompt transferability is to use multiple prompts during its creation stage. However, the improvement in cross-prompt transferability of these base- line approaches converges quickly with the increase in prompts. To further improve the cross- prompt transferabilityy, we proposed Cross-Prompy Attack (CroPA), which creates more transferable adversarial images by utilising the learnable prompts. These prompts are optimised in the opposite direction of the adversarial image to cover more prompt embedding space.
In the experiments, prompts for three popular vision-language tasks are used, including image clas- sification, image captioning, and visual question answering (VQA). We examined the effectiveness of our approach on three prevalent VLMs, Flamingo (Alayrac et al. (022). BLIP-2 (Li et al. 2023)
and InstructBLIP (Dai et al. 2023). Experimental results have demonstrated that CroPA consistently outperforms the baseline methods under different settings with different attack targets.
Our contributions can be summarized as follows:
We introduce cross-prompt adversarial transferability, an important perspective of adversar- ial transferability, contributing to the existing body of knowledge on VL.Ms' vulnerabilities.
We propose a novel algorithm Cross-Promm Attack (CroPA), designed to enhance cross- prompt adversarial transferability.
Extensive experiments are conducted to verify the effectiveness of our approach on various VLMs and tasks. Moreover, we provide further analysis to understand our approach.
2 RELATED WORK
Adversarial transferability Foundational studies by (Szegedy et al. 2013 Goodfellow al. 2014) unveil the property of neural networks to misclassify images by adding seemingly imperceptible adversarial perturbations to the inputs. The created adversarial samples can also fool unseen mod- els (Gu et al. 2023bb Yu et al. |2023] [Liu et al., 2016 Papernot et al. 2016). Besides, Mopuri et al.. al. (2017) shows that adversarial perturbation
(207)) Maoosvi-Dezzfod when added to different images. Beyond models and images, the domain of adversarial transferabil-
be still deceptive
ity extends its reach to different tasks (Naseer et al., 2018 2019 Lu et al., 2020; Salzmann et al.) 2021]. For example, adversarial examples designed to attack image classification systems are not Timited in their scope but also fail other tasks, such object detection. In light of the revealed transferability of adversarial examples across models, images, and tasks, the recent advancements in vision- language models introduce a new dimension to be explored. Specifically, this work delves into the transferability across textual prompts within the realm of VI.s..
A Adversarial Robustness of Vision-Languaage Models The majority of prior research on adversarial attacks on vision- languale models are mostly task-specific attacks. For example, there is series of works to manipulate the model output in image captioning tasks (Xu et al.) 2019) Zhang et al.)
2020 Aafaq et al. 2021 Chen et al. (0077a). Similarly, in visual question answering, works such as
2018 Kaushik et al. 2021; Kovatchev et al. 2022 Li et al. 2021; Sheng et al. 2021 Zhang et al. 2022a) mislead the attention region in object detectors affect the model output. Nevertheless, the vision-language models used in these methods are highly sask-spe,,,ic, utilizing lightweight CNN-RNN architectures that lack the capability for in-context learning. Con-
Fooling VQA (Xu et al.
2

---

Published as a conference paper at ICLR 2024
sequently, adapting these methods contemporary VLMs gives challenges. There recent works on the adversarial robustness of large VLMs that consider the adversarial attack from the vision modality. Concretely. Zhao al. (2023) explored the adversarial robustness of recent large vision- language models such as BLIP (Li et al., 2022) and BLIP-2 (Li et al. 2023) under the black box setting including query-hased and wrassferbbased methods craft adversarial examples. Instead of transferability across models, this work introduces cross-transferability
APPROACH
In this section, first describe the concept of cross-prompt adversarial transferability. We then present the baseline approach utilising one or multiple prompts to craft adversarial examples, and the CroPA method which incorporates learnalle prompt enhance cross-prompt transferability.
3.1 PROBLEM FORMULATION
Consider Tu to be a clean image without perturbations induced and let x; denote a prompt. The function f represents VLM. The term 6, signifies the visual perturbation added to the image Fy and is bound by the constraints ||6, lpp S Ev, where by is the image perturbation magnitude.
In targeted attack, the objective is generate visual perturbation, denoted as dg. which when applied to the original input zv. creates an adversarial example Zv + 8,. This adversarial example is structured mislead the model into producing predefined targeted output text T, regardless of the given prompt.
Targeted Attack:
Contrarily, in non-targeted attack, the adversarial example is crafted not to lead the model to specific predefined output but rather to any incorrect output. The goal here to ensure that the model's s output, when fed with the adversarial example, diverges from the output generated with clean, unaltered image as input.
Non-Targeted Attack:
Attack success rate (ASR) is the evaluation metric for cross-prompt transferability, which is defined as the ratio of the number of successful attacks to the total number of attacks. For the targeted attack, attack is considered to be successful only if the prediction exactly matches our target text. For non-targeted attacks, the attack is successful if the model is misled to generate the text different from the prediction with the clean image.
3.2 BASELINE APPROACH
To generate adversarial examples for VLMs, an image perturbation can be optimized based single prompt; this method is referred to as Single-P. To enhance the cross-prompt transferability of the perturbations, straightforward approach is to utilize multiple prompts while updating the image perturbation, a method denoted by Multi-P. The algorithms of Single-P and Multi-P are detailed below.
Let X, = {#/..r .....,,......................... .x represent a collection textual prompt instances. The ultimate goal is derive visual perturbation, 8.. ensuring that for every instance from X, the model yields either the predefined target text 7' in the targeted attack setting, or text deviating from the original output non-targeted attack setting.
The optimization objectives for targeted and non-targeted settings are formulated as follows: For the targeted attack, the objective to minimize the language modelling loss, L, associated with generating the target text T'. This optimization can be mathematically represented as:
min
C(f( &,,
Here, the goal is to alter the input subtly to mislead the model into producing the predefined text T across various prompt instances, effectively minimizing the discrepancy between the model's output and the target text.
3

---

Published as a conference paper at ICLR 2024
Visual Inputt
Vision- Languaa Model Mode
Align Outgut to Target
19
Vision Enooder
Oulpot
Ninetaen
gradient V..
Female
Large
Langage
Langugg Modiling
Textual Imput
gradient: Ft
Model
Lens
the house number In the pictume
Larget
......
Unknown
What the gender of (erson the gicture?
Unknown
c: Fixd Parameters Element-wiise addition
Forward
Backward
Overview of CroPA's framework under the targeted attack setting. Both the image perturbation de and the prompt perturbation & are learnahle but the prompt perturbation does collahorate with 6, to deceive the model. They are optimised with the opposite goals: 6, aims to minimise the language modelling loss while the & aims to maximise the language modelling loss. The update frequency of the image perturbation and prompt perturbation can be different.
Figure 1:
For the non-targeted attack, the objective is to maximize the language modelling loss L, between the text produced by the model with adversarial examples and clean images: max Σ ((f(x, +
5..
8,, #i), (3 af)). The aim here is not to guide the model to specific output but to any output diverging from what would have been produced with an unaltered input, emphasizing the maxi- mization of the discrepancy in the model's responses.
3.3 CROSS-PROMPE ATTACK (CROPA)
In order to create an adversarial image with stronger cross-prompt transferability, we propose an algorithm termed Cross-Prompl Attack (CroPA). The baseline approach constrains prompts to de- coded text representations, namely the fixed hard prompt. In the CroPA framework, we not only used varied numbers of prompts but also introduced learnable prompt perturbation for the prompt embedding during the optimisation phase.
In Figure image perturbation is optimised with the target text "unknown" to hide sensitive information such address and gender. The image perturbation 8, is optimised to minimise the loss of generating the target text "unknown". while the prompt perturbation 6g is updated the opposite direction to maximise the loss of generating the target text. The prompt perturbation gains increasingly stronger cross- cross-prompt transfcrability during this competitive update process.
provide the illustration of the CroPA framework through an example, where the
Coneretely, the optimisation steps of the adversarial image are detailed in Algorithm In the be- ginning, the image adversarial perturbation and the prompt adversarial perturbation are randomly initialised. During the forward pass, these two perturbations are added to the clean image and the clean prompt embedding respectively. After the forward pass, the gradient of language modelling loss with respect the image and text can be obtained through backward propagation. While the image perturbation is updated with gradient descent to minimize the language modelling loss be- tween the prediction and the target sentence, the adversarial prompt is updated with gradient ascent maximize the loss. The parameters 01 and a2 the updated step sizes for the adversarial image and adversarial prompts. The optimisation algorithm is PGD (Madry et al. 2017) with the L-infinity norm being the specific norm used in the experiments.
The update of the adversarial image and the adversarial text can be viewed as a min-max process.
Considering a vision-language model that takes image 2. and text prompt d; input, the objective is to obtain the perturbations 5, for x. that minimises the language modeling loss of generating the targeted sentence, and the perturbations for 2, that maximises the loss. For the targeted attack, the optimisation of the formula can be written as:
min max L(f + 6o, e + 6,),
(2)
6, 6,
4

---

Published as a conference paper at ICLR 2024
Algorithm 1 CroPA: Cross Prompt Attack
Require: Model f. Target Text T, vision input zv. prompt set Xx, perturbation size e, step size of perturbation updating a and a22, number of iteration steps K, adversarial prompt update interval N
Ensure: Adversarial example
1: Initialise 2', Ie.
2: for step K do
3: Uniformally sample the prompt 2i from X
4: if is not initialised then
5: Initialise 2²" =
end if
7: Compute gradient for adversarial image: : gr = V.. [( af),
8:
Update with gradient descent: a/e = u. if mod(step, N) = 0 then
sign(g.)
9:
10:
Compute gradient for adversarial prompt: 9: V (f 21),
11:
Update with gradient ascent: a²" x + sign(gt)
12:
end if
13:
Project x' be within the e-ball of zu: x'. Clip (x))
14:
end for
15:
return 2²4
Similarly, for the non-targeted attack, the optimisation can be expressed as: max min [(f(x, + 8,, 2t + 6t), f (I The visual perturbation ó, is optimised to maximise the loss of generating (1e, so that the model is deceived to generate the output different from the original one. Prompt perturbation & is optimised to minimise the language modelling loss of generating ( For both targeted attack and non-targeted attack, the image perturbation is clipped to the to ensure the invisibility of the image perturbation. We use the parameter N denoting the update interval to control the update frequency of image perturbation and prompt perturbation: the image perturbation for N times, and the prompt perturbation updates once. Please note that the prompt perturbations are added only during the optimisation phase and they are not added during the testing phase.
5, 6,
EXPERIMENTS
Experimental Settings The dataset consists of both images and prompts. The images are collected from the validation dataset of MS-COCO datasets (Lin et al., 2014). The prompts for VQA consist of questions both agnostic and specific to the image content, which are referred as to VQAgeneral and VQA-geciife in the following sections. The image-specific questions derive from the VQA-v2 (Goyal jet al. 2017). We craft prompts for the questions agnostic to image content, image classification, and image captioning with diverse lengths and semantics. By default, the experiments are targeted attacks with the target text set to "unknown" to avoid the inclusion of high-frequency responses in vision-language tasks. Adversarial examples number of prompts for Multi-P and CroPA is set to ten. Detailed prompts can be found in Ap- pendix B The VLMs used are Flamingo, BLIP-2, and InstructBL.IP. We adopt the open-source OpenFlamingo-99 (Awadalla et al., 2023) for Flamingo. Attack Success Rate is used as metric in our experiments. All the ASR scores reported in the following sections are averaged over three runs.
optimised and tested under 0-shot settings. The
The perturbation size is set to 16/255.
4.1 CROSS-PROMPT TRANSFERABILTY COMPARISON
The cross-prompl adversarial transferability is expected to be stronger if more prompts are given during the optimisation stage. To verify this assumption, we sample different numbers of prompts: 5, 10, 50, and 100, and test the targeted ASR. The overall performance of the ASR of the baseline methods and CroPA with different numbers of prompts tested with Flamingo, BLIP-2 and Instruct- BLIP are shown in Figure
For all the experiments conducted this section, we selected the target text "unknown" to avoid the inclusion of high- fighffreyeency responses commonly found in vision-language tasks. Based on the experimental results, we can conclude the following points: 1) CroPA consistently outperforms the baseline approach for all models. As Figure 2 shows, CroPA achieves the best overall performance in all testing models for all different prompt numbers. CroPA also achieves the best individual performance in most tasks. The detailed data can be found in the supplementary. 2) More prompts
5

---

Published as a conference paper at ICLR 2024
0.9
0.9
0.8
Targeted ASR
0.6
0.7
Targsted
0.7
0.5
0.6
0.4
Single-P
Singlo-P
0.5
Single-P
0.3
Multi-P
0.4
Multi
CroPA
CroPA
CroRA
0.2
0.3
(a) Prompts (Flamingo)
(b) Prompts (BLIP-2)
(c) Prompts (InstructBLIP)
The targeted ASR three methods tested on (a) Flamingo, (b) BLIP-2, and (c) InstructBLIP. Different numbers of prompts 1, 5, 10, 50, and 100 are used in the transferability test. Our CroPA achieve better cross- pross-prompt adversarial transferability than Single-P and Multi-P.
Figure 2:
increase the transferability, but convergence occurs rapidly. We can observe that in general more prompts increase the targeted ASR, especially when the number of prompts increases from one to five. However, starting from ten, the increase of cross-prompt transferability brought by more prompts becomes marginal, especially for the baseline approach. This pattern indicates that by adding more prompts, the baseline approach cannot surpass the performance of CroPA methods.
We also conducted experiments with cross-prompt transferability tests combined with other dimen- sions of transferability, models or images. Our experimental results have shown that adversarial transferability limited. When tested across different prompts and images, the ASRs are all near 0. The ASRs over the models with significantly different architecture are near 0. For the pair of InstructBLIP and BLIP-2, an overall ASR of around 10% is achieved and the detailed data can be found in Table 8 in Appendix C. It should be noted that the language models used in BLIP-2 and InstructBLIP are two different models: OPT-2.7b (Zhang et al. 2022b) and Vicuna-7b (Zheng et al. (023) respectively. Though the performances when combined with other perspectives of the transferability are not as strong as cross-promp transfcrability alone, CroPA still consistently outperforms the baseline methods. Morcover, existing methods for enhancing cross-model and cross-image transferability are orthogonal to the methods proposed in this study. For example, the cross-image transferability can be enhanced by computing the perturbation over larger number of images (Moosavi-Dezfooll et al. 2017). We extend the work of creating adversarial images with combined perspectives of transferability to future work.
4.2 CROPA WITH DIFFERENT TARGET TEXTS
To demonstrate that the effectiveness of the proposed CroPA methods is not constrained to the specific case of the target text "unknown", we extend our evaluation to various other target texts. The experiment includes a selection of text with varied length and usage frequency. Both common expressions such as "very good"' and less common words such as "metaphor" are tested.
From Table [ we can observe that CroPA performs the best overall and each individual task under different target text, though the ASR differs for different target prompts. This table also demonstrated that cross pross-prompt transfcrability is not associated with the semantic meaning or the word frequency of the target text. Semantically "not sure" is reasonable generation result of VLMs to indicate the uncertainty of the response. However, its targeted ASR is much lower than "metaphor", which less commonly used as response to the questions or instructions about an image. The results also show that the cross-promp transferability is not determined by the target text length. The ASR of the three-word target text not necessarily to lower than the ASR of one-word target text.
We also conducted the experiments by setting the target texts to harmful instructions such as "sui- cide", "bomb" and "kidnap". As shown in Table 9 the high ASR values indicate that the cross- prompt adversarial transferability can lead to security issues in real-world applications. From this perspective, the cross-prompt transferability is also related the jailbreak /Zou et al. (2023) for languae models, which also aims to deceive the model to generate harmful instructions.
In summary, the CroPA has been proven be stronger method than the baseline approach with dif- ferent target texts. While different target texts do affect the tross-promp adversarial transfcrability, it is unlikely that this is associated with the semantics length of the prompts themselves.

---

Published as a conference paper at ICLR 2024
Table 1: Targeted ASRs tested on Flamingo with different target texts. The mean and standard deviations of the ASRs are shown in the table. The 'Overall' column indicates the average targeted success rate across all tasks. The best performance values for each task are highlighted in bold.
Target Prompt
Method
VQAgmera
Classification
Captioning
Overall
Single-P
0.2421
15
0.21 25---
1.0512.31e--
0.22.8.044 04443
unknown
Multi-P 0,67+7.14e
0,86+22099 3
0.64+1.35ec 35e
0.31+1.41e2
0.62++8.16e
CroPA
0.92=1.07=2
0.98 722-8 72--8
0.70
0.34 1.....91-3
0.74...775--
Single-P
0.21+1.50e
0.43+7.52e
0,47+8. 3
0.34+3.014 +1eel
0.36+6.286
sorry
Multi-P
0.6021.28es
0.85.1145es
0.71 2.26-2
0.60 97e-8
0.6919.874 977-33
CroPA
0.90-3.56e.
0.96+5.25e:
0.75++8.34c
0.72+7.04e
0.83+6691e.
Single-P
0.25+1142e-3
n:26-
0.09+1125ee 25e-2
0.00+6.046 04e-8
0.17+7.03e-3
not sure
Multi-P
0.53 56e-3
0.55 25e-33
0.11 09e-3
0.02 12e-3
0.31 39--3
CroPA
0.88=1.196.2
0.86 2.6.877
0.30:18.19..
0.17:9.290
0.55
Single-P
0.35:8.31e2
0.52 17c-2
0.1514402263
0.05 726-3
0.27.8.992:2
very good
Multi-P
0.81+9.51e5
0.93+33336--3
0,40+1.91c 911ee3
0,20+ 42e
0,59+8.79c 79e-2
CroPA
0.9521.13e:
0. 26e-8
0.6412.396-5
0.2711.06m2
0.7118.61
Single-P
0.21+1.72e
0.38+ 433
7.21+8.550r
0.04 92e
0.21+7.84e:
too late
Multi-P
0.78 0.9017.93e-3
0.54
0.1711.355 1.3ee2
0.60
CroPA
0,90-1003e2
0.95+5.36c
0.73 8.8.
0. 0.20 0.20++8.65e
0.70+8.33e
P
0.26+1.46e
0.56+8.22e-3
7.50+5.022il
0.14+1. 21e-22
0.37+8.886.
metaphor
Multi-P
0.83 1.4e-2
0.92 18e-2
0.81.1.41ec
0.42 1. 35e-2
0.7521.33es
CroPA
0.96-1.390
0.99 0.
0.92#3.74e.
1.62#1.630.3
0.87
4.3 CROPA MEETS IN-CONTEXT LEARNING
In addition to the textual prompt, the Flamingo model also supports providing extra images as in- context learning examples improve the task adaptation ability. Whether these in-context learning examples have an influence on the cross-promp attack remains unclear. Therefore, we tested the ASRs of the image adversarial examples with the number of in-context learning examples different from the one provided in the optimisation stage. During the optimisation stage, the image adversarial examples are updated under the O-shot setting, namely no extra images are provided as the in-context learning examples. In the evaluations, the 2-shot setting is used, i.e. two extra images are used as the in-context learning examples. Evaluation results under the O-shot setting are also provided for comparison.
As Table 2 shows, the CroPA still achieves the best performance under the 2-shot settings. We can observe that in-context learning examples can decrease the ASRs, as these two extra in-context learning examples cause a shift in the generation condition different from the optimisation stage.
Table 2: Targeted ASRs of with and without visual in-context learning. The shot indicates the number of images added for in-context learning. The model utilised is Flamingo. The mean and standard deviations of the ASRs are shown in the table. The best performance values for each task are highlighted in bold.
Method
VQAgmeral
VQAspaifie
Classification
Captioning
Overall
Multi-P (shot=0)
0.67 14ee3
0.86 09c-3
0.64+ 1.35e-3
0.62 16e-3
CroPA (shot=0)
0.92=
0.98 72e-3
0.70 42e-3
0.34 19e-3
0.74
Multi-P (shol=2)
2..9-4
0.81 4.4ee2
0.50=1.12e22
0.25
0.54 188e-33
CroPA (shot=2)
0. 1. 3
0.96
1
0.76:1131e2
0.26=9.41e2
0. 0.70=1.0m 770=1.000 1.09e
4.4 CONVERGENCE OF CROPA
In this section, we conduct experiments to compare the performance of baseline and CroPA methods over different update iterations. As shown in Figure 3 we present the results of the overall targeted ASRs with attack iterations from 300 to 1900 every 200 iterations. The number of prompts used for Multi-P and CroPA in optimisation is set to ten.
For all the methods, adding more attack iterations can increase ASRs at the beginning, but the per- formances eventually converge. For the Single-P method, the improvement in cross-prompt trans- ferability by using more iterations has quickly become marginal after 300 iterations. However, for the Muli-P and CroPA, the performance can still increase after 1000 epochs.
7

---

Published as a conference paper at ICLR 2024
0.7
Targeted ASR
0.8
Single-P
0.5
Multi-
CroPA
0.3
0.2
400
600
800
1000
1200
1400
1800
Number of Ieerations
Figure 3: Targeted ASRs of Bascline Methods (Single-P and Multi-P) and CroPA over attack Itera- tions. With the same number of attack iterations, our CroPA significantly outperforms baselines.
The figure also demonstrated that CroPA methods do not rely on extra attack iterations to gain better performance compared to the baseline approaches. Given the same attack iterations, the CroPA consistently achieves better performance compared to the Single-P and Multi-P methods. CroPA requires only 500 iterations achieve the target ASR above 0.6 while the stronger baseline Multi-P requires over 1000 iterations.
Overall, Section 4.1 and this section have shown that the cross- cross-prompl transferability of the baseline method is limited compared to CroPA with the increase of the prompt numbers and iterations.
4.5 CROPA WITH DIFFERENT UPDATE STRATEGY
In this section, explore the effect of different update strategies. As described in previous sections, the update frequency of both the image perturbation and the prompt perturbation can be different. A special case of CroPA is CroPAjjim. where the image perturbation and the prompt perturbation have the same update frequency. We compare the result CroPA with CroPAyoin on different tasks with different in-context learning settings. Similar to Section 4.3 adversarial examples optimised under 0-shot and tested under 0-shot and 2-shot settings.
As shown in Table 3 the CroPA outperform the CroPAjaim overall and most individual tasks with different in-context Tearning image examples. The stronger performance CroPA derives from its flexibility in choosing the update step size of the prompt perturbation. As presented in Appendix [A] the CroPAjoint is sensitive to the step size of the prompt embedding: if the prompt update size is too large the optimisation fails to converge. CroPA is more tolerant of large prompt update sizes by reducing the prompt update frequency.
Table Our CroPA with alternative optimization outperforms the one with vision and prompt joint optimization in most cases. The mean and standard deviations of the ASRs are shown in the table. The best performance values for each task are highlighted in bold
Method
VQAgeeral
VQAspecific
Classification
Captioning
Overall
CroPAjoia (shot=0)
0.86-1.44e3
0.95+9.000
991e33 91e-3
0.73+5. 40e-3
1.ee
0.71+6.99ec
CroPA (shot=0)
0.92-1.07c-2
0.98 0.98-6.776.9
0.70+3.42e-3
0.34+3.188
0.74+
CroPAjan (shot=2)
0.76
0.94 74m33
0.71
0.2524.20es3
0.66
CroPA (shot=2)
0.84_3.184 188-3
0.96 18-8
0.76 1.81=22
0.2629.41es
0.70 1.00-22
4.6 UNDERSTANDING THE EFFECTIVENESS CROPA METHODS
Visualisation of the Prompt Embedding Coverage To explore the underlying reasons for the bet- ter performance of CroPA compared the baseline approach, we visualise the sentence embedding of the original prompt and perturbated prompts by CroPA, which is obtained by the averaging em- bedding of each token.
As demonstrated in Figure 4al the orange plus symbol denotes the original prompts while the purple star symbol denotes the original embedding added with the perturbation 5, It can be observed that there is almost overlap between the prompt embedding perturbed by CroPA and the original prompt embedding. This verifies that the adversarial prompt effectively increases the coverage of the original embedding.
8

---

Published as a conference paper at ICLR 2024
159
CroPA
100
t-SNE Dimension
t- Dimension
Original Prompts
Semp
PromptSer2 SeomptSet1 CroRA
Prompts CroPA
t-N 1
E-N Dimension
(a) Visualisation of the prompt embeddings and its prompt embedding created by CroPA. The orange plus symbol denotes the original prompt and the pur- ple star symbol denotes the embedding by adding the adversarial prompt perturbation to the original prompt embedkligg
(h) Comparison between the embedding coverage difference between prompts generated by CroPA and extra Prompt Set 2. The circle denote the em- bedding coverage of Prompt Set and Prompt Set 2, while the blue circle represents the coverage of Prompt Set I with the prompts generated by CroPA.
Figure 4: Visualisation of the prompt embeddings with t-SNE Van der Maaten & Hinton 2008
To have clearer comparison between the baseline approach, which relies on simply adding more prompts, and the CroPA methods, we visualise the coverage of prompt embeddings of these two methods in Figure [ shows. The embedding Prompt Set denoted by the orange dots, while the embeddings of Prompt Set 2 are denoted as the cyan dots. In the CroPA, only Prompt Set is provided. By introducing the learnable prompt perturbation to Prompts Set 1, the prompt embed- dings that have been covered during optimisation denoted by the purple stars. The blue eclipse is the approximated coverage of the prompt embedding for CroPA using Prompt Set 1. For the baseline method, both Prompt Set and Prompt Set provided and the red eclipse approximately repre- sents the coverage of their coverage. It can be observed that with only Prompt Set 1, the area covered by the CroPA broader than the one covered by the embeddings of Prompt Set I and Prompt Set 2.
The visualisation of difference in the prompt embedding coverage explains the reason why the CroPA methods can outperform the baseline approach even the number of prompts used in opti- misation is less than the baseline approach.
Prompt Embedding Decoding We explored the decoding of the adversarial prompt embedding to a human-rcadable text format. The embedding of each token is decoded to the readable text closest terms of cosine distance using the pre-trained embedding look-up table of the language models. The results show that all the perturbed embeddings are still closest to their original tokens. This finding also supports the effectiveness of CroPA: There exists prompt embedding that cannot be represented by human-readable text. Therefore, even if given sufficient number of prompts in the baseline approach, it still can not cover all the prompt embedding space of the adversarial prompt in the CroPA framework.
5 CONCLUSION
In this paper, we first raise an interesting and important question, can single adversarial example mislead all predictions of a linguuel model with different prompts? We formulate the essence of the question as the cross-prompt adversarial transferability of adversarial perturbation. Extensive experiments show that intuitive baseline approaches only achieve limited transferability and our proposed CroPA improves the transferability significantly on different VLMs in different multi- modal tasks. One of the ways further improve the practical applicability of method is to implement the optimization with query-based strategies (Chen et al.] 20176 Ilyas et al.) 2018), which we leave future work.

---

Published as a conference paper at ICLR 2024
Acknowledgement This work is supported by the UKRI grant: Turing AI Fellowship EP/W002981/1 1/1, and EPSRC/MURI grant: EP/N019474/1, We would also like to thank the Royal Academy of Engineering and FiveAl.
REFERENCES
Nayyer Aafaq, Naveed Akhtar, Wei Liu, Mubarak Shah, and Ajmal Mian. Controlled caption gen- eration for images through adversarial attacks. arXiv preprint arXiv:2107.03050, 2021.
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in Neural Information Processing Systems, 35:23716- 23736, 2022.
Anas Awadalla, Trena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. Opentlamingo: An open- source framework for training large autoregressive vision-language models. arXiv preprint 0rX11:2308..13300 2023.
Hongge Chen, Huan Zhang, Pin- Yu Chen, Jinfeng Yi, and Cho-Jui Hsich. Attacking visual language grounding with adversarial examples: A case study on neural image captioning. arXiv preprint arXiv:1712.02051/ 2017a.
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order opti- mization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop artificial intelligence and security, pp. 15-26, 2017b.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. Instructblip: Towards general-purpose vision- language models with instruction tuning. ArXiv, abs/2305 2bs//230500600, 2023. URL https://app,
semanticschon wwwp:orpo//Cssuuooooo TD: 258 616266
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6577 2014.
Yash Goyal, Tejas Khot, Douglas Summers-Stay Dhruv Batra, and Devi Parikh. Making the in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6904-6913, 2017.
Jindong Gu. Hengshuang Zhao, Volker Tresp, and Philip HS Torr. Segpgd: An effective and efficient adversarial attack for evaluating and boosting segmentation robustness. In European Conference Computer Vision, pp. 308-325. Springer, 2022.
Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He,, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, and Philip Torr. A systematic survey of prompt engineering on vision- language foundation models. arXiv preprint arXiv:2307.12980 2023a.
Jindong Gu, Xiaojun Jia, Pau de Jorge, Wenqain Yu, Xinwei Liu, Avery Ma, Yuan Xun, Anjun Hu, Ashkan Khakzar, Zhijiang Li, et al. survey on transferability of adversarial examples across deep neural networks. arXiv preprint arXiv:2310.
2023b.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International conference on machine learning, pp- 2137- 2146. PMLR, 2018.
Divyansh Kaushik, Douwe Kiela, Zachary C Lipton, and Wen-tau Yih. On the efficacy of adver- sarial data collection for question answering: Results from a large-scale randomized study. arXiv preprint arXiv: 106.0000072.. 2021.
10

---

Published as a conference paper at ICLR 2024
Venelin Kovatchev, Trina Chatterjee, Venkata S Govindarajan, Jifan Chen, Eunsol Choi, Gabriella Chronis, Anubrata Das, Katrin Erk, Matthew Lease, Junyi Jessy Li, et al. longhorns at dade 2022: How many linguists does it take to fool question answering model? systematic approach to adversarial attacks. arXiv preprint arXiv:2206. 14729, 2022.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping danguage-image pre- training for unified vision-language understanding and generation. In International Conference Machine Learning, pp. 12888-12900 PMLR, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023.
Linjie Li, Jie Lei, Zhe Gan, and Jingjing Liu. Adversarial vqa: A new benchmark for evaluating the robustness of vqa models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2042-2053 2021.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks. arXiv preprint arXiv:1611.0277 2016.
Yantao Lu, Yunhan Jia, Jianyu Wang, Bai Li, Weiheng Chai, Lawrence Carin, and Senem Veli- pasalar. Enhancing cross-task black-box transferability of adversarial examples with dispersion reduction. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recog- nition, pp. 940-949, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks, arXiv preprint 1rXiv:1706.06088 2017.
Seyed-Mohsen Moosavi- Dezaooil,, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1765-1773, 2017.
Konda Reddy Mopuri, Utsav Garg, and R Venkatesh Babu. Fast feature fool: A data independent approach universal adversarial perturbations. arXiv preprini arXiv:1707.06572 2017.
Muhammad Muzammal Naseer, Salman H Khan, Muhammad Haris Khan, Fahad Shahbaz Khan, and Fatih Porikli. Cross-domain transferability of adversarial perturbations. Advances in Neural Information Processing Systems, 32, 2019.
Muzammal Naseer, Salman H Khan, Shafin Rahman, and Fatih Porikli. fask-generalizable adver- sarial attack based on perceptual metric. arXiv preprint arXiv: 1XXiv:88/1.000222 1.09020, 2018.
Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv: 16050007, 2016.
Mathieu Salzmann et al. Learning transferable adversarial perturbations. Advances in Neural Infor- mation Processing Systems, 34:1 3399,, 2021.
Sasha Sheng, Amanpreet Singh, Vedanuj Goswami, Jose Magana, Tristan Thrush, Wojciech Galuba, Devi Parikh, and Douwe Kiela. -uumar-adversarial visual question answering. Advances in Neural Information Processing Systems, 34:20346-20359, 2021.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
11

---

Published as a conference paper at ICLR 2024
Florian Tramer, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint arXiv:1704.03553, 2017.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine
learning research, 9(11), 2008.
Boxi Wu, Jindong Gu, Zhifeng Li, Deng Cai, Xiaofei He, and Wei Liu. Towards efficient adversarial training on vision transformers. In European Conference on Computer Vision, pp. 307-325. Springer, 2022.
Xiaojun Xu, Xinyun Chen, Chang Liu, Anna Rohrbach, Trevor Darrell, and Dawn Song. Fooling vision and language models despite localization and attention mechanism. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4951-4961, 2018.
Yan Xu, Baoyuan Wu, Fumin Shen, Yanbo Fan, Yong Zhang, Heng Tao Shen, and Wei Liu. Ex- act adversarial attack to image captioning via structured output learning with latent variables. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4135-4144 2019.
Xu Yang, Kaihua Tang, Hanwang Zhang, and Jianfei Cai. Auto-encoding scene graphs for image captioning. In Proceedings of the IEEE/CVF conference computer vision and pattern recogni- tion, pp. 10685-10694, 2019.
Ting Yao, Yingwei Pan, Yehao Li, and Tao Mei. Exploring visual relationship for image captioning. In Proceedings of the European conference computer vision (ECCV), pp. 684-699, 2018.
Wenqian Yu, Jindong Gu, Zhijiang Li, and Philip Torr. Reliable evaluation of adversarial transfer- ability. arXiv preprint arXiv:2306.8885 2023.
Jiaming Zhang, Qi Yi, and Jitao Sang. Towards adversarial attack on vision-language pre-training models. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 5005- 5013, 2022a.
Shaofeng Zhang, Zheng Wang, Xing Xu, Xiang Guan, and Yang Yang. Fooled by imagination: Adversarial attack to image captioning via perturbation in complex domain. In 2020 IEEE Inter- national Conference on Multimedia and Expo (ICME), pp. 1-6. IEEE, 2020.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.0106 2022b.
Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models. arXiv preprint ar05v:2305. 16934, 2023.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.055:
2023.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En hancing vision-language understanding with advanced large language models. arXiv preprint 1rX1v22304.10592, 2023.
Andy Zou, Zifan Wang. J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307. / 15043, 2023.
12

---

Published as a conference paper at ICLR 2024
A EFFECT OF PROMPT PERTURBATION UPDATE STEP SIZE
Selecting the correct prompt updating step size vital for the CroPAjoint variant. We present the results utillssng the validation dataset to probe the optimal prompt update step size. The image adversarial update step size is fixed to 1/255. The number of prompts during optimisation set to ten. The update step size used in this experiment are 0.01 0.001, 0.0001, le-05 and le-06.
As depicted in Figure 5a we can observe that if the update step size set to 0.01, though the loss decreases during the first one hundred iterations, it fails converge eventually. The corresponding performance recorded in Figure 5b) validates this, it achieves the lowest targeted ASR compared to the other step size.
For the rest of the learning rate, we can observe that the optimisation processes are roughly con- verged though the the values it converge can be different. It is notable that the update step size 0.001, which is larger than the rest of the update step size, achieves the best performance even if the final loss value converges is not the smallest compared the other prompt update step size.
0.81
Tergeted ASR
0.0011
Bersbion
(a) The optimsation loss CroPAjoiat algorithm with different update step sizes of the prompts. The algorithm with different update step sizes of the prompts.
(b) The targeted attack success rate the CroPAjoiat
image adversarial update step size is fixed to 1/255.
Figure Effect of different update step size of prompt perturbation of CroPAjoint
Also, we can notice that at around five hundred iterations, the optimisation loss increases slightly, but the performance of the targeted success rate increases significantly. This is because larger step size of the prompt can be perceived as a stronger adversary for the image adversarial perturbation. During this learning process, the image adversarial perturbation is optimised counteract this challenging prompt input and thereby obtains stronger cross-prompt adversarial transferability. This empirical finding consistent with the idea in machine learning that the models are optmised on challenging cases to improve the model's performance. This finding is also verified from the other perspective. We can observe that the performance with the step size set to le-06 increases most slowly compared to the performance with a larger update step size of prompts.
In conclusion, the optimal choice of the prompt update step size should allow the image perturbation to converge reasonably large value range allow the perturbation gain stronger cross-prompt adversarial transferability.
B PROMPTS FOR DIFFERENT TASKS
Prompts for VQA
Any cutlery items visible in the image?
Any bicycles visible in this image?
Any boats visible the image?
Any bottles present in the image?
Are curtains noticeable in the image?
Are flags present in the image?
Are flowers present in the image?
Are fruits present in the image?
13

---

Published as a conference paper at ICLR 2024
Are glasses discernible in the image?
Are hills visible the image?
Are plates discernible in the image?
Are shoes visible in this image?
Are there any insects in the image?
Are there any ladders in the image?
Are there any man-made structures in the image?
Are there any signs or markings in the image?
Are there any street signs in the image?
Are there balloons in the image?
Are there bridges in the image?
Are there musical notes in the image?
Are there people sitting in the image?
Are there skyscrapers in the image?
Are there toys in the image?
Are toys present in this image?
Are umbrellas discernible in the image?
Are windows visible in the image?
Can birds be seen in this image?
Can stars be seen in this image?
Can we find any bags in this image?
Can you find crowd in the image?
Can you find hat in the image?
Can you find any musical instruments in this image?
Can you identify clock in this image?
Can you identify a computer in this image?
Can you see beach in the image?
Can you see a bus in the image?
Can you see mailbox in the image?
Can you see a mountain in the image?
Can you see a staircase in the image?
Can you see a stove or oven in the image?
Can you see a sunset the image?
Can you see any cups or mugs in the image?
Can you see any jewelry in the image?
Can you see shadows in the image?
Can you see the sky in the image?
Can you spot a candle in this image?
Can you spot a farm in this image?
Can you spot a pair of shoes in the image?
Can you spot a rug or carpet in the image?
Can you spot any dogs in the image? Can you spot any snow in the image?
Do you notice a bicycle in the image?
Does ball feature in this image?
Does a bridge appear in the image?
Does cat appear in the image?
Does a fence appear in the image?
Does fire feature in this image?
Does a mirror feature in this image?
Does table feature in this image?
Does it appear to be nighttime in the image?
Does it look like an ouldoor image?
Does it seem to be countryside in the image?
Does the image appear be a cartoon or comic strip?
Does the image contain any books?
Does the image contain any electronic devices?
Does the image depict road?
Does the image display a river?
14

---

Published as a conference paper at ICLR 2024
Does the image display any towers?
Does the image feature any art pieces?
Does the image have lamp?
Does the image have any pillows?
Does the image have any vehicles?
Does the image have fürniture?
Does the image primarily display natural elements?
Does the image seem like it was taken during the day?
Does the image seem to be taken indoors?
Does the image show any airplanes?
Does the image show any benches?
Does the image show any landscapes?
Does the image show any movement?
Does the image show any sculptures?
Does the image show any signs? Does the image show food?
Does the image showcase a building?
How many animals are present the image?
How many bikes are present in the image?
How many birds are visible in the image?
How many buildings can be identified in the image?
How many cars can be seen the image?
How many doors can you spot in the image?
How many flowers can be identified the image?
How many trees feature in the image?
Is a chair noticeable in the image? Is computer visible in the image?
Is a forest noticeable in the image? painting visible in the image?
Is a path or trail visible in the image? phone discernible in the image?
Is a train noticeable in the image? sand visible in the image? Is the image displaying any clouds? the image set city environment?
Is there plant in the image?
there source of light visible in the image?
Is there television displayed in the image? there grass in the image?
Is there text in the image?
Is water visible in the image, like sea, lake, or river?
How many people are captured in the image? How many windows can you count in the image?
How many animals, other than birds, are present?
How many statues or monuments stand prominently in the scene?
How many streetlights are visible?
How many items of clothing can you identify?
How many shoes can be seen in the image?
How many clouds appear in the sky?
How many pathways or trails are evident?
How many bridges can you spot?
How many boats are present, if it's s a waterscape?
How many pieces of fruit can you identify?
How many hats are being worn by people?
How many different textures can you discern?
How many signs or billborrs are visible?
How many musical instruments can be seen?
How many flags are present in the image?
How many mountains or hills can you identify?
15

---

Published as a conference paper at ICLR 2024
How many books visible, any?
How many bodies of water, like ponds or pools, are in the scene?
How many shadows
you spot?
How many handheld devices, like phones, are present?
How many pieces of jewelry can be identified?
How many reflections, perhaps in mirrors or water, are evident?
How many pieces of artwork or sculptures can you see?
How many staircases or steps are in the image?
How many archways tunnels can be counted?
How many lools equipment ure visible?
How many modes of transportation, other than cars and bikes, can you spot?
How many lamp posts or light sources are there?
How many plants, other than trees and flowers, feature in the scene?
How many fences barriers can be seen?
How many chairs or seating arrangements can you identify?
How many different patterns motifs evident in clothing objects?
How many dishes or food items are visible on a table setting? How many glasses or mugs can you spot? How many pets or domestic animals are in the scene?
How many electronic gadgets can be counted?
Where is the brightest point in the image?
Where are the darkest areas located?
Where can one find leading lines directing the viewer's 's eyes?
Where is the visual center of gravity in the image?
Where are the primary and secondary subjects positioned?
Where do the most vibrant colors appear?
Where is the most contrasting part of the image located?
Where does the image place emphasis through scale or size?
Where do the textures the image change or transition?
Where does the image break traditional compositional rules?
Where do you see repetition or patterns emerging?
Where does the image exhibit depth or layers?
Where are the boundary lines or borders in the image?
Where do different elements in the image intersect or overlap?
Where does the image hint at motion or movement?
Where are the calm or restful areas of the image?
Where does the image become abstract or less defined?
Where do you see reflections, be it in water, glass, or other surfaces?
Where does the image provide contextual clues about its setting?
Where the most detailed parts of the image?
Where do you see shadows, and how do they impact the composition?
Where can you identify different geometric shapes?
Where does the image appear to have been cropped or framed intentionally?
Where do you see harmony or unity among the elements?
Where are there disruptions or interruptions in patterns?
What is the spacing between objects or subjects in the image?
What foreground,
hid-ground, and background elements can be differentiated? What type of energy or vibe does the image exude?
What might be the sound environment based on the image's 's content?
What abstract ideas or concepts does the image seem to touch upon?
What is the relationship between the main subjects in the image?
What items in the image could be considered rare or unique? What is the gradient or transition of colors like in the image?
What might be the smell or aroma based on the image' content?
What type of textures
be felt if one could touch the image' content?
What boundaries or limits are depicted in the image?
What is the socioeconomic context implied by the image?
What might be the immediate aftermath of the scene in the image?
What seems to be the main source of tension or harmony the image?
16

---

Published as a conference paper at ICLR 2024
What might be the narrative or backstory of the main subject? What elements of the image give it its primary visual weight?
Would you describe the image bright or dark?
Would you describe the image as colorful or dull?
Prompts for Image Classification
Identify the primary theme of this image in one word. How would you label this image with a single descriptor?
Determine the main category for this image.
Offer a one-word identifier for this picture. If this image were a file on your computer, what would its name be?
Tag this image with its most relevant keyword.
Provide the primary classification for this photograph.
How would you succinctly categorize this image?
Offer the primary descriptor for the content of this image. If this image were a product, what label would you place on its box?
Choose single word that encapsulates the image's content.
How would you classify this image in a database? In one word, describe the essence of this image. Provide the most fitting category for this image. What is the principal subject of this image?
If this image were in store, which aisle would it belong to? Provide a singular term that characterizes this picture.
How would you caption this image in a photo contest?
Select a label that fits the main theme of this image. Offer the most appropriate tag for this image.
Which keyword best summarizes this image? How would you title this image in an exhibition? Provide a succinct identifier for the image's 's content. Choose word that best groups this image with others like it. If this image were in a museum, how would it be labeled?
Assign central theme to this image in one word. Tag this photograph with its primary descriptor. What is the overriding theme of this picture?
Provide a classification term for this image. How would you sort this image Identify the main subject of this image concisely. If this image were a magazine cover, what would its title be? What term would you use to catalog this image? Classify this picture with a singular term. If this image were a chapter in book, what would its title be?
collection?
Select the most filting classification for this image.
Define the essence of this image in one word.
How would you label this image for easy retrieval? Determine the core theme of this photograph. In a word, encapsulate the main subject of this image. If this image were an art piece, how would it be labeled in a gallery?
Provide the most concise descriptor for this picture. How would you name this image in photo archive? Choose word that defines the image's 's main content.
What would be the header for this image in a catalog? Classify the primary essence this picture.
What label would best fit this image in slideshow? Determine the dominant category for this photograph. Offer the core descriptor for this image.
this image were textbook, how would it be labeled in the index? Select the keyword that best defines this image' 's theme.
17

---

Published as a conference paper at ICLR 2024
Provide a classification label this image.
If this image were a song title, what would it be? Identify the main genre of this picture.
Assign the most apt category to this image. Describe the overarching theme of this image in word.
What descriptor would you use for this image in a portfolio? Summarize the image's content with single identifier. Imagine you 'r explaining this image to someone over the phone. Please describe the image in one word?
Perform the image classification task this imuge. Give the label in one word. Imagine child trying to identify the image. What might they excitedly point to and name? If this image were turned into jigsaw puzzle, what would the box label say to describe the picture inside?
Classify the content of this image. If you were to label this image, what label would you give? What category best describes this image? Describe the central subject of this image in a single word. Provide a classification for the object depicted in this image. If this image were in photo album, what would its label be?
Categorize the content of the image.
If you were to sort this image into category, which one would it be? What keyword would you associate with this image?
Assign relevant classification to this image.
If this image were a gallery, under which section would it belong?
Describe the main theme of this image in one word.
Under which category would this image be cataloged in a library? What classification tag fits this image the best?
Provide a one-word description of this image's 's content.
you were to archive this image, what descriptor would you use?
Prompts for Image Captioning
Elaborate on the elements present in this image. In one sentence, summarriee the activity in this image. Relate the main components of this picture words. What narrative unfolds in this image? Break down the main subjects of this photo.
Give an account of the main scene in this image.
In a few words, state what this image represents.
Describe the setting location captured in this photograph.
Provide an overview of the subjects or objects seen in this picture.
Identify the primary focus or point of interest in this image. What would be the perfect title for this image? How would you introduce this image in presentation?
Present a quick rundown of the image's 's main subject. What's 's the key event or subject captured in this photograph?
Relate the actions or events taking place in this image.
Convey the content of this photograph a single phrase.
Offer succinct description of this picture
Give a concise overview of this image.
Translate the contents of this picture into senience. Describe the characters or subjects seen in this image. Capture the activities happening in this image with words.
How would you introduce this image to an audience? State the primary events or subjects in this picture
What are the main elements in this photograph?
Provide an interpretation of this image's 's main event or subject. How would you title this image for an art gallery?
18

---

Published as a conference paper at ICLR 2024
What scenario setting is depicted in this image?
Concisely state the main actions occurring in this image.
Offee short summary of this photograph 's contents.
How would you annotate this image in an album? If you were to describe this image on the radio, how would you do is? In your own words, narrate the main event in this image.
What are the notable features of this image? Break down the story this image is trying to tell.
Describe the environment or backdrop in this photograph. How would you label this image in catalog? Convey the main theme of this picture succinctly.
Characterize the primary event or action in this image. Provide a concise depiction of this photo's content. Write brief overview of what's 's taking place in this image. Illustrate the main theme of this image with words. How would you describe this image in gallery exhibit?
Highlight the central subjects or actions in this image. Offer a brief narrative of the eventy in this photograph. Translate the activities in this image into brief sentence.
Give a quick rundown of the primary subjects in this image. Provide a quick summary of the scene captured in this photo. How would you explain this image to a child?
What are the dominant subjects or objects in this photograph?
Summarize the main events or actions in this image. Describe the context or setting of this image briefly. Offer a short description of the subjects present in this image. Detail the main scenario setting seen in this picture.
Describe the main activities or events unfolding in this image. Provide a concise explanation of the content in this image. If this image were in a textbook, how would it be captioned?
Provide a summary of the primary focus of this image. State the narrative or story portrayed in this picture.
How would you introduce this image a documentary? Detail the subjects or events captured in this image. Offer a brief account of the scenario depicted in this photograph.
State the main elements present in this image concisely. Describe the actions or events happening in this picture.
Provide a snapshot description of this image's 's content. How would you briefly describe this image's 's main subject or event?
Describe the content of this image. What's 's happening in this image? Provide a brief caption for this image. Tell a story about this image in one sentence. If this image could speak, what would it say?
Summarize the scenario depicted in this image.
What is the central theme or event shown in the picture? Create headline for this image.
Explain the scene captured in this image. If this were postcard, what message would it convey?
Narrate the visual elements present in this image. Give short title to this image.
How would you describe this image to someone who can't 't see it? Detail the primary action subject in the photo. If this image were the cover of a book, what would its title be?
Translate the emotion event of this image into words.
Compose a one-liner describing this image's 's content. Imagine this image in a magazine. What caption would go with it? Capture the essence of this image in brief description.
19

---

Published as a conference paper at ICLR 2024
Narrate the visual story displayed in this photograph.
C DETAILED DATA
Table 4: Targeted attack success rates tested on Flamingo. The columns labeled 'VQAgeneral and 'VQAspccit: denote the VQA prompts for general and specific types of questions respectively. The columns 'Classification' "Captioning" refer the success rates the image classification 'Overall' column indicates the average targeted success rate across all tasks. The 'Num' column signifies number of prompts used during optimisation. 'CroPAjuim' and 'CroPA' represent the CroPA methods utilizing joint and altemnating updating strategies respectively. The best performance values for each task are highlighted bold.
captioning tasks respectively, The
No. of Prompts
Method
VQAgmeml
VQA
Classification
Captioning
Overall
Single-P
0.24
0.39
0.21
0.05
0.22
CroPApiim
0.30
0.47
0.16
0.09
0.26
CroPA
0.52
0.69
0.38
0.17
0.44
Baseline
0.63
0.82
0.57
0.22
0.56
5
CroPAjaim
0.82
0.93
0.69
0.30
0.69
CroPA
0.90
0.96
0.56
0.39
0.70
Baseline
0.67
0.86
0.64
0.31
0.62
10
CroPAjpint
0.86
0.95
0.73
0.31
0.71
CroPA
0.92
0.98
0.70
0.34
0.74
Baseline
0.57
50
CroPAjaim
0.88
0.94
0.73
0.33
0.72
CroPA
0.95
0.99
0.67
0.40
0.75
Baseline
0.70
0.85
0.57
0.29
0.60
CroPAjiim
0.90
0.95
0.74
CroPA
0.96
0.99
0.68
0.44
0.77
Table 5: Targeted attack suecess rates tested on BLIP-2, The best performance values for each task are highlighted in bold.
No. Prompts Method
VQAgeaeal
VQA
Classification
Captioning
Overall
Baseline
0.24
0.34
0.45
0.32
0.34
CroPApiin
0.41
0.48
0.48
0.41
0.45
CroPA
0.52
0.63
0.65
0.58
0.60
Baseline
0.51
0.59
0.62
0.58
0.58
5
CroPAjain
0.80
0.83
0.75
0.81
0.80
CroPA
0.81
0.83
0.80
0.84
0.82
Baseline
0.68
0.81
0.71
10
CroPAjoint
0.83
0.83
0.75
0.79
0.80
CroPA
0.86
0.90
0.82
0.84
0.86
Baseline
0.67
0.74
0.67
0.72
0.70
CroPA
0.79
0.84
CroPA
0.90
0.93
0.87
0.91
0.90
Baseline
0.67
0.76
0.68
0.66
0.69
100
CroPA
0.87
0.93
0.81
0.87
0.87
CroPA
0.92
0.92
D CROPA WITH NON-TARGETED ATTACK GOAL
As presented in Table we tested the effectiveness our approach under the non-target attack setting. In this setting. the attack is considered to be successful if the model is misled to produce any different predictions. The CroPA method also demonstrated superior performance compared to the baseline approach under different numbers of prompts used in the optimisation.
20

---

Published as a conference paper at ICLR 2024
Table 6: Targeted attack success rates with RandomRotation as the defense strategy. During the optimisation phase, no additional data augmentation is used. The best performance values for each task are highlighted.
Setting
Method
VQAspecif:
Classification
Captioning
Overall
Without Defense
Multi-P
0.67
0.86
0.64
0.31
0.62
CroPA
0.92
0.98
0.70
0.34
0.74
With Defense
Multi-P
0.58
0.79
0.52
0.26
0.54
CroPA
0.89
0.95
0.61
0.34
0.70
Table 7: The targeted ASR results tested on Flamingo given different perturbation sizes: 8/255, 16/255 and 32/255. The best performance values for each task are highlighted in bold.
Perturbation size
Method
VQAgmeral
VQAspwifie
Classification
Captioning
Overall
8/255
Multi-P CroPA
0.37
0.59
0.56
0.06
0.39
0.53
0.75
0.48
0.03
0.45
16/255
Multi-P
0.67
0.86
0.64
0.31
0.62
CroPA
0.92
0.98
0.70
0.34
0.74
32/255
Multi-P
0.85
0.95
0.47
0.32
0.64
CroPA
0.98
0.99
0.59
0.42
0.75
Table 8: The cross model test under different settings. "BLIP2 to InstructBLIP" means the perturbations optimised on BLIP2 are tested on the InstructBLIP and "InstructBLIP to BLIP2" are different. The language model for BLIP2 is OPT-2.7b and the model for InstructBLIP is Vicuna-7b. The best performance values for each task highlighted in bold.
Settings
Method
VQAgeneral
VQAgpciiic
Classification
Captioning
Overall
BLIP2 to InstructBLIP
Multi-P CroPA
0.00
0.01
0.04
0.03
0.02
0.00
0.04
0.15
0.11
0.08
InstructBIIPP to BLIP2
Multi-P
0.04
CroPA
0.01
0.05
0.13
0.04
0.06
Table 9: The targeted ASR results when the target texts to harmful instructions. The best performance values for each task are highlighted in bold.
Target Texts
Method
VQA general
VQ specific
Classification
Captioning
Overall
suicide
Multi-P CroPA
0.69
0.83
0.75
0.61
0.72
0.84
0.91
0.91
0.78
0.86
kidnap
Multi-P
0.87
0.92
0.69
0.73
0.80
CroPA
0.94
0.96
0.88
0.81
0.90
bomb
Multi-P
0.68
0.82
0.90
0.60
0.75
CroPA
0.80
0.90
0.94
0.70
0.84
Table 10: The targeted ASR results when the target texts longer. The best performance values for each task highlighted in bold.
Target Texts
Method
VQA generala
VQAspecilic
Classification
Captioning
Overall
I am sorry
Multi-P
0.60
0.85
0.71
0.60
0.69
CroPA
0.90
0.96
0.75
0.72
0.83
cannot answer
Multi-P
0.40
0.66
0.33
0.07
0.37
CroPA
0.58
0.67
0.33
0.20
0.45
do not knnow
Multi-P
0.67
0.75
0.41
0.03
0.47
CroPA
0.70
0.80
0.43
0.04
0.49
need a new phone
Multi-P
0.68
0.86
0.85
0.53
0.73
CroPA
0.83
0.85
0.77
0.70
0.79
21

---

Published as a conference paper at ICLR 2024
Table 11: Non-targeted ASRs of the baseline and CroPA method when different number of prompts are presented. The mean and standard deviations the ASRs are shown the table. The best performance values for each task are highlighted in bold.
No. of Prompts
Method
gQAgeeeal
VQAspsciti
Classification
Captioning
Overall
Single-P
0.48-1.21e2
0.63
0.58:3
0.76
0.61:9.33e3
CroPA
0.50=111e2
1.64444.93er 33e-3
0.64 34e-3
0.74+ -.0e-2
0.63 93ee-3
Multi-P
0.50-1.41e22
0.69 43e-2
0.70
0.74
0.66:1.26e2
5
CroPA
0.64:1.89e.2
0.75
0.76 29e-22
0.79
0.74:1
10
Multi-P P
0.49
0.70 39e-3
0.3921.13e-2
0.763 43e-3
0.66=
CroPA
0.62 8.7228
0.7711.41e-2
0.77:1.07e>
0.79 1.8 98e-8
0.74:1.0844
Multi-P
0.52=9.
0.72+1.48er
0,68
0.73+ 25e-3
0.66=9.87e-3
50
CroPA
0.68=1.13e2
0.84+1.07e.2
0.81+1
0.84 6.03e3
0.79 98e
Multi-P
0.5121.23e2
0.7211.236-2
0.7321.31e-2
0.76.1 .44e22
0.6821.29--2
100
CroPA
0.69 8.02
0.84+1.22e.
0.89#1.01c
0.80 0.08e
0.81+1003e
22


Reviews:
[{"review_id": "489eh6tmx9", "reviewer": "Reviewer_HSuq", "summary": "The authors propose a method called Cross-Prompt Attack (CroPA) that creates adversarial images, destined for vision-language models (VLMs), that transfer across prompts.  They train on multiple prompts using a variant of projected gradient descent to learn both the image perturbation and the text embedding perturbation. They consider both the targeted and non-targeted scenarios. CroPA achieves better coss-prompt transferability compared to baseline methods when evaluated with OpenFlamingo-9B, BLIP-2, and InstructBLIP on different tasks.", "questions": "- This is a clarification question. Do you only add the prompt perturbations during the optimization phase or do you also add them during evaluation?\n\n- What is the ASR of Multi-P when transferred across models or images?\n\n- Do you have a conjecture regarding the inability of CroPA to achieve transferability across models or images?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper introduces an aspect of adversarial transferability that was not emphasized before. The problem that the paper tries to address is clearly formulated and is worth investigating.  \n- The experimental setup covers a wide range of scenarios: targeted and non-targeted attacks, multiple VLMs (OpenFlamingo-9B, BLIP-2,InstructBLIP) and different multi-modal tasks (VQA, classification, captioning). The baselines are strong.", "weaknesses": "- The method does not achieve transferability across models or images in addition to cross-prompt transferability. This might limit  its practical applicability.  \n\n- Clarifications are required in some parts of the paper. Certain statements about the “textual prompts” are misleading. For example, in the abstract, the method is described as follows:  “This proposed method updates the visual adversarial perturbation with learnable textual prompts”. This might suggest that the method directly modifies the textual prompts or discrete prompt tokens whereas it is actually modifying the embedding of these textual prompts. Moreover, the authors should explicitly state the underlying optimization algorithm (is it projected gradient descent?) , the norm (is it L-infinity ?) and the selected value of the perturbation size for easier comparison with related work.\n\nMinor issues: there are some typos in the paper (for example in the conclusion “baseline approaches only archive” -> “achieve”).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a54AsQDu2m", "reviewer": "Reviewer_FoXe", "summary": "This paper studies a novel perspective on adversarial transferability in the context of Vision and Language Models (VLM). In particular, it explores the problem of learning for adversarial visual patterns that can seamlessly traverse various textual prompts. The author introduces the method of a cross-prompt attack, akin to the principles of Generative Adversarial Networks (GANs), where the goal is to simultaneously learn visual perturbations and prompt perturbations with opposing objectives. Through compelling demonstrations, the study reveals that this competitive process significantly enhances the visual perturbation's ability to traverse different prompts effectively.", "questions": "How does the proposed method work for the longer prompt? Does the transferability still hold?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The problem being studied (cross-prompt adversarial transferability) and the proposed method seems very novel to me, and the author performed extensive experiments to validate the strength of the proposed method. I am convinced that this paper will provide valuable insights and knowledge to the research community.", "weaknesses": "I don’t see a major weakness, but have some suggestions for the naming. The term \"prompt perturbation\" initially appears a bit confusing, as it implies an intention to deceive the model (as it’s commonly used for adversarial “attach” rather than “defense”), which is not the case in this paper. Instead, it’s used in an opposite direction other than image perturbation to encourage stronger cross-prompt transferability during this competitive update process. I would recommend that the author consider selecting a more fitting name.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cz5xBHa1D0", "reviewer": "Reviewer_av8s", "summary": "This paper aims to generate a single adversarial image capable of misleading all predictions made by VLMs, regardless of the input prompts. To accomplish this objective, it introduces a Cross-Prompt Attack, characterized by a minimax optimization process involving image perturbations and learnable textual prompts. Extensive experiments have validated that, in a white-box setting, the proposed attack scheme can outperform other attack methods.", "questions": "In addition to the concerns in the weakness section, I also have the following question:\n\nIs the improvement achieved by the proposed Cross-Prompt Attack over Multi-P due to the increased number of textual prompts generated through perturbations during minimax optimization? One approach to achieve a similar effect is to utilize another language model to rephrase the text prompts into new ones, thus serving as data augmentation.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed method intuitively makes sense and consistently outperforms other baseline attack schemes in extensive experiments.\n\n2. The paper is well-written and easy to follow.", "weaknesses": "1. The primary concern for this paper is whether the proposed attack can indeed raise significant security issues. For instance, when considering the exemplary target prompts in Table 1, they have the potential to disrupt the functionality of VLMs, but it remains unclear how they may pose security risks in real-world applications. This security concern would be more valid if the attack successfully tricks VLMs into producing contradictory predictions or harmful instructions. Otherwise, the objective of this study appears to be more like a pioneering exploration of VLMs' adversarial robustness.\n\n2. The author mentions another potential use case for the proposed framework, which is to \"prevent malicious usage of large VLMs for unauthorized extraction of sensitive information from personal images.\" If this is indeed the case, the proposed method should also be assessed in a black-box setting, as users' VLMs cannot be known in advance. However, this set of experiments is absent from the paper. Previous jailbreaking attacks aimed at emerging large language models have shown substantial transferability across models [1]. I am curious whether this holds true for various VLMs as well.\n\n3. I am curious whether some straightforward defense techniques can defeat the proposed attack, such as incorporating data augmentation on the input images [2]. This could provide insights into whether the optimized adversarial perturbation converges to an unstable local minimum and can be easily defended against.\n\n[1] \"Universal and Transferable Adversarial Attacks on Aligned Language Models\", A. Zou et al., arXiv 2023.\n\n[2] \"The Best Defense Is a Good Offense: Adversarial Augmentation Against Adversarial Attacks\", I. Frosio et al., CVPR 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y6OqklY519", "reviewer": "Reviewer_Fnt8", "summary": "The paper proposes the Cross-Prompt Attack (CroPA) that generalizes the pixel-space image perturbation for multiple prompts. The authors propose to optimize learnable textual prompts in the opposite direction of the pixel-space adversarial perturbation for transferability. Evaluations on small-scale VLMs showed that CroPA was effective in fooling the target model, and the pixel-space perturbations are transferable across multiple types of prompts.", "questions": "1. Please address my concerns stated in the weakness section. Given the current status of the paper, I appreciate the problem novelty to rate it as borderline acceptance. However, the authors' responses shall address the weakness mentioned above. I look forward to further discussion and I will consider revising the rating based on the soundness of the response.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written and well-presented. The motivations are sufficiently reasonable to motivate the framework.\n\n2. The idea of jointly optimizing pixel-space perturbation and text-space perturbation is novel.\n\n3. The evaluations include multiple kinds of VLM paradigms with several major language tasks, which are comprehensive.", "weaknesses": "1. The paper assumes the white-box access for VLMs, which enables the framework to have backward gradients to update in multimodal input space. However, this assumption may not be generalized to all mainstream VLMs as they are fast scaling up. For instance, the authors evaluated the effectiveness of OpenFlamingo-9B and InstructBLIP, whose LLM component is not of large-scale. The applicability of CroPA on large-scale VLMs (e.g., those embedded with LLaMA-65B) or black-box VLMs (e.g., GPT4-V) remains challenging.\n\n2. The framework learns adversarial prompts to enhance the attack's effectiveness. However, very few text prompt instances are shown in the paper, and the quality of the result text prompts is not sufficiently evaluated.\n\n3. Existing adversarial attacks on VLM [1] have adopted query-based techniques to enhance the attack effectiveness. What's more, this baseline only assumes the black-box access to the VLMs, which more are generalizable. The paper should show sufficient validity or advantages (e.g., ASR, Convergence, etc.) of CroPA over existing baselines.\n\n4. Please fix the typo \"leanable\" in the introduction (page 2).\n\n[1] (NeurIPS 2023) On Evaluating Adversarial Robustness of Large Vision-Language Models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jJ54GEs2AB", "reviewer": "Reviewer_iSYx", "summary": "This work proposes the Cross-Prompt Attack (CroPA). This proposed method updates the visual adversarial perturbation with learnable textual prompts, which are designed to counteract the misleading effects of the adversarial image. By doing this, CroPA improves the transferability of adversarial examples across prompts.", "questions": "see the weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The experiments have shown the cross-prompt transferability created with a single prompt is highly limited. An intuitive approach to increase the cross-prompt transferability is to use multiple prompts during its creation stage. However, the improvement in cross-prompt transferability of these baseline approaches converges quickly with the increase in prompts. CroPA further improve the cross-prompt transferability. \n\nIt creates more transferable adversarial images by utilising the learnable textual prompts. The learnable textual prompts are optimised\nin the opposite direction of the adversarial image to cover more prompt embedding space. \n\nTo explore the underlying reasons for the better performance of CroPA compared to the baseline approach, it visualises the sentence embedding of the original prompt and perturbated prompts by CroPA, which is obtained by the averaging embedding of each token. The visualisation of difference in the prompt embedding coverage explains the reason why the CroPA methods can outperform the baseline approach even if the number of prompts used in optimisation is less than the baseline approach.\n\nIt experiments with different settings or policies. For example, it explores the effect of different update strategies. It tests the ASRs of the image adversarial examples with the number of in-context learning examples.", "weaknesses": "The technical contribution may be limited. The optimization method is general to use gradients for updating the adversarial perturbations. This optimization method is widely adopted in adversarial attacks.  It is very similar to adversarial training. To achieve cross prompt transferability, it perturbs the prompts without limitations to maximize the loss and obtain the worst prompt, then it trains the image perturbations based on the worst prompts and the final obtained perturbation can fool the model for different prompts because it already deals with the worst prompts. The novelty may be limited. \n\n\nI am still not very clear about some experiment settings. For example, what is the value of perturbation size $\\epsilon$? Larger $\\epsilon$ usually means stronger attacks. So what is the performance with a different $\\epsilon$?\n\nIt seems that the generated perturbations are used to test for the same model that it was trained. Can the generated perturbations be transferred to other VLMs and what are the performance? It is better to discuss the transferability to demonstrate the generalization of the method. \n\nFor the non-targeted attack after equation (2), the optimisation is expressed to maximize over token and minimize over image. It is almost the same as the targeted attack. I am wondering if there is a mistake here and it should be to maximize over image to make the results of adversarial examples different from the original one. Minimizing over images and the final image perturbation would make the outputs of adversarial examples similar to clean images. It is better to provide more details about this non-targeted attack. I agree that by maximizing the tokens, the generated tokens can mislead the VLM, but here we focus on image perturbations and use the image perturbation to test the VLM, right?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)