VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models
Authors: Ziyi Yin
Muchao Ye
Tianrong Zhang
Tianyu Du
Jinguo Zhu
Han Liu
Jinghui Chen
Ting Wang
Fenglong Ma
Venue: NeurIPS 2023 poster (2023)
Keywords: vision-language
adversarial attacks
pre-trained model
fine-tuned model

Abstract: Vision-Language (VL) pre-trained models have shown their superiority on many multimodal tasks. However, the adversarial robustness of such models has not been fully explored. Existing approaches mainly focus on exploring the adversarial robustness under the white-box setting, which is unrealistic. In this paper, we aim to investigate a new yet practical task to craft image and text perturbations using pre-trained VL models to attack black-box fine-tuned models on different downstream tasks. Towards this end, we propose VLATTACK to generate adversarial samples by fusing perturbations of images and texts from both single-modal and multi-modal levels. At the single-modal level, we propose a new block-wise similarity attack (BSA) strategy to learn image perturbations for disrupting universal representations. Besides, we adopt an existing text attack strategy to generate text perturbations independent of the image-modal attack. At the multi-modal level, we design a novel iterative cross-search attack (ICSA) method to update adversarial image-text pairs periodically, starting with the outputs from the single-modal level.  We conduct extensive experiments to attack three widely-used VL pretrained models for six tasks on eight datasets. Experimental results show that the proposed VLATTACK framework achieves the highest attack success rates on all tasks compared with state-of-the-art baselines, which reveals a significant blind spot in the deployment of pre-trained VL models.

VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models
Ziyi Yin' Muchao Ye Tianrong Zhang' Tianyu Du²
Jinguo Zhu Han Liu Jinghui Chen' Ting Wang" Fenglong Ma
'The Pennsylvania State University, "Zhejiang University.
3 Xi'an Jiaotong University, *Dalian University of Technology. SStony Brook University (ziy (ziylyin, muchao, tb25156, jcz5917, fenglong @psu. edu wtt.stu...t..uu....
z zjradty@zju edu. en, 1iu. han. dut @gmall com, twanglcs stonybrock. edu
Abstract
Vision-Language (VL) pre-trained models have shown their superiority on many multimodal tasks. However, the adversarial robustness of such models has not been fully explored. Existing approaches mainly focus on exploring the adversar- ial robustness under the white-box setting, which is unrealistic. In this paper, we aim to investigate a new yet practical task to craft image and text perturbations using pre-trained VL models to attack black-box fine-tuned models on different downstream tasks. Towards this end, we propose VLATTACK" to generate ad- versarial samples by fusing perturbations of images and texts from both single- modal and multimodal levels. At the single-modal level, we propose a new block- wise similarity attack (BSA) strategy to learn image perturbations for disrupting universal representations. Besides, we adopt an existing text attack strategy to generate text perturbations independent of the image-modal attack. At the mul- timodal level, we design a novel iterative cross-search attack (ICSA) method to update adversarial image-text pairs periodically, starting with the outputs from the single-modal level. We conduct extensive experiments to attack five widely-used VL pre-trained models for six tasks. Experimental results show that VLATTACK achieves the highest attack success rates on all tasks compared with state-of-the-art baselines, which reveals blind spot in the deployment of pre-trained VL models.
Introduction
The recent success of vision-language (VL) pre-trained models multimodal tasks have attracted broad attention from both academics and industry [1, 2, 3, 4, 5, 6, 7]. These models first learn the large-scale unlabeled image-text datasets and
multimodal interactions by pre-training later fine-tuned with labeled pairs on different downstream VL tasks [8. 9, 10]. In many cases, these pre-trained models have revealed more powerful cross-task learning capabilities compared to training from scratch [1 1, 12). Despite their remarkable performance, the adversarial robustness of these VI. models is still relatively unexplored.
Existing work [13, 14, 15, 16] conducting adversarial attacks in VL tasks mainly under the white- box setting, where the gradient information of fine-tuned models is accessible to attackers. However, in a more realistic scenario, a malicious attacker may only be able to access the public pre-trained models released through third parties. The attacker would not have any prior knowledge about the
Corresponding author.
"Souree code can be found in the link http:p::/gggtaa....o/////e/ee//p
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

"A painting of a woman with tree."" 4
Output
"Contradiction"
"Unknown"
Downstreum Task
Visual Entuilment
Visual Grounding
Visual question Answering
Image
Caption
VLLAttack: Generating Adversarial Perturbations
Pretruined Vision-Language Model
the duo are sining a song
the giant cruise ship
Which way is the cat's head turned? a
What does the image describe?
Input
Ground
Truth
"Neutral"
"Front
food Inside of them.
"Two bowls that have
Figure I: An illustration of the problem of attacking block-box downstream tasks using pre-trained vision-language models.
parameters learned by downstream VL models fine-tuned on private datasets. Towards bridging this striking limitation, we investigate new yet practical attack paradigm generating adversarial perturbations on a pre-trained VL model to atluck various black-box downstream tasks fine-tuned on the pre-trained one.
However, such an attack setting is non-trivial and faces the following challenges: (1) Task -pecifc challenge. The pre-trained VI. models are usually used for fine-tuning different downstream tasks, which requires the designed attack mechanism to be general and work for attacking multiple tasks. As illustrated in Figure 1, the attacked tasks are not only limited to close-set problems, such as visual question answering. but also generalized to open-ended questions, such as visual grounding. (2) Model-specific challenge. Since the parameters of the fine-tuned models are unknown, it requires attack method to automatically learn the adversarial transferability between pre-trained and fine- tuned models on different modalities. Although the adversarial transferability [17, 18, 19, 20] across image models has been widely discussed, is still largely unexplored the pre-trained models, especially for constructing mutual connections between perturbations on different modalities.
To address all the aforementioned challenges, we propose a new yet general Vision-Language Attack strategy (named VLATTACK) to explore the adversarial transferability between pre-trained and fine-tuned VL models, as shown in Figure 2. The whole VLATTACK scheme fuses perturbations of images and texts from two levels:
Step !
Single-modal
Multimodal
Block-wise
Image
Similurity Attack
heraaivv
Query
Step 2
I
Cross-Search
Attack
Text T
BERT-Autack
Fine-tuned Model
Step x
Pre-trained Model
Pre-trained Model
Figure 2: A brief illustration of VLATTACK.
Single-modal Level. VLATTACK independently generates perturbations on single modality, fol- lowing a "from image to text" order as the former can be perturbed on continuous space. The single-modal attack can effectively detect the adversarial vulnerability of an image or text, and hence avoids redundant perturbations teractions that have been stored in the pre-trained model, we propose novel block-wise similarity attack (BSA) strategy to attack the image modality, which adds perturbations to enlarge the network block-wise distance between original and perturbed features in the pretrained model, disrupting the universal image-text representations for the downstream predictions. If BSA fails to change the prediction after querying the fine-tuned black-box model, VLATTACK will attack the text modality by employing the word-level perturbation techniques [21, 22, 23, 24]. We adopt BERT-Attack [21] to attack the text modality as its prominent performance has been widely verified in many stud- ies [25, 26, 27]. Finally, if all the text perturbations {T fail, VLATTACK will generate a list of perturbed samples T and feed them to the multimodal attack along with the perturbed image I'.
the other modality. Specifically, to fully utilize the image-text in-
Multimodal Level. If the above attack fails to change the predictions, we cross-update image and text perturbations at the multimodal level based on previous outputs. The proposed iterative cross- search attack (ICSA) strategy updates the image-text perturbation pair (I,, T in iterative way

---

by considering the mutual relations between different modal perturbations, ICSA uses text pertur- bation Tę selected from the list T the guidance to iteratively update the perturbed image I by employing the block-wise similarity attack (BSA) until the new pair (T T makes the prediction of the downstream task change. In addition, text perturbations are cross-searched according to the semantic similarity with the benign one at the multimodal attack level, which gradually increases the extent of the direction modification to preserve the original semantics to the greatest extent.
Our contributions are summarized as follows: (1) To the best of our knowledge, we are the first to explore the adversarial vulnerability across pre-trained and fine-tuned VL models. (2) We pro- pose VLATTACK to search adversarial samples from different levels. For the single-modal level, we propose the BSA strategy unify the perturbation optimization targets on various downstream tasks. For the multimodal level, we design the ICSA to generate adversarial image-text pairs by cross-searching perturbations on different modalities. (3) To demonstrate the generalization ability of VLATTACK, we evaluate the proposed VLATTACK on five widely-used VL, models, includ- ing BLIP [28], CLIP [29]. ViLT [1], OFA [5] and UniTAB [4] for six tasks: (1) VQA, (ii) visual entailment, (iii) visual reasoning, (iv) referring expression comprehension, (v) image captioning, and (vi) image classification. Experimental results demonstrate that VLATTACK outperforms both single-modal and multimodal attack approaches, which reveals significant blind spot in the robust- ness of scale VL models.
2 Related Work
Single-modal Adversarial Attack Methods. Image Attack. Traditional image attack meth- ods [30, 31] generate adversarial samples by optimizing the loss function with regard to the de- cision boundary from model outputs. Not only do the generated perturbations change the model predictions, but they can also be transferred to other convolutional neural network (CNN) struc- tures. Such a property is called transfcrability and has been extensively studied. For example, data augmentation-basee methods [32, 33] endeavor to create diverse input patterns to enhance represen- tation diversity across different models. Feature disruptive attacks [17, 34, 35] introduce the inter- mediate loss to change the local activation of image features output from the middle layers of CNNs, enabling the perturbed features to transfer to different models without knowing their structures and parameters. Text Atlack, Adversarial attacks natural language processing (NLP) tasks mainly con- centrate on word-level and sentence-level perturbations. Word-level perturbations [21, 22, 23, 24] substitute words with synonyms that share similar word embeddings and contextual information. Sentence-leve perturbations [25, 36, 37] focus on logical structures of texts through paraphrasing or adding unrelated sentence segments. All of these methods have revealed the adversarial vulner- ability of traditional NLP models to some extent. However, adapting them to a multimodal attack setting is still underexplored.
Multimodal Adversarial Attack Methods. Multimodal VL models are susceptible to adversarial attacks as perturbations can be added to both modalities. Existing methods mainly explore adver- sarial robustness on a specific VL task. For the visual question answering task, Fool-VQA [13] is proposed, which iteratively adds pixel-level perturbations on images to achieve the attack. For the image-text retrieval task, CMLA [14] and AACH [38] add perturbations to enlarge Hamming distance between image and text hash codes, which causes wrong image-tex matching results. Re- cently, Co-attack [15] is proposed, which combines image and text perturbations using word sub- stitution attacks ascertain direction for guiding the multi-step attack images. The whole framework is deployed in a white-bos box setting, where attackers are assumed to have access to the parameters of each downstream task model. Besides, Co-attack is only validated on three tasks, which is not general enough.
3 Preliminaries
VL Pre-trained Model Structure. Fine-tuning from pre-training has become a unified paradigm in the recent VL model design. Most pre- pre-trained VL models can be divided into encoder-only [1, 39, 40, 41] and encooler-decoder [4, 5, 6) structures. Two representative model structures are shown in Figure 3. Given an image-text pair (I, T), both VI models first embed each modality separately. Image tokens are obtained through an image encoder composed of a vision transformer [42] or a Resnet [43] after flattening the grid features into sequence. Text tokens are generated through
3

---

linear
<an
<ans,> <ons?
<end>
Block My
Block
Cosine Similarity
:
Transformer Decoder
Block
Block
Transformer
Encoder
Block
Block
Transformer Fncoddr
Fd..y
F
<oks>
Image Encoder
Word Encoder
Image Encoder
Word Encoder
Word Encoder
Block M.
Block M.
Word
Block
Block
Eneoder
Image
TextT
Image
Text T
Far
El0)
(a)
(b)
Text
Image I
Image I'
Text T
Block wise similarity attack. Fa is the im- only (a) and encoder-decoder (b) structures. age encoder, and F3 is the Transformer encoder.
Figure 3: A brief illustration of the encoder-
Figure 4:
word encoder made up of tokenizer [44, 45] and word vector projection. The encoder-only model then attends tokens of two modalities and a learnable special token (cls) and feeds them into Transformer encoder. Finally, the output representation from the (cls) token is fed into classification head for the final prediction (ans). For the encoder-decoder structure, the attached image and text tokens are fed into a Transformer network [46] generate sequence predictions ((nns), (ams2),.. ... (endy)] in an auto-regressive manner. The network stops regressing when an end token (end) appears. In this work, we deeply explore the adversarial vulnerability of both structures.
Threat Model. Let F denote the public pre-trained model and S represent the downstream task model, where shares most of or the same structure with F. As shown in Figure 3, the network structures of both types of public VL pre-trained models are different. The encoder-only model allows modifying the last prediction layer based on the requirement of downstream tasks but keeping the other layers the same F. However, the encoder-decodee model unifies the outputs of different downstream tasks, which leads to S having the same structure as F. Note that the downstream tasks data, and hence, all the model parameters in F are
will fully fine-tune parameters using their updated in the fine-tuning stage. Thus, all parameters in S are not accessible.
Given image-text pair (I, T), the goal of the downstream task is to predict the labels of the input pair accurately, i.c., S T) + V, where Y = {ı1. yn For the encoder-only models, the ground truth label is a one-hot vector. For the encoder-decoder models, the ground truth is sequence that consists of multiple ordered one-hot vectors. Let y denote the ground truth vector. The goal of the adversarial attack is to generate adversarial examples (I', T') using F, which can cause an incorrect prediction on S. Mathematically, our problem formulated as follows:
max (S(', T') y), s.l. ||'' I| < oi, Cos(U.(T'), U.(T)) > O.. F.T
(1)
where di is the 460 -nomm perturbation strength on the image. ds is the semantic similarity between the original and perturbed texts, which constrains the semantic consistency after perturbation. The semantic similarity is measured by the cosine similarity Cos(-, between the sentence embedding UT' and U.(T) using the Universal Sentence Encoder U, [47].
Methodology
As shown in Figure the proposed VLATTACK generates adversarial samples from steps, constrained by perturbation budget parameters parameters a, and o., The first step attacks every single modality independently to avoid unnecessary modifications. Samples that fail at the first step will be fed into our multimodal level, where we adopt a cross-search attack strategy to iteratively refine both image and text perturbations the same time. Next, we will introduce the details of VLATTACK from the single-modal level to the multimodal level.
4.1 Single-modal Attacks: From Image to Text
VLATTACK attempts to generate adversarial samples every single modality in the first step. Compared to the discrete words T, the continuous values-based image more vulnerable to attack by using gradient information". Thus, VLATTACK starts by attacking the image I.
This motivation aligns with experimental results as shown in Table

---

Image-Attack. In the black-box setting, the parameters of the fine-tuned model S are unknown. However, the pre-trained model F is accessible. Intuitively, if the feature representations learned by the pre-trained model F from the clean input I and the perturbed input I' are significantly differ- ent, such perturbation may transfer to fine-tuned models to change the predictions of downstream tasks. As shown in Figure 3, the image features can be obtained from both the image encoder and the Transformer encoder, even from different layers or blocks. To fully leverage the specific charac- teristics of the pre-trained model structure, we propose the block-wise similarity attack (BSA) to corrupt universal contextualized representations.
As shown in Figure 4, BSA perturbs images by maximizing the block-wise distances between the intermediate representations the image encoder Fa. and Transformer encoder F3 of the pre-traines trained model F. Mathematically, we define the loss function of BSA as follows:
Mj
M
Mr M
C
Cos(F (I), F2
Cos(F kid B (I, T),, F T)),
(2)
Image Encoder
Transformer Encoder
where M, is the number of blocks in the image encoder, and Mj is the number of flattened image feature embeddings generated in the i-th block*. Similarly, M& is the number of blocks in the Transformer encoder, and M is the number of image token features generated in the k-th block. FiJ is the j-th feature vector obtained in the i-th layer of the image encoder, and F is the I-th feature vector obtained in the k-th layer of the Transformer encoder. The image encoder only takes single image or I' as the input, but the Transformer encoder will use both image and text as the input. We adopt the cosine similarity to calculate the distances between perturbed and benign features as token representations attended with each other in the inner product space [46]. Note that BSA A does not rely on the information from the decision boundary, and thus, it can be easily adapted to different task settings by disrupting the benign representations.
In the image attack step, we generate an adversarial image candidate through project gradient decent optimization [30] with N, iterations, where N, N and N is the maximum number of iterations on the image attack. The remaining N N₂ attack step budgets will be used in the multimodal attack in Section 4.2. If (I' T) is adversarial sample generated by BSA, then VLATTACK will stop. Otherwise, VLATTACK moves to attack the text modality.
Text-AAtack. In some VL tasks, the number of tokens in the text is quite small. For example, the average length of the text in the VQAv2 [48] and RefCOCO [49] datasets is 6.21 and 3.57, respectively". Moreover, some of them are nonsense words, which makes it unnecessary to design new approach for attacking the text modality. Furthermore, existing text attack approaches such as BERT-Attack [21] are powerful for generating adversarial samples for texts. Therefore, we directly apply BERT-Attack to generate text perturbations. To avoid unnecessary modifications, we use the clean image I as the input instead of the generation perturbed image I'.
Specifically, an adversarial candidate Ty produced by BERT-Attack is firstly fed into a univer- sal sentence encoder U. with the benign text T to test the semantic similarity. T will be removed if the cosine similarity % between Ty and T is smaller than the threshold a. i.e., % = (T,)) (T)) < as- Otherwise, we put Tę and the benign image into the fine- tuned model 5 to detect whether the new pair (I, T perturbs the original prediction y. During the text attack, we create a list T to store all perturbed samples (T)) and the corresponding cosine similarity values {ri]. If any sample in T successfully changes the prediction of the downstream task, then VLATTACK will stop. Otherwise, VLATTACK will carry the perturbed image I' from the image attack and the perturbed text candidate list T from the text attack to the multimodal attack level. Note that BERT-Attack is based on synonym word substitution. Even for short text, the number of adversarial examples can still be large. In other words, the length of T may be large. The whole single-modal attack process is shown in Algorithm I (lines 1-15).
'If image encoder is ResNet module, the output image feature from i-th block has shape of (H, We, C.). where Hi Wi and C. denote the height, width and number of channels, respectively. We then collapse the spatial dimensions of the feature and obtain Mj = H,W,
We obtained these statistics by investigating the validation datasets.
5

---

Algorithm VLATTACK
Input: A pre-trained model F, fine-tuned model S, a clean image-text pair (I, T) and its prediction y the S., and the Gaussian distribution 24;
Parameters: Perturbation budget Ø; on I, os on Iteration number Ng.
I: //Single-modal Attacks: From Image to Text (Section 4.1)
2: Initialize I' = I + 6, é Z4(0, 1),
3: // Image attack by updating I' uuing Eq. (2) for N, steps
I' BSA(C, I', T, N., oz. F)
S(L',))) T) + y then return (I,T)
6: else
7:
// Text attack by applying BERT-attack for pertubed text T, in BERT-attack do
8:
9:
if n = Cos(U, U, (T)) > then Add the pair (T. into T:
10:
11:
if S(I, T.) then return (I, T.) end if
12:
13:
14: end for
15: end if
16: // Multimodal Attack (Section 4.2)
17: Rank T according to similarity scores [73} and get top- samples (T) Tik according to Eq. (3);
18: for l: 1-=1, 1, K do
19: if S1, Tx, then return (I, T()
20: end if
21:
Replace (I,, T2) with (I', T) in Eq. (2);
22:
I BSA(C, Igs Ty NN, oi, P)
23:
if S Tv) # then return (T TÚ)
24:
end if
25:
end for
26:
return None
4.2 Multimodal Attack
In many cases, only perturbing images or texts is hard to succeed, as single-modality perturbation is insufficient to break down the image-text correlation. To solve this problem, we propose new attack strategy disrupt dynamic mutual connections based on perturbations from different modalities.
Attack Steps. Since the maximum number of image attack steps N is predefined, is impossible test all the perturbed image-text pairs (I', TV) using the remaining budget N - N₂ if the length of T (i.e.. (T] very large. Thus, we need to rank the perturbed text samples (T²} according their corresponding cosine similarity values {y: }o in descending order to make the adversarial sample keep the high semantic similarity with the original text Let K denote the number attack steps in the multimodal attack, and we have:
if T > N -Ns: if < N N N No: (17).
K
Iterative Cross-Search Attack. A naive way to conduct the multimodal attack is to test each perturbed image-text pair (I', T2) (k = --...,KK by querying the black-box fine-tuned model S, where T is the i-th text perturbation in the ranked list T. However, this simple approach ignores learning mutual connections between the perturbed text and image. To solve this issue, we propose iterative cross-search attack (ICSA) strategy. In ICSA, VLATTACK will dynamically update the image perturbation under the guidance of the text perturbation.
Specifically, in each multimodal attack step, ICSA first determines the number of image attack steps. Since there are adversarial text candidates that will be equally used ICSA, the iteration number of image attacks that will be allocated each text sample N₂ N ICSA will take the k-th
к
pair (I, TV as the input to generate the new image perturbation l2 by optimizing the block-wise wise attack loss in Eq. (2), where I is the output from the image attack in the single-modal level, i.e., I'.
Such process is repeated until an adversarial sample (I,, TT) is found, or all the K perturbed texts

---

Table 1: Comparison of VLATTACK with baselines ViLT, Unitab, and OFA for different tasks, respectively. All results are displayed by ASR (%). B&A means the BERT-Attack approach.
Pre-trained Model
Image Only
Text Only
multimodality
Task
Dataset
SSP
FDA
BSA
B&A
R&R
Co-Attack
VLÄTTACK
VQA
VQAv2
23.89
50.36
29.27
65.20
17.24
8.69
35.13
78.05
ViLT
VR
NLVR2
21.58
35.13
22.60
52.17
32.18
24.82
42.04
66.65
VQA
VQAv2
7.04
11.84
7.12
25.04
21.04
2.94
14.24
48.78
BLIP
VR
NLVR2
6.66
6.88
10.22
27.16
33.08
16.92
8.70
52.66
VQA
VQAv2
22.88
33.67
41.80
48.40
14.20
5.48
33.87
62.20
REC
RefCOCO
21.32
64.56
75.24
89.70
13.68
8.75
56.48
93.52
Unitab
REC
RefCOCO+
26.30
69.60
76.21
90.96
6.40
2.46
68.69
93,40
REC
RefCOCOg
26.39
69.26
78.64
91.31
22.03
18.52
65.50
95.61
VQA
VQAv2
25.06
33.88
40.02
54.05
10.22
2.34
51.16
78.82
VE
SNLI-VE
13.71
15.11
20.90
29.19
10.51
4.92
18.66
41.78
OFA
REC
RefCOCO
11.60
16.00
27.06
40.82
13.15
7.64
32.04
56.62
REC
RefCOCO+
16.58
22,28
33.26
46.44
4.66
7.04
45.28
58.14
REC
RefCOCOg
16.39
24.80
33.22
54.63
19.23
15.13
30.53
73.30
visited. When T 0, VLATTACK will degenerate to the block-wise similarity attack (BSA)
and iteratively updates I for N N, steps. Finally, VLATTACK generates adversarial samples based on either single-modality or cross-modality attacks. The overall scheme of VLATTACK is summarized in Algorithm 1.
5 Experiments
5.1 Experiment Setup
Pre-trained VL Models and Tasks. Experiments are conducted on five pre-trained models For the encoder-only model, we adopt ViLT [1] and BLIP [28] for two downstream tasks, including the vi- sual question answering (VQA) task on the VQAv2 dataset [48] and the visual reasoning (VR) task on the NLVR2 dataset [50]. For the encoder-decoder structure, we adopt Unitab [4] and OFA [5]. For Unitab, evaluations are made on the VQAv2 dataset for the VQA task and on RefCOCO, Re- fCOCO+, and RefCOCOg datasets [49] for the Referring Expression Comprehension (REC) task that can be viewed as the task of bounding box generation. For OFA, we implement experiments on the same tasks Unitab and add the SNLI-VE dataset [51] for the visual entailment (VE) task. The specific structures of these models are detailed in Appendix A. To verify the overall generality, evaluate the uni-modal tasks OFA [5] using MSCOCO [52] for the image captioning task and ImageNet-1K IK [53] for the image classification task. We also evaluate CLIP [29] on the image classification task on the SVHN [54] dataset. Note that all evaluation tasks have publicly available pre-trained and fine-tuned models, which provide more robust reproducibility. The details of cach dataset and implementation can be found in Appendix B.
Baselines. We compare VLATTACK with adversarial attacks on different modalities. For attacks on the image modality. we take DR [17], SSP [35], and FDA [34] as baselines. These methods are designed perturb image features only and can be directly adapted to our problem. Other methods [18, 19, 30, 32, 33, 55] either fully rely the output from classifiers or combine feature perturbation with classification loss [20, 56, 57J. These methods can not applied our problem setting since the pre-trained and fine-tuned models usually share different prediction heads and are trained on different tasks. For attacks on the text modality, we take BERT-Attack (B&A) [21] and R&R [24] as baselines. VLATTACK also compares with Co-Attack [15], which is the only multimodal attack scheme that adds adversarial perturbations to both modalities.
5.2 Results on Multimodal Tasks
In this section, we conduct experiments on four pre-trained VL models and use Attack Success Rate (ASR) to evaluate the performance on four multimodal tasks, including VQA, VR, REC, and VE. The higher the ASR, the better the performance. Results are illustrated in Table 1, where the results of our proposed BSA and VLATTACC are highlighted. We can observe that the proposed VI-AT- TACK significantly outperforms all baselines. Compared with the best baseline on each dataset,
7

---

Table 2: Evaluation of the Uni-modal tasks on OFA. We highlight the prediction score reported by the original OFA paper with *.
Dataset
MSCOCO
ImageNet- IK
Metric
BLEU84 (L)
METEOR (4)
CIDEr (L)
ST)
ASR(T)
OFA*
42.81
31.30
145.43
25.37
DR
30.26
24.47
95.52
17.89
10.43
SSP
10.99
12.52
23.54
5.67
19.44
FDA
17.77
17.92
55.75
11.36
12.31
BSA (Ours)
3.04
8.08
2.16
1.50
41.35
Table 3: CLIP model evaluation on SVHN.
Table 4: MTTrK evaluation the ViLT model using the VQAv2 dataset. Re- sults are obtained on 650 samples.
Dataset
SVHN
Model
CLIP-ViT/16
CLIP-RN50
DR
3.32
71.62
Method
SSP
VLATTACK
SSP
6.36
84.26
Definitely Correct (1)
413
307
FDA
6.20
83.52
Not Sure (f)
58
56
BSA (Ours)
15.74
84.98
Definitely Wrong (t)
179
287
VLATTACK achieves an average gain of 29.61% on ViLT, 24.20% on BLIP, 16.48% on Unitab and 30.84% on OFA, respectively. When only adding image perturbations, our proposed BSA out- performs the best bascline by an average of 15.94% on ViLT and BLIP. BSA also achieves an average gain of 12.12% and 14.13% on Unitab and OFA, respectively. As mentioned before, text data in multimodal tasks are composed of phrases or only a few words. Consequently, text attack performs poorly on most multimodal datasets. These high ASR values have revealed significant security concerns regarding the adversarial robustness of pre-trained VL models.
5.3 Results on Uni-Modal Tasks
For the results on uni-modal tasks, we first evaluate OFA on image captioning and classification tasks on MSCOCO and ImageNet-IKK respectively. Because these tasks accept fixed text prompt for prediction, we only perturb image and compare different image attack methods. For the captioning task, choose four commonly the same as those used in OFA [5]. The lower, the better. We report ASR on the image classification task. The higher, the better. The experimental results are illustrated in Table 2. In general, BSA achieves the best performance on both tasks in terms of all metrics. For example, BSA significantly reduces the CIDEr score from 145.43 2.16. Compared with the best baseline SSP with 23.54, BSA reduces this score over 10 times. For the image classification task, BSA also achieves the best performance with 41.35% ASR. The superior performance demonstrates that the VLATTAAK can also generalize to computer vision tasks when only using BSA. Note that we do not evaluate text- to-image generation tasks because our attack framework produces the same results as BERT-Attack with text-only inputs.
metrics, including BLEU@4, METEOR, CIDEr, and SPICE,
We also evaluate CLIP on the image classification task the SVHN dataset. Concretely, the image encoder of CLIP as the pre-trained model and then fine-tune on the SVHN dataset after adding a linear classification head. For the choices of image encoder of CLIP, adopt ViT-B/16 and ResNet-50, denoted by CLIP-ViT/16 and CLIP-RN5O. We test the attack performance using 5K correctly predicted samples. All results illustrated in the following Table Since the task only accepts images as input, we compare our BSA with other image attack baselines. As shown in the table, our proposed BSA still maintains the best ASR using different image encoder structures, clearly demonstrating its effectiveness.
5.4 Systematic Validation with Crowdsoureing
We also conduct a human evaluation study to comprehensively verify the soundness of our proposed method. The experiment is developed on the results output from the ViLT model on the VQAv2 dataset using Amazon Mechanical Turk (MTurk). The baseline that we choose is SSP since out- performs others in our experiments, shown in Table 1. Specifically, we randomly sampled 650 examples and stored the generated answers after the attack. To validate the performance of both approaches, we send the original image-question pairs and the corresponding generated answer to

---

TE BSA BSA+BA
BSA+BA-Q
VLATTACK
75
ASR (%)
40
80
(a) OFA-VQAV2
(b) OFA-SNLI-VE
(c) Unish-VQQAv2
(d) Unitab-RetCOCO
Figure 5: Ablation analysis of different components in VLATTACK. We show the results of VQAv2 (a) and SNLI-VE (b) on OFA, and VQAv2 (c) and RefCOCO (d) on Unitab.
the MTurk system. We provide three choice candidates to workers, including "Definitely Correct", "Not Sure", and "Definitely Wrong". A successful attack means that the worker will label "Defi- nitely Wrong" to the pair. To make the annotations more accurate and reliable, each pair is annotated by three workers. and we report the majority choice as the final human evaluation result. The statis- ties of human evaluation results are shown in Table 4. We can observe that the proposed VLAttack still significantly outperforms the strongest baseline SSP, thus demonstrating its robustness and ef- fectiveness from human perceptual perspective.
5.5 Model Design Analysis
Ablation Study. In our model design, we propose a new block-wise similarity attack (BSA) for attacking the image modality and interactive cross-search attack (ICSA) for attacking image-text modalities together. We use the following methods to evaluate the effectiveness of each component. The results are shown in Figure 5. "IE"/TE" means that we only the image/Transforme coder when calculating the loss in Eq. (2). "BSA" uses both encoders. We set iteration N, = 40 for a fair comparison. Next, "BSA+BA" means after attacking images using BSA, we attack texts using BERT-Ataack (Algorithm Lines 1-15). "BSA+BA+Q" denotes replacing ICSA with a sim- ple query strategy by querying the black-box task with each pair (I', TD, where Tý comes from the perturbed text candidate list T. We can observe that for image attacks, IE and TE play an im- portant role in different tasks, but in general, BSA (= IE TE) outperforms them. Adding the text attack, BSA+BA performs better than BSA. This comparison result demonstrates that both modali- ties are vulnerable. BSA+BA+Q performs better than BSA+BA but worse than VLATTACK, which confirms the necessity and reasonableness of conducting the interactive cross-search attack. More results on other datasets are shown in Appendix C.
Parameter Sensitivity Analysis We discuss the effect of different iteration numbers of N and N. in VLATTACK. All experiments are conducted on the VQAv2 dataset and the ViLT model. The total iteration number N set from 10 to 80, .₂ is set to T. As depicted in Figure 6(a), the ASR performance dramatically improved by increasing N from 10 to 20 steps and then achieves the best result when N 40. We next investigate the impact of differ- ent initial iteration numbers N.. We test N, from 5 to 40, but the total iteration number N is fixed to 40. As shown in Figure 6(b), the ASR score reaches the summit when N, 5, and smoothly decreased by continually en- larging N,. Considering that the smaller initial iteration number N, increases the ratio of text perturbations, we set N₂ 20 to obtain the best trade-off between attack performance and the naturalness of generated adversarial samples in our experiments.
ASE(1)
30
(83
(5) Nemitoo Numbor 3G
Figure 6: Investigation of iteration num- ber N and N..
Bisek-wise Similarity Attack. We further visualize the effect of our proposed BSA through atten- tion maps in the fine-tuned downstream model S on the VQAv2 dataset through the ViLT model. As shown in Figure 7, the attention area is shifted from "jacket" to the background, which indicates that the representation used to answer the question extracts information from the unrelated image regions. We also combined the BSA with various optimization methods to verify its generalization ability. The experimental results are presented in Appendix D.

---

5.6 Case Study
We conduct case studies using visualization to show the effectiveness of VLATTACK for mul- timodal and uni-modal tasks, as displayed in
Cleam
Adversarial
Figure 8. For multimodal tasks in Figure 8(a), the predictions of all displayed samples remain unchanged when attacking single modality via BSA. However, perturbing both modali- ties using VLATTACK successfully modifies the predictions. An intriguing finding can be observed in the REC task, where the fine-tuned model outputs wrong predictions by stretch- ing and deforming the original bounding box predictions. The proposed VLATTACK also shows its power uni-modal tasks in Fig- ure 8(b). For example, in the image captioning task, the fine-tuned model generates a description of "person" after receiving adversarial image of a "horse", which is entirely unrelated. More studies are shown in Appendix E.
What color the man's jacket?
What colour is the man's jacket?
selsia
Figure 7: Attention maps of the clean and adver- sarial samples in the multimodal transformer coder. Attention maps from the class token (cls) images. Perturbed tokens are displayed in red.
Visual Question
Referring Expression
Visual
Image
Image
Answering
Comprehension
Entailment
Captioning
Classification
Clean
What species this pboto? curdinal
A man in parple jeans.
A black dug jumpging to eatch frisbee, Contrudiction
A pony is standing trees.
Fox Squirvel
Ad adverssrial
What (ype of bird is in chis photo? mu!
man a purple shirt and deninr.
A black dogs jumping to Neutrul
close up of person umbrelll..
Conch
(a) Multi-modal Tasks
(b) Uni-modal Tasks
Qualitative results of VLATTACK on (a) multimodal tasks and (b) Uni-modal tasks OFA. Perturbed word tokens and original predictions are displayed in red and blue, respectively. We show the predictions after the adversarial attack with underline.
Figure 8:
Conclusion
In this paper, we propose question, which aims to generate adversarial samples only based on the publicly available pre-trained models to attack fine-tuned models deployed in different VL tasks. Considering the task-specific and model-specific challenges, we proposed VLATTACK, which gen- erates perturbations by exploring different modalities in two levels. The single-modal level attack first perturbs images through novel algorithm BSA to disrupt the universal image-text represen- tations and then attacks text if the former fails, which avoids unnecessary modifications on both modalities. If both image and text attacks fail, the multimodal attack level adopts an iterative cross- search attack strategy to generate adversarial image-text combinations, By periodically substituting text candidates during the image attack process, the mutual relations between different modal pertur- bations are sufficiently explored. Experimental results demonstrate the effectiveness of the proposed VLATTACK on attacking multiple VL tasks, which reveals a significant safety concern in realistic scenarios.
Acknowledgements This work is partially supported by the National Science Foundation under Grant No. 1951729, 1953813, 2119331, 2212323, and 2238275.
10

---

References
[1] Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: Vision-and -onoo-ua-- transformer without convolution region supervision. In ICML, volume 139 of Proceedings of Machine Learning Research, pages 5583-5594, PMLR, 2021.
[2] Zhicheng Huang. Zhaoyang Zeng, Bei Liu, Dongmei Fu, and Jianlong Fu. Pixel-bert: Aligning image pixels with text by deep multi-modal transformers. CoRR, abs/2004.00849, 2020.
[3] Xizhou Zhu, Jinguo Zhu, Hao Li, Xiaoshi Wu, Hongsheng Li, Xiaohua Wang. and Jifeng Dai. Uni-perceiver: Pre-training unified architecture for generic perception for zero-shot and few-shot tasks. In CVPR, pages 16783- 16794. IEEE, 2022.
[4] Zhengyuan Yang. Zhe Gan, Jianfeng Wang, Xiaowei Hu, Faisal Ahmed, Zicheng Liu, Yumao Lu, and Lijuan Wang. Unitab: Unifying text and box outputs for grounded vision-language modeling. ECCV (36), volume 13696 of Lecture Notes in Computer Science, pages 521- 539. Springer, 2022.
[5] Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. OFA: unifying architectures, tasks, and modalities through simple sequence-tor equeenc-to--eqqeenee sequenceece learning framework. In ICML, volume 162 of Proceedings of Machine Learning Research, pages 23318 23318-23340. PMLR, 2022.
[6] Jiasen Lu, Christopher Clark, Rowan Zellers, Roozbeh Mottaghi, and Aniruddha Kemb- havi. Unified-io: A unified model for vision, language, and multi-modal tasks. CoRR, abs/2206.08916 2022.
[7] Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojciech Galuba, Mareus Rohrbach, and Douwe Kiela. FLAVA: A foundational language and vision alignment model. In CVPR, pages 15617-15629. IEEE, 2022.
[8] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visi- olinguistic representations for -ain--and-language tasks. NeurIPS, pages 13-23, 2019.
[9] Zaid Khan, B. G. Vijay Kumar, Xiang Yu, Samuel Schulter, Manmohan Chandraker, and Yun Fu. Single-stream multi-level alignment for vision-language pretraining. In ECCV (36), vol- ume 13696 of Lecture Notes in Computer Science, pages 735-751. Springer, 2022.
[10] Aishwarya Agrawal, Damien Teney, and Aida Nematzadeh. Vision-language pretraining: Cur- rent trends and the future. In ACL (Iutorial), pages 38-43. Association for Computational Linguistics, 2022.
[11] Salman H. Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shah- baz Khan, and Mubarak Shah. Transformers vision: A survey. ACM Comput. Surv, 24<100):200:1-200:
2022.
[12] Yifan Du, Zikang Liu, Junyi Li, and Wayne Xin Zhao. A survey of vision-language pre-trained models. In IJCAI, pages 5436-5443. ijeai.org, 2022.
[13] Xiaojun Xu, Xinyun Chen, Chang Liu, Anna Rohrbach, Trevor Darrell, and Dawn Song. Fool- ing vision and language models despite localization and attention mechanism. In CVPR, pages 4951 4961. Computer Vision Foundation/ IEEE Computer Society, 2018.
[14] Chao Li, Shangqian Gao, Cheng Deng, De Xie, and Wei Liu. Cross-modal learning with adversarial samples. In NeurlPS, pages 10791-10801, 2019.
[15] Jiaming Zhang. Qi Yi, and Jitao Sang. Towards adversarial attack on vision- lisqua-language pre- training models. In ACM Multimedia, pages 5005-5013. ACM, 2022.
[16] Yao Zhou, Jun Wu, Haixun Wang, and Jingrui He. Adversarial robustness through bias vari- ance decomposition: new perspective for federated learning. In Mohammad Al Hasan and Li Xiong, editors, Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA, October 17-21, 2022, pages 2753-2762 ACM, 2022.

---

[17] Yantao Lu, Yunhan Jia, Jianyu Wang, Bai Li. Weiheng Chai, Lawrence Carin, and Senem Veli- pasalar. Enhancing cross-task black-box transferability of adversarial examples with dispersion reduction. In CVPR, pages 937-946. Computer Vision Foundation IEEE, 2020.
[18] Zhipeng Wei, Jingjing Chen, Micah Goldblum, Zuxuan Wu, Tom Goldstein, and Yu-Gang Jiang. Towards transferable adversarial attacks on vision transformers. In AAAI, pages 2668- 2676. AAAI Press, 2022.
[19] Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In ICLR. Open- Review.net, 2020.
[20] Qian Huang, Isay Katsman, Zeqi Gu, Horace He, Serge J. Belongie, and Ser-Nam Lim, En- hancing adversarial example transferability with an intermediate level attack. In ICCV, pages 4732-4741. IEFF, 2019.
[21] Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. BERT-ATTACK: Association
adversarial attack against BERT using BERT. In EMNLP (I), pages for Computational Linguistics, 2020.
[22] Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In AAAI, pages 8018- 8025. AAAI Press, 2020.
[23] Jinfeng Li. Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Textbugger: Generating adversarial text against real-world applications. In NDSS. The Internet Society, 2019.
[24] Lei Xu, Alfredo Cuesta-Infante, Laure Berti-Équille, and Kalyan Veeramachaneni. R&r: Metric-guided adversarial sentence generation. In AACI/IJCNIP (Findings), pages 438-452. Association for Computational Linguistics, 2022.
[25] Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. Adversarial GLUE: A multi-task benchmark for robustness evaluation of language models. In NeurlPS Datasets and Benchmarks, 2021.
[26] Dianqi Li, Yizhe Zhang, Hao Peng. Liqun Chen, Chris Brockett, Ming-Ting Sun, and Bill Dolan. Contextualized perturbation for textual adversarial attack. In NAACL-HLT, pages 5053- 5069. Association for Computational Linguistics, 2021.
[27] Guoyang Zeng, Fanchao Qi, Qianrui Zhou, Tingji Zhang, Bairu Hou, Yuan Zang, Zhiyuan Liu, and Maosong Sun. Openattack: An open-source textual adversarial attack toolkit. CoRR, abs/2009.00 1991,, 2020.
[28] Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. BLIP: bootstrapping language- image pre-training for unified vision-language understanding and generation. In /CML, volume 162 of Proceedings of Machine Learning Research, pages 12888-12900. PMLR, 2022.
[29] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pages
PMLR, 2021.
[30] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR (Poster). Open- Review.net, 2018
[31] Ian J. Goodfellow, Jonathon Shiens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. In ICLR (Poster), 2015.
[32] Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adver- sarial examples by translation-invariint attacks. CoRR, abs/19044 abs/1904.00284, 2019.
[33] Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L. Yuille. Improving transferability of adversarial examples with input diversity. In CVPR, pages 2730-2739. Computer Vision Foundation / IEEE, 2019.
12

---

[34] Aditya Ganeshan, Vivek B. S., and Venkatesh Babu Radhakrishnan. FDA: feature disruptive attack. In ICCV, pages 8068 8078. IEEE, 2019.
[35] Muzammal Naseer, Salman H. Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A self-supervised approach for adversarial robustness. In CVPR, pages 259-268. Computer Vision Foundation IEEE, 2020.
[36] Mohit lyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. Adversarial example gen- eration with syntactically controlled paraphrase networks. NAACL-HILT, pages 1875-1885. Association for Computational Linguistics. 2018.
[37] Aakanksha Naik, Abhilasha Ravichander, Norman M. Sadeh, Carolyn P. Rosé, and Graham Neubig. Stress test evaluation for natural language inference. In COLING, pages 2340-2200 2353. Association for Computational Linguistics. 2018.
[38] Chao Li, Shangqian Gao, Cheng Deng, Wei Liu, and Heng Huang. Adversarial attack on deep cross modal hamming retrieval. In ICCV, pages
IEEE, 2021.
[39] Hao Tan and Mohit Bansal. LXMERT: learning cross- encoder representations from transformers. In EMNLP/IJCNLP (I), pages 5099-5110. Association for Computational Lin- guistics, 2019.
[40] Junnan Li, Ramprasaath R. Selvaraju, Akhilesh Gotmare, Shafiq R. Joty, Caiming Xiong, and Steven Chu-Hong Hoi. Align before fuse: Vision and language representation learning with 2021.
momentum distillation. In NeurlPS, pages
[41] Yen-Chur Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. UNITER: universal image-text representation learning. In ECCV (30), vol- ume 12375 of Lecture Notes in Computer Science, pages 104-120. Springer, 2020.
[42] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR. OpenReview.net, 2021.
[43] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778. IEEE Computer Society, 2016.
[44] Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In ACL (1). The Association for Computer Linguistics, 2016.
[45] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abss/107..n 1692, 2019.
[46] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz. Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, 0tls/1706.07622, 2017.
[47] Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guzjardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil. Universal sentence encoder for english. In EMNLP (Demonstration) pages 169-174. Associ- ation for Computational Linguistics, 2018
[48] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitmick, and Devi Parikh. VQA: visual question answering. In ICCV, pages 2425-2433. IEEE Computer Society, 2015.
[49] Licheng Yu, Patrick Poirson, Shan Yang, Alexander C. Berg, and Tamara L. Berg. Modeling context in referring expressions. In ECCV (2), volume 9906 of Lecture Notes in Computer Science, pages 65-85. Springer, 2016.
[50] Alane Suhr, Stephanie Zhou, Iris Zhang, Huajun Bai, and Yoav Artzi. A corpus for reasoning about natural language grounded in photographs. CoRR, Abs/1811 /0///811.0000 1.04911 2018.
13

---

[51] Ning Xie, Farley Lai, Derek Doran, and Asim Kadav. Visual entailment: A novel task for fine-grained image understanding. CoRR, abs/1901.06706 2019.
[52] Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO captions: Data collection and evaluation server. CoRR, abs/1504.00325. 2015.
[53] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database. In CVPR, pages 248-255. IEEE Computer Society, 2009.
[54] Ian J Goodfellow, Yaroslav Bulatov, Julian Ibarz, Sacha Amoud, and Vinay Shet. Multi-digit number recognition from street view imagery using deep convolutional neural networks. arXiv preprint arXiv: 1312.0022. 2013.
[55] Jiafeng Wang, Zhaoyu Chen, Kaixun Jiang, Dingkang Yang, Lingyi Hong, Yan Wang, and Wenqiang Zhang. Boosting the transferability of adversarial attacks with global momentum initialization. CoRR, abs/221 1.1 1236, 2022.
[56] Nathan Inkawhich, Kevin J. Liang, Binghui Wang. Matthew Inkawhich, Lawrence Carin, and Yiran Chen. Perturbing across the feature hierarchy to improve standard and strict blackbox attack transferability. In NeurlPS, 2020.
[57] Nathan Inkawhich, Kevin J. Liang, Lawrence Carin, and Yiran Chen. Transferable perturba- tions of deep feature distributions. In ICLR. OpenReview.net, 2020.
[58] Patrick Esser, Robin Rombach, and Björ Ommer. Taming transformers for high-resolution image synthesis. In CVPR, pages 12873- 12883. Computer Vision Foundation IEEE, 2021.
[59] Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial machine learning at scale. ICLR (Poster). OpenReview.net, 2017.
[60] Jinyuan Jia, Yupei Liu, and Neil Zhenqiang Gong. Badencoder: Backdoor attacks to pre- trained encoders in supe-supe learning. In 2022 IEEE Symposium Security and Privacy (SP), pages 2043-2051 IEEE, 2022.
14

---

A. Details of VL Models
This section introduces the details of VL models involved in our experiment, including ViLT, BLIP, Unitab, and OFA.
ViLT. We first select ViLT [1] as the encoder-only VL model because of its succinct structure and prominent performance on multiple downstream tasks. Given an input image I E R 2f xxxx3 and a sentence T, ViLT yields . image tokens using linear transformation on the flattened image patches, where each token is a ID vector and M = -pr tokens are encoded through Byte-Pair Encoder (BPE) [44] and a word-vecton linear projection. Then, tokens of two modalities and a special learnable token (cls) are concatenated. By attending visual and text tokens and a special token (cls) in a Transformer encoder with twelve layers, the output feature from the (cls) token is fed into task-specific classification head for the final output. Taking the VQA task as an example, the VQA classifier adopts a linear layer to output a vector with H3 elements, where H3 is the number of all possible choices in the closed answer set of the VQA task. The final output is obtained through the element with the highest response in the vector.
HW for given patch resolution (P. P). Word
BLIP. The BLIP model also adopts an encoder-only structure. Specifically, BLIP first encodes an image through twelve-layer vision transformer VIT-B/16 [42], and generates word token embed- dings through the BPE and a linear layer. Next, the image features and word token embeddings are then fed into multimodal encoder. The structure of the multimodal encoder is the same as a twelve-layer transformer decoder, where each layer contains a self-attention module, a cross- attention module and a feed-forward module. In each layer, the multimodal encoder first accepts the word token embeddings as input and attends to them through the self-attention module. Then, the updated embedding will fuse with image features through the cross-attention module. Finally, the output from the multimodal encoder will be fed into different projection heads for downstream tasks, just like the ViLT model.
Unitab adopts an encoder-decoder framework. It first embeds text T via RoBERT, [45] and flats features after encoding image through ResNet [43]. The attached visual and text token features are then fed into a standard Transformer network [46] with six encoder layers and six decoder layers. Finally, the sequence predictions [(ans)), (ans2), ... (end)) are obtained auto- regressively through projection head. The network stops regressing when end token (end) appears. For different tasks, the output tokens may come from different pre-defined vocabularies. Given the REC task as an example, four tokens [[(oo I1), (loe x2))). ((loe X3), (loe 24)) will be selected from the location vocabulary, which forms the coordinate of a bounding box. As a result, these models can handle both text and grounding tasks.
Unitab.
OFA. OFA also adopts an encoder-decoder structure. Different from Unitab, it adopts the BPE encode text and extends the linguistic vocabulary by adding image quantization tokens [58] (img) for synthesis tasks. Note that the main difference between OFA and Unitab lies in their pre-training and fine-tuning strategies rather than the model structure. For example, in the pre-training process, Unitab focuses on learning alignments between predicted words and boxes through grounding tasks, while OFA captures more general representations through multi-task joint training that includes both single-modal and multimodal tasks. Overall, OFA outperforms Unitab in terms of performance improvement.
B. Dataset and Implementation
B.1 Tasks and Datasets
To verify the generalization ability of our proposed VLATTACK, we evaluate wide array of popular vision language tasks summarized in Table 5. Specifically, the selected tasks span from text under- standing (visual reasoning, visual entailment, visual question answering) to image understanding (image classification, captioning) and localization (referring expression comprehension).
For each dataset, we sample 5K correctly predicted samples in the corresponding validation dataset to evaluate the ASR performance. All validation datasets follow the same split settings as adopted in the respective attack models. Because VQA is multiclass classification task, we select a correct
15

---

Table 5: An illustration of all datasets and tasks evaluated in our paper.
Attack Model
Attack Modality Image
Datasets
Task
Task description
OFA
Unitab ViLT
Text
VQA+2
VQA
Scene Understanding QA
SNLI-VE
VE
VL Entailment
RefCOco
REC
Bounding Box Localization
RefCOCOg
REC
Bounding Box Localization
RefCOCO+
REC
Bounding Box Localization
NLVR2
VR
Image-Text Pairs Matching
MSCOCO
Captioning
Image Captioning
ImageNet-It KK
Classification
Ohject Classification
SVHN
Classification
Digit Classification
prediction only if the prediction result is the same as the label with the highest VQA score", and regard the label as the ground truth in Eq. (1). In the REC task, correct prediction is considered when the Intersection-over-Uuu Union (IoU) score between the predicted and ground truth bounding box is larger than 0.5. We adopt the same IoU threshold as in Unitab [4] and OFA [5].
B.2 Implementation Details
For the perturbation parameters of images, follow the setting in the common transferable image attacks [18, 19] and set the maximum perturbation of each pixel to 16/255 on all tasks except REC. Considering that even a single coordinate change can affect the final grounding results great extent, the σ; of the REC task is 4/255 to better highlight the ASR differences among distinct methods. The total iteration number and step size are set 40 and 0.01 by following the projected gradient decent method [30], and N, is 20. For the perturbation on the text, the semantic similarity constraint o, set 0.95, and the number of maximum modified words is set to I by following the previous text-attack work [15, 24] to ensure the semantic consistency and imperceptibility. All experiments are conducted on single GTX A6000 GPU.
C. More Ablation Results
In Section 5.4, conduct ablation study show the effectiveness of each module in model design on VQA, VE, and REC tasks. Here, we conduct additional ablation experiments for the remaining tasks, including visual reasoning, image captioning, and image classification.
60
6.0
ASR (%6
TE
Caption Segre
5.0
BSA
BSA
BSAIBA
4.0
BLCU@@
BSA-BA+Q VLAFTACK
3.0
CIDEr
30
Figure 9: VILT-VR.
Figure 10: OFA-captioning.
Figure 9 shows the results of the ablation study on the VR task using the ViLT model. We can observe that only using the image encoder results significantly low ASR. However, by combining with the Transformer encoder (TE), BSA can achieve high ASR. These results show the reason- ableness of considering two encoders simultaneously when attacking the image modality. The result of BSA+BA demonstrates the usefulness of attacking the text modality. Although BSA+BA+Q out- performs other approaches, performance is still lower than that of the proposed VLATTACK. This comparison proves that the proposed iterative cross-search attack (ICSA) strategy is effective for the multimodal attack again.
'The VQA score calculates the percentage of the predicted answer that appears in 10 reference ground truth answers. More details be found via https http///vaaaaaaeeee wwwp://l..aaaaaaaeeeeeeaeaaaaaaa eealluataaataaeaeeee ion himl
16

---

Figure 10 shows the results of the image captioning task using the OFA model. Because the image captioning task only accepts a fixed text prompt for prediction, we only perturb the image and report the results on IE, TE, and BSA. For this task, we report BLEU @4 and CIDEr scores. The lower, the better. We can observe that the proposed BSA outperforms IE and TE, indicating our model design's 's effectiveness.
Figure 11 shows the results of the image classification task using the OFA model. The experiment is conducted on the ImageNet- 1K dataset. Similar to the image captioning task, we only attack images, and compare the results of IE. TE and BSA. The evaluation metric for this task is ASR. The higher, the better. We can have the same observations with other ablation studies, where attacking both encoders outperforms attacking a single encoder.
40-
ASR(%)
L
IF Figure 11:
TE OFA-classifiction
BSA
Table 6: Combining VLATTACK with different gradient- baase image attack schemes.
ViLT
Unitab
Method
VQAv2
NLVR2
VQAv2
ReiCOCo
ReiCOCO+
ReiCOCOg
BSA
65.40
52.32
50.38
86.00
89.20
87.39
VLATTACKMT
78.77
67.16
63.02
92.46
93.10
94.34
BSA
63.94
52.30
42.74
90.30
91.56
91.00
VLATTACK
78.07
67.50
61.22
93.98
94.04
94.76
D. Different Optimization Methods
VLATTACK can be easily adapted to various optimization methods in image attacks. To demon- strate the generalizability of method, we replace the projected gradient decent [30] in VLAT- TACK with Momentum Iterative method (MI) [55] and Diverse Input attack (DI) [33] since they have shown better performance than traditional iterative attacks [30, 59]. The replaced methods denoted by BSAMS, and VLATTACK MI using MI, BSA and VLATTACK using DI, respec- tively. Experiments
developed on ViLT and Unitab. Results shown in Table 6. Using MI and DI optimizations, BSAMI and BSApr still outperform all baselines displayed in Table in the main manuscript. Also, VLÄTTACKMI and VLATTACK outperform the image attack method BSAMI and BSADI with average ASR improvement of 9. 9.70% and 9.29% among all datasets. The gain of performance demonstrates that the proposed VLATTACK can be further improved by combining with stronger gradient-based optimization schemes.
E. Case Study
E.1 How does VLATTACK generate adversarial samples?
The proposed VLATTACK aims to attack multimodal VL tasks starting by attacking single modali- ties. If they are failed, VLATTACK uses the proposed interactive cross cross-search attack (ICSA) strategy generate adversarial samples. In this experiment, we display the generated adversarial cases from different attack steps, including the image modality in Figure 12, the text modality in Figure 13, and multimodal attack Figure 14.
Single-modal Attacks (Section 4.1). VLATTACK first perturbs the image modality using the pro- posed BSA and only outputs the adversarial image if the attack successful (Algorithm I lines 1-5). As shown in Figure 12, only attacking the image modality, VLATTACK can generate successful adversarial sample to fool the downstream task. Then, VLATTACK will stop. Otherwise, will perturb the text through BERT-Attack (B&A) and use the clean image as the input, which is illus- trated in Figure (Algorithm 1, lines 6-15). During the text attack, B&A will generate multiple candidates by replacing the synonyms of a word. Since the length of text sentences is very short in the VL datasets, we only replace one word each time. From Figure 13, we can observe that B&A first replaces "kid" with its synonym "child", but this is not an adversarial sample. B&A then moves the next word "small" and uses its synonym "cute" as the perturbation. By querying the black-bor downstream task model, VLLATAACK successes, and the algorithm will stop. Multimodal A Attack
17

---

N,-20
N
small kid small child
a cure kid
enjoying play B&A enjoying play B&A enjoying play with ballons. with ballons, with ballons
Do the clephants have long tusks?
Do the elephants have long tusks?
Figure 13: An adversarial sentence from text at- tack.
Figure 12: An adversarial image from BSA.
1
12
N,=20
N
X
What material is the table B8.
What muterials
Rank
Wh muterial tuble make of
What material the table
What materials the table made of?
What materrlals
is the tablle
3
of
built of x
X
marke of
mane
madcof
T
T1
T1
T2
T2
Figure 14: An adversarial image-text pair from multimodal attack.
(Section 4.2). If the single-mone modal attack fails, VLATTACK moves to the multimodal attack by iter- atively cross- uross-updating image and text perturbations, where image perturbations are added through BSA, and perturbations cess is repeated until an adversarial image-text pair is found (A (Algorithm 1, lines 16-24). Figure 14 shows example. In step 1, VLATTACK fails attack the image modality and outputs perturbed image denoted as r/_ In step 2, VLATTACK also fails to attack the text modality and outputs a list of text perturbations
added according to the semantic similarity. The cross-updating pro-
VLAITACK has use the multimodal attack to generate adversarial samples in step 3. It first ranks the text perturbations in T according to the semantic similarity between the original text and each perturbation. The ranked list is denoted as {T, allocates the iteration number of the image attack to generate the image perturbations iteratively. In Figure 14, this number is 6, which means we run BSA with the budget 6 to generate new image perturbation.
T Then it equally
VLATTACK takes the pair (H,Th) the input to query the black-box downstream model, where T "What material is the table make of?". If this pair is not an adversarial sample, then the pro- posed ICSA will adopt BSA to generate the new image perturbation I2 The new pair (Ig TT) will be checked again. If it is still not adversarial sample, VLATTACK will use the next text pertur- bation T "What materials the table made of?" and the newly generated image perturbation I/2 as the input and repeat the previous steps until finding a successful adversarial sample or using up all K text perturbations in T. VLATTACK employs systematic strategy for adversarial attacks on VL models, sequentially targeting single-modal and multimodal perturbations to achieve successful adversarial attacks.
E.2 Case Study on Diffferent Tasks
We also provide additional qualitative results from Figure 15 to Figure 20 for experiments on all six tasks. For better visualization, display the adversarial and clean samples side by side in single column. By adding pixel and word perturbations, the fidelity of all samples is still preserved, but predictions are dramatically changed. For instance, in the image captioning task of Figure 19, all generated captions show no correlation with the input images. Some texts may even include replacement Unicode characters, such "ffffd"" resulting in incomplete sentence structures.
18

---

Clean
Is the dog looking at the camera? Nes
What kind of annimals on the pillow? Cirr
What are the skiers holding, in their hands? Ski polex
At which course would this Are the flags pointing left be served meal! Main right? left
Adversarial
dog sturing camera? No
What kinds of animalls What are the skiers having which course would this Are the flags pointing left on the pillow??
Beur
their hands? Shis
delivered meall right? right Dimner
Figure 15: Additional quantitative results on visual question answering (VQA).
Clean
Broad BL o
dog jumping close dangerous rocks. Neutrul
dog carrying puppy home. Neutral
There dogs Entailment
The woman wearing dull dress Contrudiction Contradiction
A man is sitting inside.
Adversarial
Broad
dog is jumping near to dangerous rocks. Contradiction
dog is carrying pup home. Contradiction
There dogs Neutral
The girl wearing (text only) Neutral
A man sitting inside. (Image only) Neutral
Figure 16: Additional quantitative results visual entailment (VE).
Clean
At leust one dog is laying down. True
There is least three dogs. Fulse
the right has fewer then five animals.
Adversarial
least one dogs is laying down. Fabse
There least dogy True
the right pie has less then five unimals. False
Figure 17: Additional quantitative results visual reasoning (VR).
19

---

Clean
The first burger, immediately next to the metal knife handle.
a buseball player with number dhree on his uniform.
The half part of sandwich nearest the
a glass orange juice
woman touching
Adversarial
The first burger, immediately next the metal hlade handle.
bascball player with number 3 on his uniform
The half part of the sandwich close to the carrots
glasses orange juice
lady touching the horse
Figure 18: Additional quantitative results on referring expression comprehension (REC).
Clean
a white street sign that reads othello.
bowls train riding on track as people are walking
there motor bikes that group of cows looking
inside of them
parked on the street. at camera by trees
Adversarial
collage of different images of person.
painting of woman a free
picture of a human hend with purzzle
painting of a skull with THE Cluffi Tifffd
painting of group of zombies and jigsaw puzzle
Figure 19: Additional quantitative results on the image captioning task.
Clean
pembroke
bonnet
leonberg
planetarium
coral reef
Adversarial
dingo
how tie Figure 20: Additional quantitative results on the image classification task.
christmas stocking
jigsaw puzzle
20

---

F. Discussion: BSA v.s. BadEncoder
We notice that the idea of our proposed BSA is somehow similar to the BadEncoder [60] method, as they all use the cosine similarity as the optimization target. However, the proposed BSA is different from or better than BadEncoder.
(1) BadEncoder only utilizes the final output feature vectors from the whole encoder, ignoring the outputs from the intermediate layers/blocks. However, BSA calculates fine-grained similar- ity scores. As shown in Eq. 2 of our original paper, we distinguish the outputs from image and Transformer encoders. Such design can modify the low-level vision features and the high-level cross-modal representations. In our setting, we attack fine-tuned models of a wide diversity of down- stream tasks, including but not limited to image classification tasks like Badencoder. The parameters of these task-specific models are fully finetuned on distinct datasets, and the output representations of the encoder significantly change accordingly. Thus, instead of only attacking the output fea- ture from the last layer like BadEncoder, perturbing each intermediate feature representation from each encoder and each block can enhance the attack performance. This statement also verified in Section 5.5, where the ASR score of BSA is higher than only attacking single encoder.
(2) The motivation for adopting the cosine distance is different. In BadEncoder, CLIP cosine similarity as a loss function to calculate distances for positive/negative image text pairs, which is motivated by the pre-training strategy of CLIP. However, we adopt cosine similarity because the fine-grained token representations attend to each other in the inner product space, which is inspired by the mechanism design of the Transformer structure. Therefore, the proposed BSA method greatly differs from BadEncoder in the above two aspects.
G. Limitations
The limitations our work can be summarized from the following two aspects. On the one hand, in our current model design, for the text modality, we directly apply the existing model instead of developing a new one. Therefore, there no performance improvement on tasks that only accept texts input, such text-to-imen to-ima synthesis. On the other hand, our research problem is formu- lated by assuming the pre-trained and downstream models share similar structures. The adversarial transferability between different mre-trained and fine-tuned models is worth exploring, which we left to our future work.
H. Broad Impacts
Our research reveals substantial vulnerabilities in vision-language (VL) pre-trained models, under- lining the importance of adversarial robustness cross pre-trained and fine-tuned models. By exposing these vulnerabilities through the VLATTACK strategy, we offer inspiration for developing more ro- bust models. Furthermore, our findings underscore the ethical considerations of using VL models in real-world applications, especially those dealing with sensitive information and big data. Overall, our work emphasizes the necessity of balancing performance and robustness in VL models, with implications extending across computer vision, natural language processing, and broader artificial intelligence applications.
21


Reviews:
[{"review_id": "zOBVOwC1gV", "reviewer": "Reviewer_pd9a", "summary": "The authors propose a substitute black-box attack strategy called VLAttack to generate adversarial examples by perturbing both images and texts on pre-trained models and then transferring them to finetuned models.  At the image-modal level, they introduce a block-wise similarity attack (BSA) strategy to disrupt universal representations in the pre-trained model. At the multimodal level, they design an iterative cross-search attack (ICSA) to update adversarial image-text paris.", "questions": "Please refer to the weakness section.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. This paper focuses on an important question concerning the prevalent pretraining and finetuning paradigm with respect to adversarial robustness.\n\n2. The core idea of the proposed attack is clearly presented. \n\n3. The proposed VLAttack could potentially serve as a valuable baseline for examining the robustness of multimodal models.", "weaknesses": "Issues that affect my rating.\n\n1. Lack of technique contributions. The question asked by the authors is quite valuable. However, the proposed attack strategy, VLAttack, seems trivial.  Regarding the text modality, VLAttack directly employs the existing BertAttack[20]. For the image attack,  the classical PGD attack with iteration $N_{s}=20$ is adapted. The idea of the proposed BSA loss Eq.2 is straightforward and similar to that of BadEncoder [a] (see BadEncoder's Eq.2~5), except BadEncoder focuses on the backdoor attack. The cross-modality attack should be the most important part, where the two modality attacks boost each other is excepted. However, the proposed \"ICSA\" seems directly concatenate the image and text attacks. Did the author have attempted to jointly optimize these two attacks? \n\n[a] Jia, J., Liu, Y., & Gong, N. Z. (2022, May). Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning. In 2022 IEEE Symposium on Security and Privacy (SP) (pp. 2043-2059). IEEE.\n\n2. Insufficient evaluation on popular multimodal models (CLIP, BLIP): I recommend that the authors extend their evaluation to include more prevalent multimodal models, such as CLIP or BLIP, as done in Co-attack[15] and BadEncoder[a]. This would enable readers and subsequent works to make more meaningful comparisons.\n\n3. Ambiguity in the fine-tuned model settings: Considering the significant performance difference between VLAttack and baselines in Table 1, it is unclear how the target models are fine-tuned. Are the pre-trained model parameters fixed while training a task-specific head, or are all parameters fine-tuned throughout the pre-trained model? Providing clarity on this aspect is crucial for evaluating the impact of the proposed attack. Additional details should be included to enhance understanding.\n\n4. Absence of essential ablation studies on BSA and ICSA: The effectiveness of BSA remains unclear, and a comparison with an attack on the pre-trained model's original loss could provide valuable insights. To showcase the efficacy of ICSA, a straightforward baseline could involve setting BSA's attack iteration $N_{s}=40$, without incorporating BERT-attack. The authors should consider conducting these ablation studies to better demonstrate their contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable review. Our responses are addressed below.\\\n`>>>Q1` ***Lack of technique contributions***\\\n`>>>A`: The multimodal attack is a new research topic, and only one work Co-Attack has been proposed recently. However, its performance is even worse than state-of-the-art image attack approaches. To solve these issues, we designed a new multimodal attack approach named VLAttack, which can jointly optimize adversaries for both images and text. It first seeks potential adversarial candidates at the single-modality level. If unsuccessful, the outputs from single-modality attacks will be treated as initialization at the multi-modal attack level.\n\nIn this work, we have two technical contributions. One is the new loss for the image attack, and the other is a novel cross-modality search strategy for the multimodal attack. In the single-modality attack, we propose a new block-wise attack loss to disrupt the features in the image encoder and transformer encoder of the pre-trained model. We directly apply BERT-Attack for the text modality due to the limited length of input text, as described in lines 189-193. Finally, we propose a novel cross-search strategy on top of the outputs from single-modality attacks, which iteratively perturbs image and text modalities further. Thus, we argue that the proposed VLAttack is simple, effective, and novel for the multimodal attack topic. \n\n`>>>Q2` ***Insufficient evaluation on popular multimodal models (CLIP, BLIP)***\\\n`>>>A`: Thanks for the suggestion. We further extend our experiments on the CLIP model. The experiment is developed on the image classification task on the SVHN dataset [a]. Concretely, we use the image encoder of CLIP as the pre-trained model $F$ and then fine-tune $F$ on the SVHN dataset after adding a classification head. For the choices of image encoder of CLIP, we adopt ViT-B/16 and ResNet-50, denoted by CLIP-ViT-B/16 and CLIP-RN50. We test the attack performance using 1,000 correctly predicted samples. \nAll results are illustrated in the following table. Since the task only accepts images as input, we compare our BSA with other baselines. As shown in the table, our proposed BSA still maintains the best ASR using different image encoder structures, clearly demonstrating its effectiveness. \n|Method| DR | SSP | FDA | BSA(VLAttack) | \n|----| ---- | ---- |---- |---- |\n|CLIP-ViT-B/16|7.90 | 4.60 |6.20 |__20.10__|\n|CLIP-RN50|70.70 | 76.20 |74.20 |__77.40__|\n\n[a] Goodfellow I J, et al. Multi-digit number recognition from street view imagery using deep convolutional neural networks[J]. arXiv preprint, 2013.\n\n`>>>Q3`  ***Ambiguity in the fine-tuned model settings***\\\n`>>>A`: Thanks for your valuable comments. All task models are fully fine-tuned. In other words, all the model parameters are updated in the fine-tuning stage, but they are not accessible during the attacking stage. We treated them as black boxes. Thus, we do not have any knowledge about the fine-tuned model parameters during the attack process. We will add these details in the final version.\n\n`>>>Q4`:  ***Absence of essential ablation studies on BSA and ICSA***\\\n`>>>A`: Thanks for your comments. \n\n __*(1) Ablation study of BSA*__\\\nTo validate the effectiveness of BSA, we conduct an experiment to compare the original pre-trained loss with the proposed VLAttack on the ViLT model using the VQA task as an example. Since different pre-training tasks have their task-specific loss functions, in this experiment, we validate the masked language modeling (MLM) loss as the attacking objective for the VQA task.\n\nGiven an image-text pair, the MLM loss randomly masks 15% of words in the text, and the training target is to recover these masked tokens. The attack objective is optimized through cross-entropy. In our implementation, we replace Eq. (2) in our paper with the MLM loss and reverse the target by adding perturbation on the image to make the pre-trained model incapable of recovering the masked tokes. When attacking, we optimize the above objective for 40 iterations like BSA. The results are illustrated in the following table. \n\n|Method|MLM | BSA | VLAttack | \n|----| ---- | ---- |---- |\n|ASR|25.40 | 65.20 |__78.05__|\n\nWe can find that the performance using the original pre-trained MLM loss is far inferior to the proposed BSA and VLAttack methods. The reason is that in the attacking stage, the downstream tasks are fine-tuned with task-specific datasets, which also update the relations among tokens learned in the pertaining stage accordingly. However, if we still use the attack strategy learned in the pre-training stage, the change of token relations will make it fail to generate adversarial samples. Thus, the original MLM loss performs worse than our proposed VLAttack, even BSA.\n\n__*(2) Ablation study of ICSA*__\\\nThanks for pointing this out. We did have such baselines that involve setting BSA's attack iteration $N_s=40$, without incorporating BERT-attack, as shown in Figure 5 of Section 5.4. The experimental results show that our method equipped with ICSA (VLAttack) significantly outperforms the BSA method under the same iterations.\n\n"}, {"review_id": "x6gp3lDm6L", "reviewer": "Reviewer_3TAC", "summary": "This paper focuses on adversarially attacking the multimodal finetuned models without getting access to the finetuned weights. By utilizing the activations and the parameters of the open-accessed pretrained model, this work proposes a method called VLATTACK for creating the adversarial attack samples for downstream tasks. The creating method first attempts to learn image perturbations via block-wise similarity attack (BSA) strategy. If not succeed, the text attack using BertAttack will be facilitated. If the two single-modal attack both fail, VLATTACK will use cross-search attack (ICSA) method to construct multimodal disrupted adversarial samples. Experimental results show the effectiveness of VLATTACK on various cross-modal (image-text) tasks and model structures. Some detailed analysis on the adversarial samples are also conducted.\n\n", "questions": "How would the performance of VLATTACK be affected under different model size (like OFA-base vs OFA-large) and image resolution ($224$ vs $480$)? I will be very happy if some insights can be provided.", "limitations": "The authors have proposed some limitations in the appendix, including the text attacking methods, downstream task scope and explored model architectures, which I think is reasonable and leave space for future research.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The experimental results are obtained based on various downstream tasks and pretrained models (in different architectures), which demonstrate the generalization ability of VLATTACK.\n2. The choice of pretrained models are open-sourced, making the results more reproducible.\n3. The performance of adversarial attack is good, compared with other typical and previous attacking methods.\n4. The case study and ablation study are both well-conducted, providing insights on how the VLATTACK can achieve better attacking performance.", "weaknesses": "Generally speaking, I think this paper is a good work on the topic of adversarial attacking. If the following issues are addressed, I think it will be much better:\n1. Experiments or analysis on adversarial multimodal datasets: Some adversarially constructed datasets are proposed in previous works, based on the datasets used in this work, like Adversarial VQA. Does the created adversarial samples share some similarity with these datasets? I think some discussion or analysis on this research question can be performed and will be very welcomed.\n2. Evaluation soundness: For VQA and captioning, an extra human evaluation on whether the predictions under attack are actually wrong will be much recommended, considering the automatic metrics may make false-positives.\n3. Experiments on out-of-domain datasets: For OFA, we know the pretraining corpus include the downstream datasets (VQA, RefCOCO and MSCOCO), which may result in the adversarial samples easier to be obtained using the pretrained model. It would be very good if downstream datasets which not appear in pretraining OFA can be considered in experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable review. Our responses are addressed below.\\\n`>>> W1` ***Experiments or analysis on adversarial multimodal dataset***\\\n`>>> A`: Some methods [a,b] focus on constructing adversarial datasets, but they significantly diverge from the nature of our research work. Specifically, these adversarial datasets usually consist of new samples generated through human annotation [d] (e.g., writing questions until models give wrong predictions) or machine synthesis [a] (e.g., removing objects from images). They do not have any perturbation budget constraints and are mainly used to test the robustness of multi-modal models. \n\nHowever, in our work, we aim to add noise to the clean data under strict perturbation constraints. The generated samples maintain similar appearances to the clean ones but can cause incorrect predictions in the downstream task model. This not only tests the robustness of the model but, more importantly, exposes potential security concerns on these pre-trained models.\n\n[a] Agarwal, V., et.al. (2020). Towards causal vqa: Revealing and reducing spurious correlations by invariant and covariant semantic editing. CVPR 2020.\n\n[b] Li, L., et.al. (2021). Adversarial vqa: A new benchmark for evaluating the robustness of vqa models. ICCV 2021.\n\n`>>> W2` ***Evaluation soundness***\\\n`>>> A`: Thanks for your valuable comments and constructive suggestion. Adding a human evaluation experiment significantly increases the reliability of the proposed attack method. We conducted the human evaluation experiments using the results output from the ViLT model on the VQAv2 dataset using Amazon Mechanical Turk (MTurk). The baseline that we choose is SSP since it outperforms others in our experiments, as shown in Table 1 in our paper.\n\nSpecifically, we randomly sampled 650 examples and stored the generated answers after the attack. To validate the performance of both approaches, we send the original image-question pairs and the corresponding generated answer to the MTurk system. We provide three choice candidates to workers -- `'' Definitely Correct '', '' Not Sure '', and '' Definitely Wrong ''. A successful attack means that the worker will label ``Definitely Wrong'' to the pair. To make the annotations more accurate and reliable, each pair is annotated by three workers, and we report the majority choice as the final human evaluation result. The statistics of human evaluation results are shown in the following table. We can observe that the proposed VLAttack still significantly outperforms the strongest baseline SSP.  \n\n|Method| SSP| VLAttack| \n|----| ---- | ---- |\n|Definitely Correct $\\downarrow$|413 | __307__ |\nNot Sure $\\uparrow$|__58__| 56 |\n|Definitely Wrong $\\uparrow$|179 | __287__ |\n\n`>>> W3` ***Experiments on out-of-domain datasets***\\\n`>>> A`: Thanks for the advice. We do have such experiments, which are shown in Table 1 of our original paper. We conduct experiments on the SNLI-VE dataset for the visual entailment (VE) task, which is not involved in the pre-training process. From the results on the SNLI-VE dataset, we can observe that VLAttack achieves the best performance. Thus, even using the out-of-domain datasets, the proposed VLAttack is still effective.\n\n`>>> Q1` ***How would the performance of VLATTACK be affected under different model size\n(like OFA-base vs OFA-large) and image resolution (224 vs 480)?***\\\n`>>> A`: Thanks for the constructive suggestions. These suggested experiments are interesting and will significantly improve the quality of our paper. Due to the limited time of the rebuttal stage, we only test the influence of model size and will put the results regarding the image resolution experiment in the final version.\n\nWe evaluate different methods on the OFA-large model through the VQA task. The results are shown in the following table. We can observe that the proposed VLAttack achieves the best performance, but the results slightly decrease compared to those on OFA-base in Table 1 of our paper, which supports the claim that the model can be more robust with a larger parameter size. \n\n|Method| SSP|Co-Attack|VLAttack| \n|----| ---- | ---- | ---- |\n|ASR|33.82 | 39.41 |__75.44__|\n\n\n"}, {"review_id": "bnx4N7mYjH", "reviewer": "Reviewer_NrLG", "summary": "The paper presents VLAttack, which is a method for perturbing multimodal examples such that a multimodal model would get them wrong. VLAttack does not assume access to fine-tuned models but does assume access to foundation models that are used to create these downstream models. The authors argue that this level of access is reasonable because there are several real-world examples where the foundation model is available but downstream models are not. VLAttack makes models fail significantly more than similar adversarial attack algorithms across several datasets and tasks. ", "questions": "I tried to frame everything in the weaknesses section as questions.", "limitations": "Limitations seem adequately addressed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "As far as I am aware, the algorithm is novel (especially the visual perturbation part). And the authors showcase that it outperforms similar algorithms in the sense that it does not assume access to the fine-tuned model (which could be unreasonable) and it makes models get more errors. ", "weaknesses": "Major issues:\n\nIt is great that VLAttack makes the models get higher error rates than other approaches, but crucially it is only a good result if VLAttack does not actually change the ground truth labels. It seems that you do not validate this with, e.g. crowd workers. Unless I missed it, the only validation that you do is presenting case studies of a few individual examples. Did you do a systematic validation?\n\nLess important (but still issues):\n\nMore motivation would be nice. What scenarios are you envisioning for why VLAttack raises a safety concern? Self-driving car failures, etc.?\n\nCan you make these adversarial perturbations useful? For example, can you train a model on them to get better performance?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable review. Our responses are addressed below.\\\n`>>>Q1` ***Systematic validation***\\\n`>>>A`:  Adding a human evaluation experiment significantly increases the reliability of the proposed attack method. We conducted the human evaluation experiments using the results output from the ViLT model on the VQAv2 dataset using Amazon Mechanical Turk (MTurk). The baseline that we choose is SSP since it outperforms others in our experiments, as shown in Table 1 in our paper.\n\nSpecifically, we randomly sampled 650 examples and stored the generated answers after the attack. To validate the performance of both approaches, we send the original image-question pairs and the corresponding generated answer to the MTurk system. We provide three choice candidates to workers -- '' Definitely Wrong '', '' Not Sure '', and '' Definitely Correct ''. A successful attack means that the worker will label '' Definitely Wrong '' to the pair. To make the annotations more accurate and reliable, each pair is annotated by three workers, and we report the majority choice as the final human evaluation result. The statistics of human evaluation results are shown in the following table. We can observe that the proposed VLAttack still significantly outperforms the strongest baseline SSP.  \n\n|Method| SSP|VLAttack|\n |----| ---- | ---- |\n|Definitely Correct $\\downarrow$|413 | __307__ |\n|Not Sure $\\uparrow$|__58__ | 56 |\n|DefinitelyWrong $\\uparrow$|179 | __287__ |\n\n`>>>Q2` ***More motivation would be nice.Can you make these adversarial perturbations useful? For example, can you train a model on them to get better performance?***\\\n`>>>A`:  As mentioned in lines 21-27 of our paper, the `''pre-train $\\&$ fine-tune'' paradigm has become a prevalent trend when training vision language models. The pre-trained VL models can be efficiently deployed in different scenarios and tasks after fine-tuning, such as virtual assistants [a] and robotic control [b]. Against this background, a key question raised in our paper is whether these publicly available pre-trained models are robust, which is also the primary motivation of our work. \n\nAs a pioneering research, we believe it can further benefit various research fields, such as adversarial fine-tuning for more robust vision-language models. In fact, there has already been adversarial fine-tuning work on language models [c], but research on vision-language models remains unexplored. We believe that the proposed VLAttack can serve as significant guidance for future studies in this field.\n\n[a] Tu, T., et.al. Learning better visual dialog agents with pretrained visual-linguistic representation.CVPR 2021\\\n[b] Yecheng Jason Ma., et.al.. LIV: Language-Image Representations and Rewards for Robotic Control. ICML 2023 \\\n[c] Dong, X., et al. How should pre-trained language models be fine-tuned towards adversarial robustness?. NIPS 2021\n"}, {"review_id": "2ozSmuCW6F", "reviewer": "Reviewer_AgmN", "summary": "In this paper, the authors explore the adversarial vulnerability in visual language models. Specifically, a block-wise similarity attack is proposed to generate adversarial image examples, and the BERT-attack method is used for generating the adversarial text examples. The image and text pairs are perturbed by an iterative method with cross-search.", "questions": "It is better to provide the analysis of the semantic similarity and the perturbation degree, as mentioned in Weaknesses 1.\n\nCompared to other methods, could you provide a thorough analysis of the costs and average query times of the proposed framework?", "limitations": "I did not find a potential negative societal impact.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper explores the black-box attack for vision language models, which is novel.\n\n2. The proposed method can generate perturbed inputs to decrease the performance of the fine-tuned models on multi-modal and uni-modal datasets effectively. ", "weaknesses": "1. One important criterion for a successful adversarial attack is that the perturbed image cannot be distinguished by humans, and the semantic meaning of the text adversarial examples should be the same as the original one. However, there needs to be more analysis of the semantic similarity between the generated adversarial examples and the original text. The perturbation of the images should be measured as well. Most importantly, the perturbation of the generated image-text pairs should be analyzed.\n\n2. By applying the iterative way for generating adversarial image and text pairs, the attackers need to query the large vision language model more times. The time costs and the average query numbers for a successful attack should be provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable review. Our responses are addressed below.\\\n`>>> Q1` ***It is better to provide the analysis of the semantic similarity and the perturbation degree, as mentioned in Weaknesses 1***\\\n`>>> A`: Thanks for your constructive suggestions. We compare the semantic similarity score for the text modality. Following previous work [20][25], the score is obtained by first using the universal sentence encoder[44] to embed the output and original sentences into vectors. Then we compute the cosine similarity between the two vectors. We compute the average $L_2$-distance as the perturbation degree for the image modality between all output and original images in the 8-bit color space. The experiment is developed on the  ViLT model through the VQAv2 dataset, and the results are illustrated in the following table.  \n\n|Modality| Metric | SSP (image-only)| Co-Attack | VLAttack| \n|----| ---- | ---- |---- |---- |\n|Text|Semantic Similarity | 1.00 |0.97 |0.99|\n|Image|$L_2$ Distance | 11.51 |11.38 |11.23|\n\nWe can observe that SSP has the highest semantic similarity score since it only perturbs the image modality. Due to the limited length of text, Co-Attack and the proposed VLAttack also achieve higher semantic similarity. However, the proposed VLAttack performs better than the baseline Co-Attack. \nThere are two reasons: (1) In our model design, we first perturb the image modality and then the text modality. If the image attack succeeds, we no longer need to perturb the text. (2) Even if the single-modality attack fails, during the multimodal attack, we will first use the text perturbation candidates with the highest semantic similarity to generate adversarial samples, which also helps VLAttack to maintain semantic similarity.\n\nOur method also shows advantages in terms of $L_{2}$ distance on image perturbations, where VLAttack outperforms both baselines. \nThis advantage arises because VLAttack adopts clean images as input during the text attack stage. Consequently, the adversarial samples produced at this stage do not perturb the image, leading to a reduced average $L_2$ distance.\n\n`>>> Q2` ***Compared to other methods, could you provide a thorough analysis of the costs and average query times of the proposed framework?***\\\n`>>> A`: Thanks for your comments.  We detail the analysis of query times and computation costs as follows. \n\n__*(1) Query Times*__\nThe proposed VLAattack conducts queries in three phases: (1) During the image attack phase, a query is made after optimizing with BSA for $N_{s}=20$ times (Algorithm 1, line 5). (2) During the text attack phase, BERT-Attack is used for querying (Algorithm 1, line 11). (3) In the ICSA stage, queries are made for every $N_k$ step (Algorithm 1, line 19). \nAmong all baselines evaluated in Table 1 of our paper, only text attack methods BERT-Attack (B\\&A) and R\\&R need queries. Thus, we compare VLAttack with BERT-Attack as it achieves better performance. \n\nThe average number of queries on BERT-Attack and VLAttack are __8.61__ and __4.99__ *times/sample*, respectively. \nWe can find that even when the ICSA stage involves additional queries, VLAttack has a lower average number of queries than BERT-Attack. This is attributed to a portion of the samples that can be directly obtained in the image-attack stage using BSA, and they only query once, thus avoiding queries in the text-attack stage and the ICSA stage. \n\n__*(2) Computation Costs*__\nWe also analyze the average computation costs on the largest pre-trained model OFA. Specifically, we report the average speed in seconds for computing an adversarial sample on the VQAv2 dataset. The results are shown in the following table: \n\n|Method| SSP| BERT-Attack  | Co-Attack | VLAttack |  \n|----| ---- | ---- |---- |---- |\n|Speed (s)|7.02 | 6.07 |18.46 |9.73| \n\nWe can observe that attacking a single modality (i.e., SSP for image and Bert-Attack for text) is faster than perturbing two modalities (i.e., Co-Attack and VLAttack), which is reasonable. Besides, the speed of VLAttack is noticeably faster than that of Co-Attack. \nThis is because Co-Attack requires adding both image and text perturbations for every sample, but our method can achieve a successful attack by adding perturbations to a single modality only.\nIn sum, with the outstanding attack performance, the acceptable generation speed further demonstrates the effectiveness of our proposed VLAttack.\n\n"}, {"review_id": "vVNmoogajF", "reviewer": "Reviewer_zqVz", "summary": "The authors present a new adversarial attack framework, VLATTACK, to evaluate the robustness of large vision-language models (VLMs). The paper introduces a two-step adversarial attack approach. The first step involves attacking each modality (image and text) independently, and the second step employs a cross-search attack strategy to iteratively query the fine-tuned model for perturbation update. The paper conducts experiments on multiple pre-trained models that are fine-tuned for downstream vision tasks.\n\n", "questions": "1. Fig. 7 second column demonstrates that the model comprehension shifts from *A man in a purple shirt and **jeans*** to *A man in a purple shirt and **denim***. However, denim is the material pattern for the jeans shown in the image. The model prediction is still consistent with the image. It will be beneficial if the authors state clearly what this example stands for.\n\n2. Given the setting of this paper that we have (1) publicly available pre-trained models and (2) the clean dataset for future perturbation, I wonder if it is possible to first fine-tune the model on the clean dataset for the downstream task and then conduct the transfer-based attack? It will be beneficial for this paper to argue the infeasibility of such approaches.\n\n3. To summarize, with a fair paper presentation but several concerns stated in the weakness/question sections, I will rate it as a borderline accept at this stage. However, I look forward to the authors' response and I will consider revising the rating based on the soundness of the responses.\n", "limitations": "Yes, the authors have stated the potential limitations of the work in the supplementary materials.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written with a clear presentation that demonstrates the proposed framework.\n\n2. The VLATTACK, which is a two-step adversarial attack framework, involves both white-box gradient attack and black-box query attack. This idea is interesting with practical implementation.\n\n3. The evaluation takes account of multiple types of pre-trained models, and the BSA part of the attack overperforms SOTA image-space adversarial baselines.", "weaknesses": "1. The black-box setting is questionable. Although the adversary only has black-box access to the downstream fine-tuned model, the first stage of attack has white-box access to widely-used foundations like ViLT. There is a high chance that the victim model shares mutual information (e.g., has knowledge of the same vision-language dataset / has the same model architecture) with the white-box pre-trained model. It will be beneficial if the authors elaborate more on the source of transferability.\n2. Section 4.2 (ICSA) claims to be under the black-box setting. However, Lines 228-229 describe the process to update the perturbation by optimizing the attack loss of Eq. (2). Authors should write clearly how the gradient is obtained, and what model is white-box during this step.\n3. Section 4.1 Text-Attack part uses clean image $\\mathbf{I}$ instead of generated adversary $\\mathbf{I'}$ for text optimization. Authors claim that this is to avoid unnecessary modification. However, using $\\mathbf{I'}$ can possibly ease the difficulty of textual search and increase the success rate of adversarial textual optimization. Moreover, the authors should argue clearly what the unnecessary modification stands for.\n4. The success of the attack will require a sufficient number of queries to generate effective adversaries. It will be helpful if authors evaluate the query cost (e.g., time of inference/API call), and discuss attack effectiveness by the number of queries.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable review. \\\n`>>> W1` ***Elaborate more on the source of transferability***\\\n`>>> A`:  We agree that the pre-trained and fine-tuned models share almost the same architecture in the current setting. Such mutual information should help to improve the attack success rate. \n\nAs suggested by the reviewer, we further design an experiment of __*transfer attacks on different structures*__. Based on the VQA task, we first generate adversarial samples on the pretrained ViLT and use them to attack the fine-tuned ALBEF model. The two models have different architectures. The results are shown in the table. \n\n|Method| SSP  | Co-Attack | VLAttack | \n |-| - | - |-|\n|ASR|12.78  | 13.04 |__15.28__ |\n\nAlthough VLAttack is not explicitly designed for this scenario, it achieves the best ASR. Also, the relatively low ASR suggests potential space to explore this setting in the future.\n\n`>>> W2` ***How is the gradient obtained?***\\\n `>>> A`: Thanks for the advice. We first feed the output from the previous step \\$\\boldsymbol{I}_{k}^{'}\\$ to the pre-trained model $F$. Then we compute the BSA loss by replacing \\$(\\boldsymbol{I}_k^{'},\\boldsymbol{T}_k^{'})\\$ with \\$(\\boldsymbol{I}^{'},\\boldsymbol{T})\\$ in Eq.(2) of our paper. The gradients are then derived on the $F$ and will be clipped to produce the new perturbed image of the $k+1$ step. So the pre-trained model $F$ is the only white-box model during this process.  \n\n`>>> W3` ***Using $\\boldsymbol{I}^{'}$ in the text-attack stage may increase ASR, and what does the '' unnecessary modification '' mean?***\\\n`>>> A`: According to our experiment, using $\\boldsymbol{I}^{'}$ in the text-attack stage indeed increases the ASR, which rises from 78.05 to 78.54. But there is a trade-off between the ASR rate and the perturbation degree. \n\nUsing the clean image $\\boldsymbol{I}$ as the input of the text-attack makes the perturbation degree of the images to be zero in this stage. When using the perturbed image \\$\\boldsymbol{I}^{'}\\$ as the input, if the text attack fails, then \\$\\boldsymbol{I}^{'}\\$ will be fed into the multimodal attack, which does not affect the perturbation degree of images. However, if the text attack can succeed even with the clean image $\\boldsymbol{I}$, in such a case, using \\$\\boldsymbol{I}^{'}\\$ will increase the perturbation degree. \nSpecifically, we compute the average $L_2$-distance between all output and original images in the 8-bit color space. The average $L_2$-distance of using \\$\\boldsymbol{I}^{'}\\$  and $\\boldsymbol{I}$ in the text-attack stage are 11.69 and 11.23, respectively. So the unnecessary modification means the increase of the perturbation degree of images.\n\n`>>> W4` ***The query cost***\\\n`>>> A`: Thanks for the advice. \n\n__*(1) Query Times*__\nAmong all baselines evaluated in Table 1 of our paper, only text attack methods BERT-Attack (B\\&A) and R\\&R do queries. So we compare VLAttack with BERT-Attack as it achieves better ASR. \n\nThe average number of queries on BERT-Attack and VLAttack are 8.61 and 4.99 *times/sample*, respectively. \nEven when the ICSA stage involves extra queries, VLAttack has a lower query number than BERT-Attack. This is because some \n samples that can be directly obtained in the image-attack stage using BSA, and they only query once, thus avoiding queries in the text-attack and ICSA stages. \n\n__*(2) Computation Costs*__\nWe also analyze the computation cost on the largest pre-trained model OFA. Specifically, we report the average speed in seconds for computing an adversarial sample on the VQAv2 dataset. The results are shown in the table below: \n\n|Method| SSP | BERT-Attack | Co-Attack | VLAttack |\n |-|-|-|-|-|\n|Speed (s)|7.02 | 6.07 |18.46 |9.73|\n\nWe can find that attacking a single modality (i.e., SSP and BERT-Attack) is faster than attacking two modalities (i.e., Co-Attack and VLAttack). Also, the speed of VLAttack is noticeably faster than that of Co-Attack. \nThis is because Co-Attack requires adding both image and text perturbations for every sample, but VLAttack can achieve a successful attack by adding perturbations to a single modality only.\nIn sum, with the outstanding attack performance, the acceptable generation speed further demonstrates the effectiveness of our proposed VLAttack.\n\n`>>> Q1` ***Explanation for Fig.7 second column***\\\n`>>> A`: In referring expression comprehension task, a bounding box prediction is considered correct only if the Intersection Over Union (IOU) with the label is greater than 0.5.  Based on this metric, the bounding box prediction in Fig.7 of our paper is incorrect after the perturbation. Actually, this predicted bounding box doesn't encompass the person's head and unnecessarily includes the space above the head. \n\nBesides, substituting ''jeans'' with ''denim'' is reasonable, as it doesn't alter the original meaning of the sentence. In sum, this example stands for the claim that the VLAttack can alter the correctness of predictions by introducing minor perturbations to images and texts.\n\n`>>> Q2` ***Fine-tune the model on the perturbation dataset, and then conduct the transfer attack***\\\n`>>> A`: Thanks. We conduct the experiment on the ViLT model through VQA dataset. We first fine-tune the pre-trained model on 5K (size of the perturbation dataset) samples and then use the fine-tuned model to produce adversarial samples. We adopt the same fine-tuning process as in ViLT[1], and the fine-tuning loss is a categorical cross-entropy function. When attacking, we enlarge the loss and optimize for 40 iterations like BSA. \nResults are shown in the table below. The new method is named Surrogate Attack (SA).\n\n|Method| SA | BSA | VLAttack | \n |-| - | - |-|\n|ASR|24.39 | 65.20 |__78.05__ |\n\nIn the table, SA performs poorly compared to BSA and VLAttack. This is because large VL models struggle to train well on limited data samples. Even when provided with clean samples, these surrogate models hardly make accurate predictions, leading to inferior ASR results. \n\n\n \n\n"}]

Author rebuttal:

Meta-review: The paper presents a preliminary exploration of multimodal model robustness. The experimental results span various downstream tasks and pretrained models, showcasing VLATTACK's generalization ability. The effectiveness of the adversarial attack in comparison to other methods is noteworthy. Both the case study and ablation study are well-executed, offering insights into how VLATTACK achieves superior attack performance.  Although the contribution might be seen as somewhat modest, the paper deserves a weak acceptance due to its demonstrated novel aspects and improved experimental validation.
Decision: Accept (poster)