Ask, Attend, Attack: An Effective Decision-Based Black-Box Targeted Attack for Image-to-Text Models
Authors: Qingyuan Zeng
Zhenzhong Wang
Yiu-ming Cheung
Min Jiang
Venue: NeurIPS 2024 poster (2024)
Keywords: Black-box adversarial attack
Image-to-text model
Attention
Differential evolution
Primary area: safety_in_machine_learning

Abstract: While image-to-text models have demonstrated significant advancements in various vision-language tasks, they remain susceptible to adversarial attacks. Existing white-box attacks on image-to-text models require access to the architecture, gradients, and parameters of the target model, resulting in low practicality. Although the recently proposed gray-box attacks have improved practicality, they suffer from semantic loss during the training process, which limits their targeted attack performance. To advance adversarial attacks of image-to-text models, this paper focuses on a challenging scenario: decision-based black-box targeted attacks where the attackers only have access to the final output text and aim to perform targeted attacks. Specifically, we formulate the decision-based black-box targeted attack as a large-scale optimization problem. To efficiently solve the optimization problem, a three-stage process \textit{Ask, Attend, Attack}, called \textit{AAA}, is proposed to coordinate with the solver. \textit{Ask} guides attackers to create target texts that satisfy the specific semantics. \textit{Attend} identifies the crucial regions of the image for attacking, thus reducing the search space for the subsequent \textit{Attack}. \textit{Attack} uses an evolutionary algorithm to attack the crucial regions, where the attacks are semantically related to the target texts of \textit{Ask}, thus achieving targeted attacks without semantic loss. Experimental results on transformer-based and CNN+RNN-based image-to-text models confirmed the effectiveness of our proposed \textit{AAA}.

Ask, Attend, Attack: An Effective Decision-Based Black-Box Targeted Attack for Image-to-Text Models
Qingyuan Zeng', Zhenzhong Wang?, Yiu-ming Cheung?, and Min Jiang" "
Institute of Artificial Intelligence, Xiamen University
Department of Computing, The Hong Kong Polytechnic University Department Computer Science, Hong Kong Baptist University *School of Informatics, Xiamen University
Abstract
While image-to-text models have demonstrated significant advancements in various vision-language tasks, they remain susceptible to adversarial attacks. Existing white-box attacks on image-to-text models require access the architecture, gradi- ents, and parameters of the target model, resulting in low practicality. Although the recently proposed gray-box attacks have improved practicality, they suffer from semantic loss during the training process, which limits their targeted attack performance. To advance adversarial attacks of image-to-tes models, this paper focuses on challenging scenario: decision-bar -based black-box targeted attacks where the attackers only have access the final output text and aim to perform targeted attacks. Specifically, we formulate the decision-based black-box targeted attack a large-scale optimization problem. To efficiently solve the optimization problem, a three-stage process Ask, Atlend, Attack, called AAA, is proposed to coordinate with the solver. Ask guides attackers to create target texts that satisfy the specific semantics. Attend identifies the crucial regions of the image for attacking, thus reducing the search space for the subsequent Attack. Attack uses an evolutionary algorithm to attack the crucial regions, where the attacks are semantically related to the target texts of Ask, thus achieving targeted attacks without semantic loss. Experimental results transformer-basee and CNN+RNN- based image-to-text models confirmed the effectiveness our proposed AAA.
Introduction
Image-to-text models, referring generating descriptive and accurate textual descriptions of images, have received increasing attention in various applications, including image-captioning [1, 2]. visual- question-answerin [3, 4], and image-retrieval [5, 6]. Despite the remarkable progress, they are vulnerable to deliberate attacks, giving rise to concerns about the reliability and trustworthiness of these models in real-world scenarios. For example, one may mislead models output harmful content such as political slogans and hate speech by making imperceptible perturbations to images [7, 9, 10, 11].
To gain insight into the reliability and trustworthiness of the tnage-to-text models, a series of adversarial attack methods have been proposed to poison the outputted textual descriptions of given
The corresponding author: Min Jiang, minjiang @ edm..ddu.n Min Jiang and Qingyuan Zeng are with the Department of Artificial Intelligence, Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, School of Informatics Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

clean image
pertuitation
adverparbl
existing work
adversorial
barget model
((b)
adverurtial perturbation
ours
target model
cleon Image
adversarial Image
Imagee tagge-t-ttod
"a boy and girl are looking cameri the mirrrr
target
test-tp-truge
output test "there are two boys brushing their teeth bothroom"
"a boy and a girl are looking carera the mirror"
target
oupput toxt camera
looking
Lorget Imagge
Figure I: The semantic loss problem is existing in existing gray-box targeted attack methods.
images [7. 12. 13, 10]. Specifically, based on the attacker's level of access to information about the target model, they can be divided into three categories: white-box attacks [7, 14, 12), gray-box attacks [13, 10], and black-box attacks [15, 16, 17, 18, 19]. The white-box attacks can obtain target models' information including the entire architecture, parameters, gradients of both the image encoder and text decoder, and probability of each word the output text. Gray-box attacks can only access the architecture, parameters, and gradients of the image encoder, while black-box attacks cannot access any internal information of the target model, but only the output text of the model. Furthermore, black-box attacks can be divided into score-based and decision-bai based attacks. Score-based black-box attacks can access the probability of each word of the output text [16]. while decision- based black- box attacks only access the output text [15, 20, 21J. Because less information about the target models is provided, decision- based black-box attacks are more challenging than other categories [21]. Additionally, these attack methods can be categorized based whether the attacker is able to specify the incorrect output text, dividing them into two types: targeted and untargeted attacks [22, 13].
Although numerous adversarial attack methods for image-to-text models have been proposed, to our best knowledge, the study on black-box attacks is under-explored, especially decision- based black-box targeted attacks. This kind of attack more challenging due the following reasons. Firstly, less information on the target model can be accessed. Specifically, only the output text instead of gradients, architectures, parameters, and the probability of each word in the output text is available. Secondly, the attackers not only cause the target model to output incorrect text, but also outputs the specified target text. Existing attacks easily suffer from the loss of semantics, resulting in the inability to effectively output the specified target text. Figure (a) show that transfer+query [10] fabricates one target text to poison the target image-to-tex model, leading to this model outputting an incorrect text. However, the output text could mismatch the original semantics of the target text, as the target image-to-text model may focus on secondary information while ignoring the crucial semantics of the target text behind the target image, resulting in semantic loss. More examples are in Appendix B.
To narrow the research gap, we propose a decision- image-to-text models. In our work, only the output text of the target model can be accessed, which is closer to the real-world cases [15]. Additionally, Figure 1 (b) demonstrates our targeted attack method, which optimizes against the target text directly under the -aasiioonbaadd black-box box conditions, preventing semantic loss and maintaining semantic consistency with the target text.
black-box targeted attack approach for
Perturbing pixels in the image can change the output text. Therefore, the objective of the targeted attack can be considered to find the imperceptible pixel modification to make the output text similar to the target text. In this manner, the targeted attack can be formulated as a large-scale optimization problem, where pixels are decision variables and the optimization objective to poison the output text. Inspired by the distinctive competency of evolutionary algorithms for solving large-scale optimization problems [23, 24, 25, 26, 27], we develop a dedicated evolutionary algorithm- based framework for decision-based black-box targeted attacks
image-to-text models. However, directly applying evolutionary algorithms to solve this large-scale optimization problem could suffer from low search efficiency, due to the numerous pixels and their wide range of values. To address the issue, embed
Taiwan, Ministry of Culture and Tourism. Xiamen University, Xiamen 361005, Fujian, P.R. China (e-mail: minjiang @ xmu.edu.en; 36920221 4533155sttu..m.ccd..oo stu.x.uedu.cc).

---

dlenn image
attention heatmap
Attack
target semantics photograph
adversarial image
Ask
Attend
lake/ using /hdd I- /utting// walk
target model
"a cute using phone to take pictures of the fantastic TV" target text
picture shoto I camora --1 phone/ TV/
"the cute girl using cell phone to take picture fantastic TV" output
starning/ / photoggris/ / cste | Tastantie: clever soomi
semantics directory
Figure 2: Diagram of our decision- based black-box targeted attack method Ask, Attend, Attack.
three-step processes, i.c., Ask, Attend, Attack, into the proposed evolutionary algorithm- bbaed attacks. As shown in Figure during the Ask stage, attackers can arbitrarily specify words related certain semantics, such as photograph. Then, candidate words (e.g., camera, scenic, and phone) that are related certain semantics are searched. Meanwhile, these words are close to the clean image in the feature space of the target image-to-text model. By selecting words from the candidate words, target text (e.g., a cute girl using a phone to take pictures of the fantastic TV) related to attacker's specified semantics can be formed to poison the target model. Subsequently, based on the attention mechanism, Attnd identifies the crucial regions the clean image (e.g. attention heatmap) [28, 29], thus reducing the search space for the subsequent Attack. Lastly, Attack uses differential evolution strategy to impose imperceptible adversarial perturbations on the crucial regions, where the optimization objective is to minimize the discrepancy between the target text in Ask stage and the output text of the target model. Our contributions can be summarized as follows:
1. We first propose decision-basee black-box targeted attack Ask, Attend, Attack (AAA) for image-to-text models. Specifically, our method achieves targeted attacks without losing semantics while only the model's output text can be accessed.
2. We designed target semantic directory guide attackers in creating target and utilized attention heatmaps to significantly reduce search space. This improves the search efficiency of evolutionary algorithms in adversarial attacks and makes attacks difficult to perceive.
3. We conducted extensive experiments the Transformer- basedd VIT-GPT2 model and CNN+RNN-basee based Show-Attend Tell model, which are the two most-used image-to togge-tt-eextt models in HuggingFace, and surprisingly found that our decision-basee black-box method has stronger attack performance than existing gray-box methods.
2 Related work
2.1 White-box Attack
In white-box attacks, the attacker has full access all parameters, gradients, architecture of the target model, and the probability of each word of the output text. The authors in [7] add invisible perturbations the image make the image-to-text model produce wrong targeted text outputs. The authors in [30] add global or local perturbations to the image to make the vision and language models unable to correctly locate and describe the content of the image. The authors in [14] modify the content of the image at the semantic level to make the image-to-text model output text that is inconsistent with the original image. The authors [31] craft adversarial examples with semantic embedding of targeted captions as perturbation in the complex domain. The authors in [32] preserve the accuracy of non-target words while effectively removing target words from the generated captions. The authors in [33] generate coherent and contextually rich story endings by integrating textual narratives with relevant visual cues. The authors in [12] add limited-area perturbations to the image to make the image-to-text model fail to correctly describe the content of the perturbed area. The above methods require complete information of the image-to-text target model, including architecture, gradients, parameters, and probability distribution of the output text, which limits their practicality.

---

2.2 Gray-box Attack
To improve the practicality of adversarial attacks for image-to-text models, recent research explores how to attack with partial knowledge of the target model. All existing gray-box attack studies [34, 22, 13, 10] assume full access to the image encoder of the image-to-text model. The basic idea of gray-box targeted attacks is to reduce the distance between the adversarial image and the target image generated based on the target text in the image encoder's feature space. The authors in [34] generate adversarial images to mimic the feature representation of original images. The authors in [22] use a generative model to destroy the image encoder's features, achieving the untargeted attack. The authors in [13] minimize the feature distance in the image encoder between the adversarial image and the target image, thereby using gradient back propaaattnn to optimize the adversarial image and achieve the targeted attack. The authors in [10] combine existing gray-box method [13] with pseudo gradient estimation method [35] to achieve better performance in targeted attack. It is worth noting that they [10] call their method a black-box attack, but since they use the image encoder of the target model as the surrogate model, we classify their method as a gray-box attack. These gray-box attacks on image-to-text models are more practical than white-box attacks, but it is still unrealistic to assume that attackers can access the image encoder of the image-to-text model. Moreover, existing gray-box methods may have poor targeted attack performance due the semantic loss mentioned above.
3 Methodology
3.1 Problem Formulation
The image-to-text model G : X' + 3' maps the image domain X' to the text domain V A well-trained model should be able to accurately describe the content of the image using grammatically correct and contextually coherent text. Given a target text yt, the attacker's goal is to find an adversarial image Xadv that is visually similar to clean image x and can generate an adversarial text Yods that is semantically similar yy. We formalize the optimization problem for black-box targeted attack as:
arg max S(G(xadv), (. 91) s.t. M [xadv (i) - x(i)ll5 Xadi
(I)
where 5(+)) represents the semantic similarity function between two texts, e is the threshold for the average perturbation size per pixel, Xndv (i) and x(i) represents the value of the i-th pixel in the adversarial and clean images. n is the total number of pixels in all channels of the image.
3.2 Overview
To enhance the efficiency and stealth of decision- based black-box attacks, we propose the Ask, Attend, Attack (AAA) framework as shown in Figure 2. Ask: We compile a semantic dictionary from words within the input image's search space that align with the attacker's specified semantics. This facilitates targeted text generation, meeting the attacker's target semanties while simplifying the search process. Atlend: We employ attention visualization and surrogate model to generate attention heatmap for the target text on the image, narrowing the search to significant decision variables and enhancing perturbation stealth. Attack: We use the differential evolution in the reduced search space to find the optimal solution that can mislead the target model to output target text. The framework's 's pseudo-code is detailed in Appendix A.I.
3.3 Ask Stage
According to the target semanties. the goal of Ask is to find words in the feature space of the target model to form a target semantic dictionary. These words should be closer to the input image. Firstly. we treat each pixel in each channel of image x as a variable, which means the search space size the product of length, width, and number of channels. And then generate NP (number of population) individuals to form population based on the following formula:
xi x(i) + 10,
where x(i) is the i-th variable of clean image x, xi) is the i-th variable of the j-th individual in the population, 1) is a hyperparameter about the maximum search range, rand(-1,1) is random number from the range of -1 to 1.

---

Secondly, for each variable, random mutation occurs between different individuals. The mutation for the i-th variable of the j-th individual x, (i) as follows:
(1 = 1 (1) + (x22(1) (1),
where (i) is the mutated variable for mutation in the g-th generation of xy(1). x (1), (1) and (i) are three randomly selected individuals from the current population who are different from each other, F is the scaling factor.
Thirdly, each individual crossovers with the mutated individuals with certain probability of generat- ing candidate individuals. The formula is as follows:
(i), rand (0, s CR,
(4)
(1).
otherwise,
where CR is crossover probability factor, uy (i) is the i-th variable of the candidate individual in the g-th generation of the j-th individual in the population.
Fourthly, we use WordNet [36], synonym dictionary, to measure the similarity between the target semantics and each individual's 's output text. WordNet groups words with the same semantics into synonyms, each representing a basic concept.
use WordNet to count the same semantic words m between each individual's output text and the target semantics. We calculate the Precison=(m/1) and Recall=(m/r) where the output text word count and r is the target semanties word count. Then, calculate semantic similarity using the following formula:
- 7 ch (a² + +) Precision Recall
S Seem
(5)
a²2 Precision + Recall
where Seem is the semantic similarity between the individual's 's output text and the target semantics [36], balances the precision and recall weights, y and 6 control the penalty factor strength, ch is the number of consecutive word sets that match between the output text and the target semanties, with fewer chunks meaning more consistent word order.
Ultimately, select offspring based Ssem: choosing the current and candidate individuals that match the target semantics hetter as the next generation:
(emmG((u)) TS) 2 Ssem (semm( (I(x),,
(6)
otherwise,
where TS is the attacker's target semantics. We extract nouns, adjectives, and verbs from the output texts of all the more semantically relevant and preserved individuals of each generation, expanding the target semantic dictionary. We use g(x = (W1, "2,... w.. } to represent the target model I's output text for the next generation of individuals, where w; is the i-th word of the text and n is the word count. Then we use the following formula to extract important words and make dictionary:
Do+1 = {w E G(x]+1) w is noun, adjective or verb).
(7)
where D is the dictionary for the preserved individual. We combine the dictionaries of each preserved individual in each generation get the target semantic dictionary D D. where m is the total number of generation. The attacker selects words from dictionary D that match the specified semantics make the target text yı- Words in dictionary D near input image in feature space enhance searchability, enabling more efficient targeted attacks.
3.4
Attend Stage
The goal of Attend is to calculate the target text's attention area on the image x. Because we do not have access to the internal information of the target model, we can only calculate the Grad-CAM attention heatmap [37] with the help of surrogate model f (such as ResNet trained in ImageNet). The surrogate model's sole purpose to the compute attention heatmap. Since different models produce similar heatmaps for the same target text and input image, selecting a well-established visual model suffices
[38]. The calculation formula of attention heatmap A as follows:
(,,
Oy"
A(i,) = MAX 0,
0.7
Fa(t.))
(8)
OFF(i.))
k
5

---

where j(i,)) is the decision- making contribution of the image to the target text at pixel (i.j), (i is the pixel (i. of the feature map of the k-th convolution kernel of the last convolutional layer of the surrogate model f. Z is the feature map's pixel count, y the probability that f predicts that the image x belongs to class c". We use C {ci, c2. C1000) ) for the ImageNet category names, where & is the i-th category name. We make the category text Ve: = "a photo of + cs from the category name c;. We calculate the category c* as:
E(m)) Elya)
(9) where E is the text encoder of the pre-trained CLIP model, and c* is the closest category to the target text. We substitute et into Formula 8 to get the target text's attention heatmap A. A (i, is the pixel (i,j))ss contribution to the target text. means more contribution, and 0 means less contribution.
c" = argmar a
Table 1: Performance comparison (%) of different attack methods
VIE:GPT2
Show-Anend-
Attack Methods
METEOK
BLEU
CLIP
SPICE
METEOR
BLEU
CLIP
SPICE
Clean Sample
0.201.0.11
0.2400111
0.6410.07
0.156.0.07
0.2110.11
0.22910.13
0.6/66.0.09
0.1
transter (black)
0.306e50 11
0.246H0.11
0.69HD0077
0.165=0.00
0.211=0012
0.225+0:14
0.64840099
0,18540.11
trarr (black)
0.221-0.10 16
0.264+0.15 15
0661166118
0.167+0.07
0.219+0.11 11
0
0664400.55
0.187H0.11 14
transfer (gray)
0,414:5022 23
0.29640.14
0.21+2.09
0,32+0.16 16
0.382e0028
0.348+0.17 17
0.782-0.11
0.299H0117
transferequery (grag)
0143520:21
0.41180.12
0.83260.13
0.35
0.401
0.355=0.1 15
0.79220:11
0.311
AAA (wlls Altend)
0.541-0:25
0.519+0.19
0.84400224
0477+0
0.642=0.0 19
0.564+0 19
0.841+0066
AAA twin Ask)
0.508 21
0.38460. 18
0,795+0.25
0.412
0.364e0.21
0.32240.19 19
0.75240:08
0.376600.1
AAA
0.696:0.211
0.658s0.22
0.95240.29
063440..15
0,855
0.79980.21
0,96410.04
0,78640.14
transter (blakk)
0.30
0.24120.15
0.0220.11
0.10520000
023020011
0.61350005
0.18710000
translerqquery (black)
0.211:0014
0.25610.15 15
0.61110.15
0.18120.09
0.245
0.21610.11
0.66610.06
0.203300000 09
transfer (gray)
0.398a00224 24
0.38120.15
0.1622.11
0.325
0.36120224
0.35920.17
0.77820.11
0.29%650.16
transfer-query (gray)
0.408.0.19
0
0.82410.15
0,34120.13
0.375
0.36820115
0.78420.11
0.31120.13
AAM (edo Altend)
0.461=0
0,42300115
0808400111
0.375=00.09
0,438a0015
0444400.16
0.827+0:04
0,422+0.14
AAA twoo Asky
0.57820:25
0.36120117
0,76810.15
0.356:0.15
0.34120015
0.33720.18
0.74920:07
0,36520.13
AAA
0.556:0.31
0.50420.26
0.851200112
0,44:0.17
0.61720.25
0.574
0.913:00005
0,55350.14
3.5 Attack Stage
The goal of Attack is to search for the best individual (ad sample) that outputs the target text y/ in the smaller search space reduced by the attention heatmap. Firstly, we copy the attention heatmap A three times in the channel dimension to match the shape of the image x. We generated NP (number of population) individuals as a population with this formula:
xi x(i rand(-Alr (i), (i)) -77
(10)
where x(i is the i-th variable of clean image x, (i) is the i-th variable of the j-th individual in the population, (i) is the contribution of the i-th variable to the target text, and cand(-A()) (i)) is random number in the range from (i) to A At)). The value of A is less than 1, and its mean and
median are about [0.3.0.4]. The search space volume from the attention heatmap is much smaller than a hypersphere with radius 7) because the radius and volume have an exponential relationship. This improves the search efficiency and concealment of adversarial perturbation.
Secondly, in order to accelerate convergence and better find the global optimal solution, we use the following CurrentToBest mutation [39]]
xx( + F (x (1) - xf22 + F * (x (i)
(11)
where xi( is the i-th variable of the j-th individual in the g-th generation, (i) is the mutated variable, xfest is the best fitness individual in the g-th generation population, x%1 and x2 are two randomly selected individuals in the g-th generation population, and F is the scaling factor. The main advantage of this mutation strategy is that it combines the information of the current individual x; and the best fitness individual Xbest, which can better guide the search process towards the direction of the optimal solution.
Thirdly, we use Formula 4 to calculate the candidate individual uy (t). We design the following formula to calculate the deep feature similarity Setip between two texts (u and u):
F(u). E(v)
S Selip =
(12)
[Bu E())l

---

CUP sccre11 e:33.3
CL G.Proore112 4-3171
Figure 3: We compared the convergence curves of populations with and without Attend under the same perturbation size in (a-b). The fitness function is Sctip in Formula 12, where lower values mean stronger attacks. The dashed line is the average fitness value, and the solid line is the best fitness value. The green line AAA and the red line AAA w/o Attend. (c) shows the attention heatmap. (d) and (e) show the visual effects of adversarial image with and without Attend, with minimal perturbation of 100% attack success rate.
ResNet50
DenseNet121
Vision-Transfrrc
ResNet50
DenseNet121
Vision- Trann-rranformer
M:0.86 8:0.81 C99.6 5..71 M:0.82 8:0.78 CO.91 5:.47 M:0.88 8:0.84 C:0.95 5:0.2 M:0.88 8:0.89 C:0.66 59.71 M:0.22 8:0.91 C.0.99 5:0.75
Mi0.91 8:0.9 C:0.88 5:0.75
Figure 4: Grad-CAM attention heatmaps of different surrogate models for the same target text woman is holding a pair of shoes. is METEOR, BLEU, is CLIP, S is SPICE.
where E is the text encoder the pre-trained CLIP model. Text discrete and complex, so it cannot calculate the distance directly [40]. Therefore, features of the texts, and then calculate the feature distance to obtain the similarity Sclip between the texts. The closer Selip is to O, the higher the similarity between the two texts and v.
the CLIP text encoder E to extract the deep
Ultimately, we select offspring using the following formula:
Sdip(g(ug), ye) S Sz)p((((x)).
x-+1_
(13)
otherwise,
where x the next individual with closer feature distance between the output text and the target text yt- After performing the above evolutionary calculations multiple times, the optimal solution (adversarial sample) for outputting the target text found.
Evaluation and Results
4.1 Experiment setups
Model and dataset We experimented with the two most-used image-to-text models Hug- gingFace; VIT-GPT2 (Transformer-based)) [41] and Show-Attend-eal (CNN+RNN-basee based) [42]. VIT-GPT2 was trained on ImageNet-21k. Show-Attend-Tell was trained MSCOCO-2014, We only used the target model's 's output text, not its internal information like gradients, parameters, or word probability. Following this work [13], used Flick30k
dataset, which has 31783 images and 5 caption texts each. We removed samples with less than 0.7 similarities between predicted text and truth text ensure the target model's accuracy on clean images.
Evaluation metrics We used these evaluation metrics in our experiments: (1) B#EU an early machine translation metric that measures text precision [43]. 1 means similar, and means dissimilar. (2) METEOR, a more comprehensive metric that considers synonyms, stems, word order, etc [36]. 1 means similar, and 0 means dissimilar. (3) CLIP, the distance between the CLIP text encoder's deep features for two texts [44]. means similar, and 0 means dissimilar: (4) SPICE, an evaluation metric tailored for image-to-text models [45]. means similar, and means dissimilar. (5) e, the mean perturbation size of each pixel of the adversarial sample [13].
7

---

M:0.99 B:1.0 C:1.O a man is locking at tel
M:0.81 8.0.82 82 C:0.97 man looking at his phone in mirror
M:0.52 8:0.53 C.O.B man standing in the street with ce phone
M:0.14 8:0.17 C:0.67 young boy playing with tailet
phone mirror
(b)
ours
(8
(h)
existing work
standing living roumi holding a wi rermute
londing
standing
standing living
at rernole in rourn
rour with chair M:0.47 B:0.43 C:0.72
r00m holdinng wii remole
M:0.34 6:0.32 C:0.64
M:0.67 8:0.59 C:0.77
M:0.29 8:0.21 C:0.69
Figure 5: Performance of adversarial image attacks varies with perturbation size c. The e of (a) and (f) is 25, e of (b) and (g) is 15, e of (c) and (h) is 10, of (d) and (i) is 5. (e) is our attention heatmap of the target text on the image. (i) is the target image generated based on the target text used in existing works. M is METEOR score, B is BLEU score, and C is CLIP score.
4.2 Experiment results
Comparison experiment of existing gray-box attacks. We evaluate state-of-the-art gray-box attacks [13, 10] on image-to-text models. We designate the gray-box attack [13] as transfer (gray) and the one [10] as transfertquery (gray). To simulate a black-box environment, we adapted these gray-box attacks by employing the CLIP model's image encoder in lieu of the target model's encoder, resulting in "transfer (black)" and "transfer+query (black)" variants. As depicted in Table adversarial samples generated by the original gray-box attacks exhibit a marked increase in textual similarity to the target text when compared to clean samples. Conversely, the black-box adaptations maintain a similarity level akin to that of clean samples, indicating a significant loss of attack capability upon changing the image encoder. This underscores the dependency of gray-box attacks on the target model's image encoder. Our proposed method AAA demonstrates superior attack performance in black-box scenarios compared to the existing methods in their native gray-box settings. This is attributed to the semantic loss inherent in existing gray-box attacks, which constrains their attacking potential. It is noteworthy that our work represents the first black-box attack on image-tu to-text models. So we can only compare our approach with existing gray-box attacks. We have adapted these gray-box attacks into black-box version solely to demonstrate their ineffectiveness in a black-box scenario.
Ablation experiment of our black-box attack. We conducted ablation experiments on our AAA method. AAA (w/o Attend) means no attention heatmap to reduce the search space. but the proportional reduction of the search range. AAA (w/o Ask) means the target text not from the target semantic dictionary, but random words. Table 1 shows that losing any module decreases our attack performance. In addition, Ask performs worse than AAA (w/o Attend), indicating that finding a target text with lower search difficulty contributes relatively more to the performance of our targeted attack.
Qualitative experiment of attention. We presented the optimization curves of AAA and AAA (w/o Attend) in Figure 3. Figure 3 (a) and (b) illustrate the best and average fitness values during AAA and AAA (w/o Attend) optimization of VIT-GPT2 and Show-Attend-oe inclusion of Attend expedites and enhances the convergence of the population, with an equivalent perturbation size. Consequently, AAA exhibits more effective concealment in adversarial perturbations, maintaining the same level of attack efficacy, as depicted in Figures (d) and (e). Furthermore, we evaluated the impact of selecting different surrogate models during Attend. Notably, the sole function of the surrogate model is to compute the attention heatmap. Figure 4 demonstrates that, despite significant structural variances among several surrogate models, they produce strikingly similar attention heatmaps for the same target text and input images. This similarity arises from mapping the target text to the most pertinent category within the surrogate model's 's label space (as Formula 9).
It is evident that the
8

---

VIEGPT2
VIEGET2
Sham--AttendAAtan
Shen-AAlend-Ae Altack
(a]
(b)
(c)
(4)
30.
300
cmmpuaaion Timn Dassorng
Computaiion Timm (espord)
Figure 6: Comparison of computation time for generating single adversarial sample using different adversarial attack methods. The y-axis is a measure of similarity between the generated text and the target text, with higher values indicating better target attack performance. The x-axis represents the computation time, and the shorter the time required to find a stable solution, the better.
The position of the same category of objects on the same picture is constant, and the model needs to focus on the object first, no matter what structure it is [38]. Performance comparisons, shown in Figure 4, indicate that the similarity in attention heatmaps across different surrogate models leads to similar final attack performances. Therefore, we opted for stable, well-established, pre-trained model, such as ResNet-50, to serve as our surrogate model.
Qualitative experiment of different perturbation sizes. We used the words mirror, cell phone, man, looking at from the target semantic dictionary (as shown in Appendix B.2) to make the target text a man is looking at a cell phone mirror. We compared output texts of our black-box method AAA and the existing gray-bos method [10] for adversarial samples with different c, the average pixel perturbation size, in Figure The same conclusion drawn from both methods that bigger perturbation causes worse concealment and better attack performance; too small perturbation causes attack failure. Moreover, (f) and (j) in Figure 5 show that the existing methods have a semantic loss that limits their attack performance. Subjectively, target image (i) accurately draws the semantics of the target text, and the output text of adversarial image (f) perfectly describes the content of the target image (j). However the adversarial sample (f)'s output text does not have the semantics of the target text. Our method does not have semantic loss, so our black-bor method AAA does a better targeted attack than the existing gray-box method. More examples of semantic loss are in Appendix B.
Comparison experiment on computation time. We evaluated the computational efficiency of various attack methodologies for generating adversarial samples in image-to-text models. As depicted in Figure 6, our black-box attack method AAA, demonstrates longer computation time to reach an optimal solution compared to existing gray-box attacks. For instance, the transfer approach [13] illustrated in Figure 6 (a) produces an adversarial sample with CLIP score of 0.82 within mere 29 seconds, while the transfer+query approach [10] achieves a CLIP score of 0.85 in just 97 seconds. Conversely, our AAA method requires 151 seconds to generate adversarial sample with superior CLIP score of 0.951. The shorter computation times of the existing gray-box methods are expected due to their ability to access real gradients, which significantly expedites the optimization process. Given that adversarial attacks are not time-sensitive operations and considering that our AAA method delivers more potent attack capability and is applicable in a broader range of realistic black- box scenarios, the trade-off for a higher computational cost is deemed acceptable. Additional experiments on similarity measurements are included in the Appendix B.5.
Further analyses. Firstly, we show the impact of different forms of target semantics TS in Ask on the target semantic dictionary, as shown in Appendix B.2. More ambiguous target semantics can enrich the target semantic dictionary, which also means that the attacker has more choices when designing yt- Secondly, we show the effect of different word selection strategies of yt based on target semantic dictionary on the final attack effect, as shown in Appendix B.3. Thirdly, we compare the convergence curves of different population sizes and choose a population size of 40 based on the trade-off of attack performance and convergence efficiency, shown in Appendix B.4. Furthermore, we compare the effects of different evolutionary algorithms on attack performance and convergence efficiency, as shown in Appendix B.6. Additionally, to better observe the attack effect of our framework, we show more examples of attention heatmaps A, optimization convergence curves, target text yt. and output text, as shown in Appendix B.7. Lastly, we discuss the limitations of our framework, defense strategies, and future work in Appendix C.

---

5 Conclusion
In our research, we introduce a novel and practical approach for adversarial attacks on image-to- text models. We propose the Ask, Attend, Attack (AAA) framework, a recision-based black-box attack method that achieves targeted attacks without semantic loss, even with access limited the target model's 's output text. Our framework uses the target semantic directory to guide the creation of target text and attention heatmap to reduce the search space, thereby improving the efficiency of evolutionary algorithms and making our attack harder to detect. Our extensive experiments on the Transformer-basee VIT-GPT2 model and the CNN+RNN-basee Show-Attend-Tie model demonstrate that our decision-based black-box method outperforms existing gray-box methods in targeted attack performance. These findings highlight the vulnerabilities in current image-to-tets models and underscore the need for more robust defense mechanisms, significantly contributing to the field of adversarial machine learning and enhancing the security of vision-lanpe laooonlangugge systems.
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant No. 62276222 and the Public Technology Service Platform Project of Xiamen City, Grant No.3502220231043..
References
[1] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Proceedings of the International Conference Machine Learning (ICML), pages
PMLR, 2022.
[2] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint 2.Xv:23011.1299 2023.
[3] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Ziticck and Devi Parikh. Vqa: Visual question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2425-2433. IEEE, 2015.
[4] Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: Vision-ama aan-uang transformer without convolution or region supervision. In Proceedings of the International Conference on Machine Learning (ICMI.. pages 5583-5594. PMLR, 2021.
[5] Qiong Cao, Li Shen, Weidi Xie, Omkar Parkhi, and Andrew Zisserman. Vggface2: A dataset for recognising faces across pose and age. In Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition (FG), pages 67-74. IEEE, 2018
[6] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee, Vilbert: Pretraining task-agnostic visiolin- guistic representations for sisiionand-languaaee tasks. Proceedings of the Neural Information Processing Systems (NIPS). 32, 2019.
[7] Hongge Chen, Huan Zhang, Pinyu Chen, Jinfeng Yi, and Cho-Jui Hsieh. Attacking visual language grounding with adversarial examples: case study neural image captioning. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 2587-2597. Association for Computational Linguistics, 2018.
[8] Baoer Liu, Qingyuan Zeng, Jianbin Huang, et al. Ivim using convolutional neural networks predicts microvascular invasion in hec. European Radiology, 32(10):185-7195, 2022.
[9] Yunpeng Gong, Liqing Huang, and Lifei Chen. Person re-identification method based on color attack and joint defence. Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 4313 4322, 2022.
[10] Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang. Chongxuan Li, Ngai-Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models. In Proceedings of the Neural Information Processing Systems (NIPS), 2023.
10

---

[11] Yunpeng Gong, Chuangliang Zhang, Yongjie Hou, Lifei Chen, and Min Jiang. Beyond dropout: Robust convolutional neural networks based on local feature masking. In 2024 International Joint Conference Neural Networks (IJCNN), pages 1-8. IEFE, 2024.
[12] Hyun Kwon and SungHwan Kim, Restricted-area adversarial example attack for image caption- ing model. In Wireless Communications and Mobile Computing (WCMC). Hindawi, 2022.
[13] Raz Lapid and Moshe Sipper. I see dead people: Gray-box adversarial attack on image-to-text models. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2023.
[14] Anand Bhattad, Minjin Chong, Kaizhao Liang, Bo Li, and D. A. Forsyth. Unrestricted adver- sarial examples via semantic manipulation. In Proceedings of the International Conference on Learning Representations (ICLR). ICLR, 2020.
[15] Yinpeng Dong, Hang Su, Baoyuan Wu, Zhifeng Li, Wei Liu, Tong Zhang, and Jun Zhu. Efficient decision-basee based black-box adversarial attacks on face recognition. 2019.
[16] Yucheng Shi, Yahong Han, Qinghua Hu, Yi Yang, and Qi Tian. Query- black-box adversarial attack with customized iteration and sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence. 45(2), 2022.
[17] Yunpeng Gong, Yongjie Hou, Zhenzhong Wang. Zexin Lin, and Min Jiang. Adversarial learning for neural pde solvers with sparse data. arXiv preprini arXiv:2409.02431, 2024.
[18] Yunpeng Gong, Qingyuan Zeng, Dejun Xu, et al. Cross-modality attack boosted by gradient- evolutionary multiform optimization. arXiv preprint arXiv:2409.17977 2024.
[19] Yunpeng Gong, Zhun Zhong, Zhiming Luo, Yansong Qu, Rongrong Ji, and Min Jiang. Cross-modality perturbation synergy attack for person re-identification. arXiv preprint arXiv:2401. 10090, 2024.
[20] Shuai Jia, Yibing Song, Chao Ma, and Xiaokang Yang. lou attack: Towards temporally coherent black-box adversarial attack for visual object tracking. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[21] Kaixun Jiang, Zhaoyu Chen, Hao Huang, Jiafeng Wang, Dingkang Yang, Bo Li, Yan Wang, and Wenqiang Zhang. Efficient decision- black- box patch attacks on video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4379 2023.
[22] Hanjie Wu, Yongtuo Liu, Hongmin Cai, and Shengfeng He. Learning transferable perturbations for image captioning. ACM Transactions on Multimedia Computing, Communications and Applications, 18(2), 2022.
[23] Mohammad Nabi Omidvar, Xiaodong Li, and Yi Mei. Cooperative co-evolution with differ- ential grouping for large scale optimization. IEEE Transactions evolutionary computation, 18(3):378- 393, 2013.
[24] Zhenzhong Wang, Haokai Hong, Kai Ye, Guangen Zhang, Min Jiang, and Kay Chen Tan. Manifold interpolation for large-scale multiobjective optimization via generative adversarial networks. IEEE Transactions on Neural Networks and Learning Systems, 34(8)44631-4655, 2023.
[25] Zhenzhong Wang, Lulu Cao, Liang Feng, et al. Evolutionary multitask optimization with lower confidence bound-based solution selection strategy. IEEE Transactions on Evolutionary Computation, 2024.
[26] Zhenzhong Wang, Dejun Xu, Min Jiang, et al. Spatial-temporal knowledge transfer for dynamic constrained multiobjective optimization. IEEE Transactions on Evolutionary Computation, 2024.
(27] Zhenzhong Wang, Qingyuan Zeng, Wanyu Lin, et al. Multi-view subgraph neural networks: Self-supervised learning with scarce labeled data. arXiv preprint 2xXhv22404112569 2024.
11

---

[28] Qingyuan Zeng and Wu Zhou. An attention based deep learning model for direct estimation of pharmacokinetic maps from dce-mri images. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 2368 2375. IEFF, 2021.
[29] Qingyuan Zeng, Baoer Liu, Yikai Xu, al. An attention- deep learning model for predicting microvascular invasion of hepatocellular carcinoma using an intra-voxel incoherent motion model of ceffusioo-weightee magnetic resonance imaging. Physies Medicine & Biology, 66(18): 1 2021.
[30] Xiaojun Xu, Xinyun Chen, Chang Liu, Anna Rohrbach, Trevor Darrell, and Dawn Song. Fooling vision and language models despite localization and attention mechanism. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2018.
[31] Shaofeng Zhang, Zheng Wang, Xing Xu, Xiang Guan, and Yang Yang. Fooled by imagination: Adversarial attack to image captioning via perturbation in complex domain. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2020.
[32] Jiay Ji, Xiaoshuai Sun, Yiyi Zhou, Rongrong Ji, and Fuhai Chen. Attacking image captioning towards
target words removal. In Proceedings of the ACM International Conference on Multimedia (ACM MM). ACM, 2020.
[33] Qingbao Huang, Chuan Huang, Linzhang Mo, Jielong Wei, Yi Cai, Ho-fung Leung, and Qing Li. Igseg: Image-guided story ending generation. In Findings of the ACL: International Journal of Conference on Natural Language Processing, 2021.
[34] Akshay Chaturvedi and Utpal Garain. Mimic and fool: tank-tgnostic adversarial attack. IEEE Transactions on Neural Networks and Learming Systems, 32(4):1801-1805 2020.
[35] Yurii Nesterov and Vladimir Spokoiny, Random gradient- minimization of convex functions. Foundations of Computational Mathematics, 17:527-566, 2017.
[36] Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization (ACL WIEEMMTS), pages 65-72, 2005.
[37] Ramprasaath R. Selvaraju, Michael Cogswell, and Abhishek Das. Grad-cam: Visual explana- tions from deep networks via gradient-based localization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 618-626, 2017.
[38] Jiakai Wang, Aishan Liu, Zixin Yin, Shunchang Liu, Shiyu Tang, and Xianglong Liu. Dual attention suppression attack: Generate adversarial camouflage in physical world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8574, 2021.
[39] Jingqiao Zhang and Arthur C Sanderson. Jade: adaptive differential evolution with optional external archive. IEEE Transactions on evolutionary computation, 13(5):945-958, 2009.
[40] Jyun- Yu Jiang, Mingyang Zhang, Cheng Li, Michael Bendersky, Naday Golbandi, and Marc Najork. Semantic text matching for long-form documents. In Proceedings of the World Wide Web Conference (WWW), pages 795-806, 2019.
[41] NLP Connect. vit-got2-image-capiionng (revision 0e334c7). https://hugiingaacecom
nlpconnett/vit--pt2-- Sewpptt-immage-one inw-eiiaagee-teee capteeccaptiiiing 2022.
[42] Kelvin Xu, Jimmy Ba, and Kiros Jamie. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the International Conference Machine Learning (ICMI.), pages 2048-2057. PMLR, 2015.
[43] Kishore Papineni and Salim Roukos Bleu: method for automatic evaluation of machine translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 311 311-318, 2002.
12

---

[44] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal. Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. In Proceed- ings of the International Conference on Machine Learning (ICML), pages 8748-878 8763. PMI.,, 2021.
[45] P. Anderson, B. Fernando, M. Johnson, and S. Gould. Spice: Semantic propositional image caption evaluation. Proceedings of the European Conference on Computer Vision (ECCV), pages 382-398. Springer, 2016.
[46] Zhenzhong Wang, Qingyuan Zeng. Wanyu Lin, Min Jiang, and Kaychen Tan. Generating diagnostic and actionable explanations for fair graph neural networks. In Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI). 2024.
[47] Rainer Storn and Kenneth Price. Differential evolution- a simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization, 11:341-359, 1997.
[48] Wael Khatib and Peter J. Fleming. The stud ga: mini revolution? In Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN) pages 683-691. Springer Berlin Heidelberg, 1998.
13

---

Overview
In this appendix, we describe implementation details, additional experiment results and analyses to support the methods proposed in the main paper. In addition, we show more examples of black-box adversarial attacks using AAA, each of which includes clean image, attention heatmap, adversarial image, optimization curve, target text, output text, and attack performance.
Reproducibility
Our source code and data are included the supplemental material and uploaded, and we will publish the code GitHub after the paper is accepted. We provide concise and understandable pseudo-code below.
Contents
Introduction
2
Related work
2.1 White-box Attack
2.2
Gray-box Attack
3
Methodology
3.1
Problem Formulation
3.2
Overview
3.3
Ask Stage
3.4
Attend Stage
3.5
Attack Stage
4
Evaluation and Results
4.1
Experiment setups
4.2
Experiment results
5
Conclusion
10
A Additional implementation details
16
A.1 Pseudo code of our proposed framework
16
A.2 Basic setups
16
A.3 Standard deviation in the experiments
16
A.4 Evaluation metrics
17
Additional experiments
17
B.1 Analysis of semantic loss
17
B.2
Comparison experiment of target semantic dictionary
18
B.3
Word selection strategies for target semantic dictionaries
19
B.4
Comparison experiment on population size
20
B.5
Comparison experiment on computation time
20
14

---

B.6 Comparison experiment of optimization algorithms
21
B.7 Visualization of more adversarial samples
22
Discussion
22
C.I Limitation
22
C.2 Future work
22
15

---

A Additional implementation details
A.1 Pseudo code of our proposed framework
Algorithm Ask, Attend, Attack (AAA) Framework
1: Input: Image x, Target text y/s Target semantics TS, Surrogate model f. Pre-trained CLIP model E'
2: Output: Adversarial image Xadv that generates text Yadv semantically similar to ye
3: Initialize hyperparameters: population size NP, mutation factor F, crossover probability CR, perturbation threshold c, maximum search range 7)
4: Initialize target semantic dictionary D +-
5: function Ask(x. TS)
6: Generate initial population with perturbations using Eq. (2)
7:
for each generation do
8:
Perform mutation using Eq. (3)
9:
Perform crossover using Eq. (4)
Calculate semantic similarity Seem using Eq. (5)
11: Select offspring based on Ssem using Eq. (6)
12:
Update D with relevant words from (xx)+t) using Eq. (7)
13: end for return D
15:
end function
function ATTEND(x, 3/. f)
17: Determine the category c" closest to using Eq. (9)
Attention heatmap A is calculated by surrogate model using Eq. (8)
19: return A
20: end function
21: function ATTACK(X, Vt. A)
Generate initial population with attention-guice perturbations using Eq. (10)
23:
for each generation do
24:
Perform CurrentToBest mutation using Eq. (11)
25:
Perform crossover using Eq. (4)
26:
Calculate deep feature similarity Selip using Eq. (12)
27: Select offspring based Setip using Eq. (13)
28: end for
29:
return Best individual as Xadv
30: end function
31: D Ask(x, TS)
32: D1 < The attacker create sentence from the dictionary D
A ATTEND(x, Vt.
34: Xadv <- ATTACK(X, Yt. A)
A.2 Basic setups
We set the population size NP to 40, scaling factor F to 0.7, cross probability factor C.R. to 0.7, ? to 0.5, 1, and 3, and 7 to required in the experiment divided by the average of attention heatmap A. Our device uses three GPUs of RTX2080ti with 11GB memory, and a CPU of Intel(R) Core(TM) 10400F. Our operating system linux, the evolutionary algorithm framework uses the Geatpy library, and the deep learning framework uses Pytorch.
A.3 Standard deviation in the experiments
In the quantitative experiment of our paper, experiments were repeated for 10 times, and the optimal performance was obtained for each experiment, and the mean value and standard deviation were finally obtained.
16

---

Target Image:
Target Image:
Target Image:
Target Image:
(c)
Target Text:
Target Text: picture of a man phone
Target Text:
Target Text:
group of people sitting down
man is watching tv
a bird sitting on a bird flying near a street
Output Text:
Output Text:
Output Text:
Output Text: bird flying
a woman white dress is talking to man in a white dress person on cell phone
person is taking picture of
sitting on chair to fire hydrant
lodge with
bird perehed on
Similarity: M:0.04 B:0.05 C:0.71 B:0.08
Similarity: M:0.73 B:0.51 C:0.88 B:0.4
Similarity: B:0.17 B:0.18
Similarity:
M:0.43 B:0.5 C:0.84 B:0.33
Figure 7: More examples of semantic loss of existing gray-box targeted attacks. The target text the prror-generated text of the maage-to-text model that the attacker wants to obtain. The target image the image generated by using the sott-oo-image model (Stable Diffusion) based on the target text. The output text is based on the target image using the image-to-ted target model VVTT-GPT2/Show- Attend-Tell), similarity indicates the similarity between the target text and the output text. We also show the similarity between the target text and the output text. M stands for METEOR score, B for BLEU score, C for CLIP score, and S for SPICE score.
A.4 Evaluation metrics
(1) iteration, the number of iterations for the differential evolution algorithm Attack to find the optimal solution (no more fitness convergence). Fewer iterations mean fewer queries and faster attack. (2) e, the mean perturbation size of each pixel of the adversarial sample. Smaller value means higher concealment of adversarial perturbation. (3) diversity, the number of words in the target semantic dictionary from Ask. More words mean more diversity. (4) correlation, the average CLIP score between each word in the target semantic dictionary and the target semantics. The higher correlation, the more relevant the words in the target semantic dictionary are to the target semantics.
B Additional experiments
B.1 Analysis of semantic loss
We show more examples of the semantic loss phenomenon, as shown in Figure 7. In order to realize the targeted attack with the existing gray-box methods, necessary convert the target text into the target image with the help of text-to-image model (such as Stable Diffusion). Then the distance between the adversarial image and the target image narrowed, that the text decoder of the image-to-text target model mistakes the adversarial image as the target image and outputs the description the target image incorrectly. The target image often contains more semantic information than the target text, and the image-to-text target model may focus on the semantic information that not specified by the attacker, which leads semantic loss. For example, in Figure 7 (c), the text-to-iman model generates the target image corresponding to the target text (a man is watching tv) very accurately, and the image-to-te mn-getto-text target model also generates the output text (a man sitting on chair next to a fire hydrant) of the target image very accurately, but the output text and the target text are very different. This means that even there a gray-box method that can completely make the features of the adversarial image identical to the features of the target image, the image-to-tex target model can only generate the output text after semantic loss, and the targeted attack performance limited by semantic loss.
17

---

250
200
(a)
(b)
190
150
120
130
80
50
20
bird
flying
air
cat
small
sitting
king
ground
dog
laying
looking
holding
mirror
phone
camera
taking
picture
selfle
girl
smile
Figure 8: The top ten words target semantic dictionary for different target semantics, with
word frequency on the vertical axis. (a) is for animal; (b) is for photograph.
B.2 Comparison experiment of target semantic dictionary
We showed the target semantic dictionary's 's diversity and correlation for different perturbations in Table More perturbation means more word choices for target text. The correlation between dictionaries and target semantics is not affected by the size of perturbations. We also see that one vague word for target semantic makes more diversity and relevance in the dictionary than the detailed sentences. This is because word has vague semantics, resulting in more words that are closer the input image the feature space being added to the dictionary. So we suggest using simple words as target semantics, attackers get richer dictionaries to make target text.
Table 2: Target semantic dictionaries for different semanties. animal word means the vague word animal, while animal sentence means a dog running after a cat. photograph word means the vague word photograph, while photograph sentence means photo of parking lot.
semantic
animall word
animal senience
photograph word
photogruph sentence
diversity?
50.6
65.4
90.1
38.9
54.1
79.6
51.7
62.3
87.7
43.1
52.6
73.5
comelation (%))
0.746
0.742
0.744
0.653
0.65
0.654
0.842
0.841
0.83
0.765
0.761
0.758
Table 3: Output text under different word selection strategies.
Strategy
Target Text
Output Text
Similarity
a bird is flying
a bird is flying great
A
through air
through the air
a girl is taking pictures by cam- era
a girl is using
A
camera to take great pictures
a camera is fly- ing through the air
a man is holding a medium
B
camera
giraffe is eat- ing grass
person is cutting a piece of food
C
bad
a boy captur- ing beautiful moment
a man is looking his cell phone
C
bad
the helicopter is hovering in the sky
man is holding knife in his hand
D
bad
18

---

pepudalion 30
pepudation
Sox Oapostvv sike
02-
200
D15
25
130
Nomber
33
prexratton 40
Conrage Digettin
Sol Dbscc Vike
0.2
01-
34-
3.
130
Gomoniteee Namtor
Preroge Obeecivv
Aettage
03
02-
Figure 9: The best fitness curve and average fitness curve of the population under the same target text and different population sizes.
Table 4: The time (s) required for one optimization iteration under different population sizes.
population
10
20
40
60
80
100
iteration time
0.41
0.65
1.14
1.65
2.14
2.56
B.3
Word selection strategies for target semantic dictionaries
We showed the words and frequencies in the target semantic dictionary for different semantics in Figure 8. We compared different word selection strategies for targeted attacks with these dictionaries. The results show that: (1) Words in the dictionary do better when the semanties are similar, while words outside may fail; (2) Words from two dictionaries in one sentence decrease the performance.
We used four word selection strategies based on two dictionaries in Figure 8 to compare how different target texts yt affect our method: (A) All words in y are from the same dictionary; (B) Some words in Vt are from each of the dictionaries; (C) Yt is artificially created with the target semantics (animal or photograph), but without any words from the target semantic dictionary; (D) ye is artificially
Table 5: Performance (%) of different evolutionary algorithms and average number of iterations to find the optimal solution.
CTB-DE
R-DE
S-GA
iteration]
46.47+37.1
57.35+43.62
15.41+11.52
METEORT
0.69620.209
0.538+0.264
0.327+0.254
BLEUT 0.65820.219
0.54620.218
0.27920.17
CLIP
0.95#0.291
0.87140.112
0.748+0.096
19

---

VIT:GP12
VIEGP12
Attend---iiie Attack
Mpw
630
060
Figure 10: Comparison of computation time for generating a single adversarial sample using different adversarial attack methods. The y-axis is measure of similarity between the generated text and the target text, with higher values indicating better target attack performance. The x-axis represents the computation time, and the shorter the time required to find stable solution, the better.
created with different semantics from both target semantics (animal and photograph), and without any words from either target semantic dictionary. The output texts of the adversarial images obtained from different word selection strategies are shown in Table 3.
The first row of Table 3 shows that strategy (A) can achieve strong targeted attack, making the output text very similar to the target text. This is because words in the same dictionary are close to each other in the feature space. Strategy (B) selects the words flying and air from dictionary animal in Figure 8 (a), and camera from dictionary photograph in Figure (b), to form the target text. The third row of Table shows that the output text and the target text V/ are not very similar. The output text only contains the word camera in dictionary (b). This is because the feature distance between the two dictionaries is large, even though they are both close to the input image and easy to search in the feature space. It is hard to optimize the target text y/ that contains words from both target semantic dictionaries. Strategy (C) randomly creates Y/ based on the animal and photograph semantics, without using any words from dictionary (a) and (b). For example, giraffe is an animal, but not in dictionary (a), and capture beautiful moment is related photograph, but not in dictionary (b). The output text and the target text are totally different, indicating a failed targeted attack. Strategy (D) randomly creates y/ with different semantics from both target semantics, and without any words from either target semantic dictionary. The targeted attack also fails. Therefore, we recommend selecting words from one target semantic dictionary for the target text ye, which will greatly improve the success rate of our method's 's targeted attack.
B.4 Comparison experiment on population size
We show convergence curves with the same target text but different population sizes NP observe how they affect the optimization iteration process of Attack. Figure 9 shows that when NP is 10 and 20, the best fitness values are 0.2 and 0.1, corresponding to CLIP scores of 0.8 and 0.9 for the output texts and target texts, respectively. When NP is larger than 40, the output text and the target text are completely consistent (CLIP score 1). This means that larger NP can find better solutions with fewer iterations [24, 46]. However, a larger NP also increases the computation time per iteration, as Table 4 shows. Moreover, as this variables per individual, a larger NP demands more hardware resources [47, 391. Considering all factors, we set the population size NP to 40.
large-scale optimization problem with 196608 decision
B.5 Comparison experiment on computation time
In Figure 6 of the main paper, we show the computational efficiency of two metrics, CLIP score and BLEU score. In this part, we will supplement the other two metrics, METEOR score and SPICE score. As shown in Figure 10, the computation time of the existing gray-box attack methods to find the optimal solution is still shorter than that of our black-box attack method. For example, the transfer approach [13] illustrated in Figure 10(a) produces an adversarial sample with METEOR score of 0.34 within a mere seconds, while the transfer+query approach [10] achieves a METEOR score of 0.49 in just 119 seconds. Conversely, our AAA method requires 179 seconds to generate an adversarial sample with a superior METEOR score of 0.75. Because our method more practical and performs better, the additional computation time is acceptable.
20

---

image
attention heatmap
adversarial image
optimization curve
Text:
aa phoso of pain of scitsons
Output
a close up of bair of scissors
METEOR-0.33 BELW-O 71 CLIP-0.96
Text:
Iilel slar sli in roomt will mai on his phone
Text: a mar standing in room with ghone
METEOR-O.ER 0088 BELU=O 71 CLIP-0.96
Target Text:
group of birds standing the
a group of birds are standing in the weter
METEOR-1.3 BELU-10 CUP-11D
Target Text:
is loking sellie iniioor
Output Text:
is taking picture of horzelf a miirrrr
Target Test:
train comnng down the tracks
Output Text:
coming down the tracks neat a forest
CUP-10
Target
Output Text:
taking
METEOR-O BELU-0.82 CUP-0.97
Target Text:
photo of bunch of fowers sunny day
Dutput Text:
large llowers lange group
METEOR-0.34 BELU-3.35 CLIP-0.88
Figure 11: Attention heatmaps, optimization convergence curves, target text, output text and attack performance for more adversarial samples.
B.6 Comparison experiment of optimization algorithms
We compared different optimization strategies Attack: CurrentToBest Differential Evolution (CTB- DE) [39], Rand Differential Evolution (R-DE) [47], and Stud Genetic Algorithm (S-GA) [48]. Table shows the genetic algorithm needs the fewest iterations, but easily gets stuck in local optima, leading to poor attack performance. Differential evolution needs more iterations but finds better solutions. Also, the CurrentToBBes mutation does better and faster than the random mutation. So adopted the TorrentToBest differential evolution strategy in Attack.
21

---

B.7 Visualization of more adversarial samples
We presented attention heatmaps A, optimization convergence curves, target text ye, and output text for more adversarial samples, as shown in Figure 11.
C Discussion
C.1 Limitation
Our work represents the first black-box targeted attack image-to-text models, with the core idea utilizing evolutionary algorithms to solve a large-scale optimization problem. The drawbacks of evolutionary algorithms, which are also the limitations of our work, include: (1) Low optimization efficiency. Gradient-based algorithms use the gradient information of the objective function, which powerful guide regarding the optimization direction. Evolutionary algorithms do not directly gradient information but search through random mutation and crossover operations. Compared to gradient optimization algorithms, evolutionary algorithms require more iterations to find the optimal solution. (2) High number of queries. Each individual in the population requires access to the target model in every iteration, and the service provider of the image-to-text target model can simply set limit on the number of accesses to defend against our attack.
C.2 Future work
Our black-box targeted attack framework Ask, Attend, Attack image-to-text models employs classic evolutionary algorithms. In our future work, we will explore how our framework AAA can be combined with the current state-of-the-ar (SOTA) evolutionary algorithms, which have the fastest convergence efficiency, to mitigate the limitations mentioned above.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope. We have clearly stated our novel methodology and its implications in the abstract and introduction, and these are further elaborated upon and validated in the main body of the paper.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Our experiments in Section 4.2 and our conclusions in Section 5 describe the limitations of our approach.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
23

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: Our paper presents a framework for adversarial attacks based on evolutionary algorithms. The underlying techniques employed, such as differential evolution and similar- ity computation, are well-establisher with their theoretical foundations extensively proven in existing literature.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Thcorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide experimental Settings in section 4.1, we use public data sets, and we provide pseudocode and source code in supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
24

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have packaged our source code and data into a zip file and uploaded it to the system supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
public/ge htt///gddss//ee /ttg//ee//ooeeuumiiseeeesseseses CoodeSubmissionPoo oonyyicc)))) for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /nips. cc/ wwwpppcccc/aceececcceeceeeetetle wwwpcc/gggg////eeeieeeeie /ttpgd/ee//ooeuuuiiieeeeee www.eSSummmiiooooiiiioooo
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: We provide experimental Settings in section 4.1, and we provide pseudocode and source code in supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Docs the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
In our performance comparison experiments, we report the results of 10 repeated experiments mean + standard deviation. This provides measure of the variability of the results, and gives a better understanding the range of performance we can expect from our method.
Justification:
25

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative етгог rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide the GPU models in Section 4.1 and a comparison experiment on computation time in Section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http///eeuripp.peeee
Answer: [Yes]
Justification: Our research adheres to the NeurIPS Code of Ethics in all respects. We have ensured fairness and transparency in all our experiments, respected the rights of all participants, and our research does not violate any laws or regulations.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
26

---

Question:
Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our research may have potential negative societal impacts. Malicious actors could potentially use our black-box adversarial attack techniques to generate adversarial samples deceive commercial image-to-text models, thereby damaging the reputation of the models. An effective solution could be to set a threshold for the number accesses.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Answer: [Yes]
The original intention of our proposed adversarial attack technique is to provide a baseline for researchers in the robustness of deep neural networks, but it could still potentially be misused by malicious actors. To prevent these actors from using our method attack commercial image-to-text models, we have proposed defensive measure, which is to set limit on the number of accesses.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
27

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We used open data sets and correctly referenced the source code papers.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have introduced new assets in the form of source code in our paper. This code is well documented, with comments explaining the functionality of different sections. We have included this code in the supplementary material zip file that we have submitted alongside our paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
28

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "d8NB7VXEsY", "reviewer": "Reviewer_xQiz", "summary": "This paper proposes a new adversarial attack, termed AAA (ask, attend, attack) attack towards image-to-text models. In the Ask stage, the attacker iteratively generates candidates to generate individuals closer to the target semantics in the feature space of the target model. During the Attend Stage, the attacker utilizes a surrogate model to generate a heatmap to guide the attack. Eventually, in the attack stage an evolutionary algorithm is adopted to search for individuals with closer feature distance between the output text and the target text.", "questions": "* In line 262, \"AAA (w/o Ask) means the target text is not from the target semantic dictionary, but random words\". Why the author didn't use the target semantics as the target text? It is confusing that AAA (w/o Ask) adopts random words as the target text while achieving comparable performance to gray-box attacks.", "limitations": "See weaknesses and questions.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* This paper presents the first black-box targeted adversarial attack towards image-to-text models.\n* The proposed attack is shown to be even more effective than existing gray-box attacks.\n* Comprehensive evaluations are conducted including detailed ablation studies on each component.", "weaknesses": "* The presentation of Section 3.3 is confusing. Steps 1,2 and 3 are randomly shuffling the input images. It is not well-motivated or clearly explained why the attacker needs to generate the perturbed images following these steps.\n* The Ask stage seems to be \"optimizing\" the target text so that the target text would be naturally closer to the input image, which, from my perspective, makes the attack less \"targeted\" as it is actually easing the attack. As shown in Table 1, the Ask stage heavily contributes to the effectiveness of the AAA attack.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "To Weakness 1：\n\n\tSection 3.3 (Ask stage) aims to obtain prior knowledge for searching the target text. Prior knowledge shortens the search path from image to target text as much as possible, improving the search efficiency of Attack stage (line 137).\n\n\tSpecifically, we perform a random search within a small hypersphere centered on the image $x$ with a radius $\\eta$, ensuring all potential words are close to the image (Equations 2, 3, and 4). Then, we compile a target semantic dictionary from the words related to the attacker’s desired semantics (Equations 5, 6, and 7). \n\nTo Weakness 2：\n\n\tI agree with your opinion that Ask stage makes the attack easier, which is exactly our goal and is reasonable：\n\n\t1) Technically Reasonable: Prior knowledge can reduce the search difficulty of evolutionary algorithm. In addition, adversarial attacks often rely on prior information. For example, existing image-to-text gray-box attacks depend on the prior information of gradients, and black-box attacks for classification tasks rely on the prior information of similarity between surrogated and target models, while our prior information is the target semantic dictionary. It is worth noting that our prior information is more easily obtainable in reality, while their prior knowledge is difficult to acquire.\n\n\t2) Fair Comparison: We applied the same prior knowledge (target semantic dictionary) to both our method and the existing methods, meaning that all approaches in the comparative experiments utilized the identical target text, thereby guaranteeing a fair comparison.\n\n\t3) Reasonable use scenarios: Attackers typically need to meet their target semantic requirements without fixating on specific words. For example, if an attacker wants to map an illegal image to a legal text related to “dogs”, and the target semantic dictionary (prior knowledge) includes the words “puppy” and “jogging” but not “little dog” and “running”, the attacker can choose the sentence “The puppy is jogging” as the target text, avoiding “The little dog is running”. The reason is that although both sentences meet the attacker’s needs, the former is derived from the dictionary and is therefore easier to search for.\n\nTo Question 1：\n\n\tUsing target semantics as the target text for AAA (w/o Ask) is unfair. In our paper, AAA (w/o Ask) randomly creates a complete and coherent sentence from all English words as the target text, which is fair.\n\n\tThe reason why the former is unfair: The target text must be a sentence, not a word, because the output text of an image-to-text model is always a complete sentence. If the target semantics (a single word, such as \"animal\" or \"photograph\") is used as the target text to calculate the similarity with the output text (a complete sentence), it is difficult to achieve a high similarity score.\n\n\tThe reason why the latter is fair: The sole difference between AAA and AAA (w/o Ask) lies in the range of words for the target text. For AAA, the range is \"words within the target semantics dictionary,\" whereas for AAA (w/o Ask), it is \"all English words.\" Our experimental setup for AAA (w/o Ask) fairly demonstrates the impact of word selection range on the attack. Additional comparative experiments with various word selection ranges are conducted in Appendix B.3, which can provide a better understanding.\n\n\tIt is important to emphasize that when comparing our method AAA with existing gray-box methods, the target texts are identical sentences, ensuring the fairness of the experiment.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SMurX6yIMw", "reviewer": "Reviewer_8Kf2", "summary": "The authors tackle the challenging problem of adversarial attacks on Image-to-Text Models, focusing specifically on the black-box attack scenario where an attacker has no access to the internal workings of the model, only its output. To address this challenge, they propose a novel framework called Ask, Attend, Attack (AAA), which leverages evolutionary algorithms to mount targeted attacks. The AAA framework consists of three stages: Ask, where the attacker generates target texts that satisfy specific semantic constraints; Attend, where crucial regions of the image are identified without access to the image encoder; and Attack, where an evolutionary algorithm is used to attack these identified regions with semantically related attacks that match the target texts from the Ask stage. This approach cleverly avoids the issue of semantic mismatch encountered in previous gray-box attacks. The authors demonstrate the effectiveness of their proposed framework through experiments on two distinct models.", "questions": "1. I see that the authors chose the l_1 adversarial attack domain. Can AAA be generalized for the more common l_{inf} attack?\n2. How are the different parameter values chosen?\n3. In Table 1, the epsilon values chosen are 25 and 15. These seem rather big when compared to image classification domain. How do the results look when epsilon is made even smaller?", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The authors aptly acknowledge the complexity of the domain they tackle in this paper, which underscores the significance and novelty of their contributions.\n2. The authors provide a clear and comprehensive description of the three stages comprising their proposed AAA framework. Notably, each stage is carefully designed to address the semantic mismatch issue that has been previously observed in related research, offering a thoughtful and intentional approach to tackling this challenge.\n3. The experimental outcomes presented in this manuscript are encouraging, as the proposed AAA framework demonstrates strong performance across the two models examined. Additionally, I commend the authors for including an ablation study, which provides valuable insights into the effectiveness of individual components and sheds light on their contributions to the overall framework's success.", "weaknesses": "1. While I appreciate the effort invested in this paper, I do find some aspects lacking in terms of clear motivation. For instance, the example provided to justify the attack setting, where an attacker seeks to elicit political slogans and hate speech by introducing imperceptible perturbations to the image, seems somewhat weak compared to more established examples in the domain of image classification. Additionally, I would have liked a clearer explanation for the specific population definition chosen for the Ask stage; without further justification, it's difficult to fully understand the reasoning behind this design choice.\n2. The authors effectively emphasize the practicality of the attack setting, which is indeed a significant strength of this paper. However, I do have some reservations about the feasibility of the proposed solution. Specifically, I'm concerned that the attacker would need to interact with the victim model numerous times to generate the target image, which seems impractical and may not be a realistic scenario in many real-world applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "To Weakness 1：\n\n\tTo underscore the significance of research into image-to-text black-box targeted attacks, we will add an example of societal harm and highlight a societal benefit: 1) Harm Example: Social media companies use image-to-text AI for content moderation of user-uploaded images. Black-box targeted attacks on image-to-text AI could enable attackers to alter the semantic content of images from illegal to legal, thus bypassing the moderation process and uploading the illegal images to social media. 2) Benefit: The introduction of image-to-text black-box targeted attacks can alert AI service providers to vulnerabilities in their AI products, providing a benchmark to improve the security of their AI products.\n\n\tIn Section 3.3, the Ask stage’s population consists of randomly perturbed images, where each perturbed image is a chromosome, and each pixel is a gene. The goal of the Ask stage is to obtain prior knowledge (target semantic dictionary) to reduce the search difficulty of evolutionary algorithm (Attack stage). Target semantic dictionary contains words that are 1) close to the image in the input space, and 2) related to the attacker’s specified semantics. Equations 2, 3, and 4 show a random search within a small hypersphere centered on the input image $x$ with a radius $\\eta$, ensuring all potential words are close to the input image. Equations 5, 6, and 7 store words related to the attacker’s specified semantics from the output text of each chromosome in the target semantic dictionary, helping the attacker generate a target text that meets their requirements (target semantics) and can be efficiently searched.\n\nTo Weakness 2：\n\n\tIn Appendix C.1, we discuss the limitations of our framework in real-world applications, specifically the need for numerous interactions with the target model. This is due to our use of a basic differential evolution algorithm, which requires many iterations to converge to the optimal solution. A potential improvement (Appendix C.2) is to combine our framework with the current state-of-the-art evolutionary algorithms, significantly reducing the number of interactions needed.\n\nTo Question 1：\n\n\tIn fact, our framework uses the $l_{inf}$ attack domain, not the $l_1$ domain. As shown in Equation 10, the value range for the i-th variable (pixel) of the clean image $x$ is [$-A(i) \\cdot \\eta$, $A(i) \\cdot \\eta$], where $A(i)$ is the contribution weight (0 to 1) and $\\eta$ is the perturbation size parameter. This means each pixel has its own perturbation limit ($l_{inf}$ attack domain), adjusted by its contribution weight. For example, less important areas (like the background) have reduced perturbation limits. Since each pixel has a different perturbation limit, we represent the constraint in Equation 1 using the $l_1$ attack domain.\n\nTo Question 2：\n\n\tThe parameter values are determined through ablation studies. For example, Appendix B.2 determines the target semantic strategy, Appendix B.3 determines the word selection strategy, Appendix B.4 determines the population size, Appendix B.6 determines the evolutionary algorithm, Figure 4 in the main text determines the surrogate model, and Figure 5 in the main text determines the perturbation size, etc.\n\nTo Question 3：\n\n\tIn Section 4.2 (Qualitative Experiment of Different Perturbation Sizes), we have compared attacks with epsilon values of 25, 15, 10, and 5 (Figure 5). Our conclusions are:\n\n\t1) As perturbation size decreases, both our method and existing methods become more stealthy, but their attack performance decreases.\n\n\t2) As perturbation size increases, our black-box method achieves nearly 100% target attack performance, while existing gray-box methods have lower performance due to semantic loss issues.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pDEzRLFcod", "reviewer": "Reviewer_83Kr", "summary": "The paper focuses on the vulnerability of image-to-text models to adversarial attacks, particularly in a decision-based black-box targeted attack scenario. The authors design a three-stage attack process: (i) Ask: Guides attackers to create target texts that fulfill specific semantic requirements. (ii) Identifies crucial regions of the image for the attack, reducing the attack's search space. (iii) Utilizes an evolutionary algorithm to attack these critical regions, aligning with the semantics from the Ask stage to achieve targeted attacks without semantic loss. Tests on transformer-based and CNN+RNN-based models demonstrate the success of the proposed AAA method in performing effective targeted attacks.", "questions": "1. How can authors guarantee target words that should be closer to the input space as many rounds of mutation proceed?\n\n2. Can authors provide more explanation about why selecting a visual model is good as the surrogate model? Will the model architecture or parameters have impacts on the feature map? I guess for the common datasets such as the ImageNet and common architectures, these models could be similar. But what if different model architectures were trained on a more diverse dataset? As a black-box attack, it is hard to guarantee the consistency of feature map generation among different models.\n\n3. Are the mean and median of heatmap A is around [0.3, 0.4] for all types of data?", "limitations": "The authors mentioned the limitation of a high number of queries and low optimization efficiency.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. A novel decision-based black-box targeted attack approach is proposed.\n\n2. The idea of reducing the search space is interesting.", "weaknesses": "1. The technical contribution is limited. Many concepts and ideas are directly adopted from existing work. There are many steps that use existing models such as CLIP models, which only work on some benchmark datasets.\n\n2. The writing quality needs to be enhanced. For instance, the title should be \"An Effective Decision-Based Black-Box Targeted Attack for Image-to-Text Models\". Any typos in $S_{sem}$ and $S_{seg}? Some technical details are unclear. For example, how can authors guarantee target words that should be closer to the input space as many rounds of mutation proceed?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "To Weakness 1：\n\n\tOur technical contribution lies in carefully crafting specific designs to improve search efficiency and make our framework applicable in more difficult scenarios. Each design and its technical contribution is summarized as follows.\n\n\t1) Improving search efficiency from different perspectives:\n\n\ta) Reducing the search difficulty: Designing a target semantic dictionary as prior knowledge shortens the search path from image to target text as much as possible, improving the search efficiency of Attack stage (line 137), \n\n\tb) Reducing the computation time: The Attack stage requires calculating the cosine similarity matrix of the entire population’s text in each iteration, which is time-consuming. The CLIP model’s text encoder can process large amounts of text in parallel. Therefore, we disable the visual encoder of CLIP and use the text encoder to compute the cosine similarity matrix between the target text and the output text of all chromosomes (Equation 13), speeding up the search.\n\n\tc) Reducing the search space: The large number of decision variables (pixels) leads to a vast search space and low search efficiency. We combine the Grad-CAM formula with decision variables to design Equation 10, reducing the range of decision variables in unimportant areas (e.g., background), significantly reducing the search space.\n\n\t2) Dealing with different difficult scenarios: \n\n\ta) Black-box：Grad-CAM requires gradient information and is not suitable for black-box scenarios, nor can it handle the image-to-text task where the number of output text categories is infinite. Based on previous research (cited [28], CVPR 2021) and our experiments (Figure 4), we designed a surrogate model strategy to generate attention maps in black-box scenarios. And we designed a CLIP-based mapping formula (Equation 9) to map the output text to the category set of the surrogate model, enabling the generation of attention maps for image-to-text tasks with unlimited output categories.\n\n\tb) Perturbation imperceptibility: The perturbations of adversarial examples need to be imperceptible to the human eye. The Grad-CAM-based Equation 10 we designed can also enhance the stealthiness of adversarial examples. This is because a smaller search range for decision variables means smaller perturbations at each pixel.\n\nTo Weakness 2：\n\n\tThank you for pointing out these typos. The term $S_{seg}$ in line 167 is a typo and should be corrected to $S_{sem}$ . We have checked the rest of the paper and found no other typos. We will correct this in the final version. Your question about target words is addressed in Question 1 below.\n\nTo Question 1：\n\n\tThe mutation process in the Ask stage does not change the position of any words in the input space. Instead, it filters words related to the attacker’s specified semantics within a small hypersphere centered on the image (Equation 2). All target words are within a distance of radius $\\eta$ from the image, ensuring that the target text (created based on the target semantic dictionary) is also positioned close to the image. This effectively reduces the search difficulty and improves the search efficiency in the Attack stage (Table 1 and Appendix B.3).\n\nTo Question 2：\n\n\tDNN with different architecture and parameters exhibit similar attention patterns for the same image, as concluded in previous adversarial attack work (cited [28], Figure 4, CVPR2021) and validated by our qualitative and quantitative experiments on diverse CNN and Transformer models (Figure 4). The role of attention maps in our framework is to reduce the search space of unimportant regions (e.g., background), enhancing the stealthiness of adversarial images and reducing the search difficulty in the Attack stage. Even if different surrogate models produce slightly different attention maps, the cost is limited to additional computation in unimportant regions. Since adversarial attacks are not time-sensitive tasks, this cost is acceptable.\n\nTo Question 3：\n\n \tYes, this is a pattern we observed through statistical analysis of the heatmap value distribution. We have added the numerical distribution of all heatmaps as well as the median and mean values to the supplementary file in the rebuttal system.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers, thank you for recognizing our work as novel, important, and experimentally adequate. And your comments have greatly improved the quality and clarity of our manuscript.  We will address your concerns one by one.

Meta-review: All reviewers have provided positive scores for this submission, highlighting its strengths in novelty and extensive experiments. Given the unanimous positive feedback and the recognition of its contribution to the area, the AC carefully reviewed the paper and concurred with the reviewers' assessments, therefore supporting the decision to accept this submission.
Decision: Accept (poster)