SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Examples of how the datasets are utilized. In our experiments, we use the ImageNet-1K [20] validation images as the clean images  $(x_{\mathrm{cle}})$  to be attacked, and we randomly select a caption from MS-COCO [44] captions as each clean image's targeted text  $c_{\mathrm{tar}}$ . Therefore, we ensure that each clean image and its randomly selected targeted text are irrelevant. To implement MF-ii, we use Stable Diffusion [72] to generate the targeted images (i.e.,  $h_\xi(c_{\mathrm{tar}})$  in the main paper). Here, we provide several examples of <clean image - targeted text - targeted image> pairs used in our experiments (e.g., Table 1 and Table 2 in the main paper), as shown in Figure 9.
Clean image (From ImageNet-1K)
"Two giraffes standing near each other in the zoo."
Targeted Image rated by Stable Diff
Figure 9: An illustration of the dataset used in our MF-ii attack against large VLMs. By utilizing the text-to-image generation capability of Stable Diffusion, we are able to generate high-quality and fidelity targeted images given any type of targeted text, thereby increasing the attacking flexibility.
"A teen riding a skateboard next to some stairs."
"A large dirty yellow truck, parked in a yard."
"A lamb is eating food in the trough."
"A sandwich is sitting on a black plate."
Text-to-image models for targeted image generation. It is natural to consider the real images from MS-COCO as the targeted images corresponding to the targeted text (caption) in our attack methods. Nevertheless, we emphasize that in our experiments, we expect to examine the targeted text  $c_{\mathrm{tar}}$  in a flexible design space, where, for instance, the adversary may define  $c_{\mathrm{tar}}$  adaptively and may not be limited to a specific dataset. Therefore, given any targeted text  $c_{\mathrm{tar}}$ , we adopt Stable Diffusion [72], Midjourney [51] and DALL-E [67, 68] as text-to-image models  $h_\xi$  to generate the targeted image  $h_\xi(c_{\mathrm{tar}})$ , laying the foundation for a more flexible adversarial attack framework. In the meantime, we observe empirically that (1) using targeted texts and the corresponding (real) targeted images from MS-COCO, and (2) using targeted texts and the corresponding generated targeted images have comparable qualitative and quantitative performance.
Hyperparameters. Here, we discuss the additional setups and hyperparameters applied in our experiments. By default, we set  $\epsilon = 8$  and the pixel value of all images is clamped to [0, 255]. For each PGD attacking step, we set the step size as 1, which means we change the pixel value by 1 (for each pixel) at each step for crafting adversarial images. The adversarial perturbation is initialized as  $\Delta = 0$ . Nonetheless, we note that initializing  $\Delta \sim \mathcal{N}(\mathbf{0},\mathbf{I})$  yields comparable results. For query-based attacking strategy (i.e., MF- tt), we set  $\sigma = 8$  and  $\delta \sim \mathcal{N}(\mathbf{0},\mathbf{I})$  to construct randomly perturbed images for querying black-box responses. After the attack, the adversarial images are saved in PNG format to avoid any compression/loss that could result in performance degradation.
Attacking algorithm. In addition to the illustration in the main paper (see Figure 4), we present an algorithmic format for our proposed adversarial attack against large VLMs here. We clarify that
we slightly abuse the notations by representing both the variable and the optimal solution of the adversarial attack with  $x_{\mathrm{adv}}$ . For simplicity, we omit the input  $c_{\mathrm{in}}$  for the victim model (see Section 3.1). All other hyperparameters and notations are consistent with the main paper or this appendix. Because we see in Table 2 that MF-it has poor transferability on large VLMs, we use MF-ii + MF- tt here, as shown in Figure 4. In Algorithm 1, we summarize the proposed method.
Algorithm 1 Adversarial attack against large VLMs (Figure 4)
1: Input: Clean image  $x_{\mathrm{cle}}$ , a pretrained substitute model  $f_{\phi}$  (e.g., a ViT-B/32 or ViT-L/14 visual encoder of CLIP), a pretrained victim model  $p_{\theta}$  (e.g., Unidiffuser), a targeted text  $c_{\mathrm{tar}}$ , a pretrained text-to-image generator  $h_{\xi}$  (e.g., Stable Diffusion), a targeted image  $h_{\xi}(c_{\mathrm{tar}})$ .  
2: Init: Number of steps  $s_1$  for MF-ii, number of steps  $s_2$  for MF-tt, number of queries  $N$  in each step for MF- tt,  $\Delta = 0$ ,  $\delta \sim \mathcal{N}(0,\mathbf{I})$ ,  $\sigma = 8$ ,  $\epsilon = 8$ ,  $x_{\mathrm{cle}}$ . requires_grad() = False.  
# MF-ii  
3: for  $i = 1; i \leq s_1; i + +$  do  
4:  $x_{\mathrm{adv}} = \text{clamp}(x_{\mathrm{cle}} + \Delta, \min = 0, \max = 255)$   
5: Compute normalized embedding of  $h_{\xi}(c_{\mathrm{tar}}): e_1 = f_{\phi}(h_{\xi}(c_{\mathrm{tar}})) / f_{\phi}(h_{\xi}(c_{\mathrm{tar}})).$  norm()  
6: Compute normalized embedding of  $x_{\mathrm{adv}}: e_2 = f_{\phi}(x_{\mathrm{adv}}) / f_{\phi}(x_{\mathrm{adv}}).$  norm()  
7: Compute embedding similarity: sim =  $e_1^\top e_2$   
8:Backpropagate the gradient: grad = sim/backward()  
9: Update  $\Delta = \text{clamp}(\Delta + \text{grad}. \text{sign}), \min = -\epsilon, \max = \epsilon)$   
10: end for  
# MF- tt  
11: Init:  $x_{\mathrm{adv}} = x_{\mathrm{cle}} + \Delta$   
12: for  $j = 1; j \leq s_2; j + +$  do  
13: Obtain generated output of perturbed images:  $\{p_{\theta}(x_{\mathrm{adv}} + \sigma \delta_n)\}_{n=1}^N$   
14: Obtain generated output of adversarial images:  $p_{\theta}(x_{\mathrm{adv}})$   
15: Estimate the gradient (Eq. (4)): pseudo-grad = RGF $(c_{\mathrm{tar}}, p_{\theta}(x_{\mathrm{adv}}), \{p_{\theta}(x_{\mathrm{adv}} + \sigma \delta_n)\}_{n=1}^N)$   
16: Update  $\Delta = \text{clamp}(\Delta + \text{pseudo-grad}. \text{sign}), \min = -\epsilon, \max = \epsilon)$   
17:  $x_{\mathrm{adv}} = \text{clamp}(x_{\mathrm{cle}} + \Delta, \min = 0, \max = 255)$   
18: end for  
19: Output: The queried captions and the adversarial image  $x_{\mathrm{adv}}$
Amount of computation. The amount of computation consumed in this work is reported in Table 3, in accordance with NeurIPS guidelines. We include the compute amount for each experiment as well as the  $\mathrm{CO}_{2}$  emission (in kg). In practice, our experiments can be run on a single GPU, so the computational demand of our work is low.
In our main paper, we demonstrated sufficient experiment results using six cutting-edge large VLMs on various datasets and setups. In this section, we present additional results, visualization, and analyses to supplement the findings in our main paper.
B.1 Image captioning task by BLIP-2
In Figure 10, we provide additional targeted response generation by BLIP-2 [42]. We observe that our crafted adversarial examples can cause BLIP-2 to generate text that is sufficiently similar to the predefined targeted text, demonstrating the effectiveness of our method. For example, in Figure 10, when we set the targeted text as “A computer from the 90s in the style of vaporwave”, the pretrained BLIP-2 model will generate the response “A cartoon drawn on the side of an old computer”, whereas the content of clean image appears to be “A field with yellow flowers and a sky full of clouds”. Another example could be when the content of the clean image is “A cute girl sitting on steps playing with her bubbles”, the generated response on the adversarial examples is “A stuffed white mushroom sitting next to leaves”, which resembles the predefined targeted text “A photo of a mushroom growing from the earth”.
Table 3: The GPU hours consumed for the experiments conducted to obtain the reported values.  $\mathrm{CO}_{2}$  emission values are computed using https://mlco2.github.io/impact [39]. Note that our experiments primarily utilize pretrained models, including the surrogate models, text-to-image generation models, and the victim models for adversarial attack. As a result, our computational requirements are not demanding, making it feasible for individual practitioners to reproduce our results.
Experiment name	Hardware platform	GPU hours	Carbon emitted in kg
Table 1 (Repeated 3 times)	NVIDIA A100 PCIe (40GB)	126	9.45
Table 2 (Repeated 3 times)	2448	183.6
Figure 1	NVIDIA A100 PCIe (40GB)	12	0.9
Figure 2	18	1.35
Figure 3	36	2.7
Figure 5	12	0.9
Figure 6	12	0.9
Figure 7	24	1.8
Hyperparameter Tuning	NVIDIA A100 PCIe (40GB)	241	18.07
Analysis	120	9.0
Appendix	480	36.0
Total	-	3529	264.67
B.2 Joint generation task by UniDiffuser
Unidiffuser [5] models the joint generation across multiple modalities, such as text-to-image or image-to-text generation. In Figure 11, we show additional results for the joint generation task implemented by Unidiffuser. As can be seen, our crafted adversarial examples elicit the targeted response in various generation paradigms. For example, the clean image could be generated conditioned on the text description "A pencil drawing of a cool sports car", and the crafted adversarial example results in the generated response "A close up view of a hamburger with lettuce and cheese" that resembles the targeted text. As a result, Unidiffuser generates a hamburger image in turn that is completely different from the semantic meanings of the original text description.
B.3 Visual question-answering task by MiniGPT-4 and LLaVA
The multi-round vision question-answering (VQA) task implemented by MiniGPT-4 is demonstrated in the main paper. Figures 12 and 13 show additional results from both MiniGPT-4 [109] and LLaVA [46] on the VQA task. In all multi-round conversations, we show that by modifying the minimal perturbation budget (e.g.,  $\epsilon = 8$ ), MiniGPT-4 and LLaVA generate responses that are semantically similar to the predefined targeted text. For example, in Figure 12, the monkey worrier acting as Jedi is recognized as an astronaut riding a horse in space, which is close to the targeted text "An astronaut riding a horse in the sky". Similar observations can be found in Figure 13.
B.4 Interpretability of the attacking mechanism against large VLMs
GradCAM [75] is used in the main paper to interpret the targeted response generation. We present additional visualization results to help understand the mechanism that deceives these large VLMs; the results are shown in Figure 14. Similarly to our findings in the main paper, we show that, when compared to the original clean image, (a) our crafted adversarial image can lead to targeted response generation with different semantic meanings of the clean image's text description; (b) when the input question is related to the content of the clean image, such as 'How many people in this iamage?'', GradCAM will highlight the corresponding area in the clean image, while ignoring the same area in the adversarial image; (c) when the input question is related to the targeted text, such
as "where is the corn cob?", GradCAM will highlight the area of the adversarial image that is similar to the targeted image. More results can be found in Figure 14.
C Additional discussion
In this section, we clarify on the sensitivity when we perturb adversarial examples, and failure cases to help better understand the limitations of our attacks.
C.1 Sensitivity of adversarial examples to random perturbation
To evaluate the sensitivity of our crafted adversarial examples, we add random Gaussian noises with zero mean and standard deviation  $\sigma_{\mathrm{noise}}$  to the obtained adversarial images  $x_{\mathrm{adv}}$ , and then feed in the perturbed adversarial examples for response generation. The results are shown in Figure 15. We observe that our adversarial examples are reasonably insensitive to this type of perturbation, and we also make the following observation: as the amplitude (i.e.,  $\sigma_{\mathrm{noise}}$ ) of the Gaussian noises added to  $x_{\mathrm{adv}}$  increase, the effectiveness of our learnt adversarial perturbation diminishes and the targeted responses revert to the original. For instance, in Figure 15, when  $\sigma_{\mathrm{noise}} = 0$ , we can obtain the generated targeted response "A red and black bird sitting on top of a tree branch" that resembles the targeted text; when  $\sigma_{\mathrm{noise}} = 0.025$ , it changes to "A red and black bird is sitting on top of a sunflower"; and finally the response degrades to "A large painting of three sunflowers in a field". Additional results are shown in Figure 15.
C.2 Failure cases
While we have demonstrated convincing results of our method in the main paper and in this appendix, we note that the adversarial attack success rate for these large VLMs is not one hundred percent. Here, we present a few failure cases discovered during our experiments, leaving them for future work to improve performance. Specifics are shown in Figure 16.
Figure 10: Additional results of image captioning task implemented by BLIP-2.
UniDiffuser: Joint generation
Figure 11: Additional results for joint generation task implemented by Unidiffuser.
Figure 12: Additional results of VQA task implemented by MiniGPT-4.
Figure 13: Additional results of VQA task implemented by LLaVA.
$x_{\mathrm{adv}}$
$h_{\xi}(c_{\mathrm{tar}})$
GradCAM of  $x_{\mathrm{adv}}$
(a)
Figure 14: Visually interpreting our attacking mechanism. To better understand the mechanism by which our adversarial examples deceive large VLMs, we provide additional visual interpretation results (via GradCAM [75]) as supplements to Figure 7 of the main paper. Similar to our previous findings, we demonstrate: (a) An example of  $x_{\mathrm{cle}}$ ,  $x_{\mathrm{adv}}$ , and  $h_\xi(c_{\mathrm{tar}})$ , along with the responses they generate; (b) GradCAM visualization when the input question  $c_{\mathrm{in}}$  is related to the clean image. (c) GradCAM will highlight regions similar to those of  $x_{\mathrm{adv}}$  if we provide the targeted text (or other texts related to  $c_{\mathrm{tar}}$ ) as the question.
"A flower is laying in the ground. I'm sure many other creatures."
An odd mushroom growing from a single cell
GradCAM of  $h_{\xi}(c_{\mathrm{tar}})$
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0$
"A red and black bird sitting on top of a tree branch."
Example-1
Target: "A black and red bird sitting on top of a branch."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.001$
"A yellow and red bird sitting on top of a tree."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.01$
"A yellow and red parrot sitting on top of a palm tree."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.025$
"A red and black bird is sitting on top of a sunflower."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.05$
"A large painting of three sunflowers in a field."
Example-2
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0$
"A close up view of a hamburger with lettuce and cheese."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.001$
"A hamburger with lettuce and cheese.
Target: "A hamburger with lettuce and cheese.
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.01$
"A close up shot of a hamburger with lettuce in it."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.025$
"A drawing of a burger and a car on a piece of paper."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.05$
"A pencil drawing of a sports car."
Example-3
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0$
"A colorful panoramic view of Ithaca Mountains."
Target: "The view from the top of a hill overlooking the mountains."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.001$
"A colorful deer panoramic view of the Andes Mountains."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.01$
"A painting of colorful bears and mountains in the background."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.025$
"A painting of a cat at a valley and mountains in the background."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.05$
"A painting of a fox looking up at the sky."
Example-4
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0$
"A small white dog sitting in the grass near a stream in Autumn."
Figure 15: Sensitivity of adversarial examples to Gaussian noises. Our adversarial examples are found to be relatively insensitive to post-processing perturbations like Gaussian noises. Alternatively, interesting observations emerge when gradually increasing the standard deviation  $\sigma_{\mathrm{noise}}$  of the Gaussian noises added to  $x_{\mathrm{adv}}$ , where the effectiveness of our learned adversarial examples becomes marginal and the targeted responses (in green) gradually degrade to the original, correct response (in black). In addition, we note that an intermediate state exists in which the generated response is a combination of the targeted text and the original generated response (in red).
Target: "A small white dog sitting on the ground in autumn leaves."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.001$
"A small white dog sitting in the grass near a stream."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.01$
"A colorful dog sitting in the woods with autumn."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.025$
"An oil painting of a Terrier dog on a bridge."
$x_{\mathrm{adv}}:\sigma_{\mathrm{noise}} = 0.05$
"An oil painting of a bridge over a river."
Example-1
$x_{\mathrm{cle}}$
"What is the content of this image?"
The image features a small, adorable black and gray dog sitting on a brick walkway or sidewalk. The dog appears to be in a relaxed position, panting and looking up at the camera.
$\pmb{x}_{\mathrm{adv}}$
The image features a collage of photos of various dogs. Some of the dogs are close up, while others are in the background. There is a prominent black dog in the scene with its tongue sticking out.
Example-2
Target:A black and white terrier looks up at the camera
"How do you think of this image?"
The image is an artistic illustration of a goldfish swimming in a dark, rippling pool with a dark background. The fish is the main focal point of the image, and it appears to be captured in an artistic, almost cartoonish style.
Figure 16: Failure cases found in our experiments. The generated adversarial image responses appear to be a state in between the text description of the clean image and the predefined targeted text. In this figure, we use LLaVA [46] as the conversation platform, but similar observations can be made with other large VLMs. On the other hand, we discovered that increasing the steps for adversarial attack (we set 100 in main experiments) could effectively address this issue (note that the perturbation budget remains unchanged, e.g.,  $\epsilon = 8$ ).
The image features a small white and black fish, possibly a sea or tropical fish, swimming in a large tank. The fish is the main focus of the image, occupying a significant portion of the frame.