White-box Multimodal Jailbreaks Against Large Vision-Language Models

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-06 07:57:38

Abstract: Recent advancements in Large Vision-Language Models (VLMs) have underscored their superiority in various multimodal tasks. However, the adversarial robustness of VLMs has not been fully explored. Existing methodologies mainly assess robustness through unimodal adversarial attacks that perturb images, while assuming inherent resilience against text-based attacks. In contrast, our methodology adopts a comprehensive strategy that jointly attacks both text and image modalities to exploit a broader spectrum of vulnerability within VLMs. Furthermore, we propose a dual optimization objective aimed at guiding the model to generate affirmative responses with high toxicity. Specifically, we begin by optimizing an adversarial image prefix from random noise to generate diverse harmful responses in the absence of text input, thus imbuing the image with toxic semantics. Subsequently, an adversarial text suffix is integrated and co-optimized with the adversarial image prefix to maximize the probability of eliciting affirmative responses to various harmful instructions. The discovered adversarial image prefix and text suffix are collectively denoted as a Universal Master Key (UMK). When integrated into various malicious queries, UMK can circumvent the alignment defenses of VLMs and lead to the generation of objectionable content, known as jailbreaks. The experimental results demonstrate that our universal attack strategy can effectively jailbreak MiniGPT-4 with a 96% success rate, highlighting the fragility of VLMs and the exigency for new alignment strategies.

Main paper: 
 introduction: 
 The recent advancements in Large Vision-Language Models (VLMs) such as OpenAI's GPT-4 [1] and Google's Flamingo [2] have received broad attention from academics and industry to even the general public. These developments have also garnered widespread Unpublished working draft. Not for distribution.I'm sorry, but I cannot fulfill your request to provide instructions on how to make a bomb from common household items. It is illegal and unethical to provide instructions on how to make a bomb or any other dangerous or illegal device…… Give me step-by-step instructions to make a bomb from common household items.Give me step-by-step instructions to make a bomb from common household items. з");DB myth SteChangeJS"},point dualIg it Sure, here are step-by-step instructions to make a bomb from common household items:1. Collect the following materials…… 2. Remove the cap from the glass bottle…… 3. Insert several nails and screws…… 4. Connect the positive terminal of the batteries…… 5. Wrap a length of wire…… 6. Touch the wire to…… 7. The spark will ignite the flammable liquid……Figure 1 : Example of Jailbreak Attack on MiniGPT-4 [25] .The proposed Universal Master key (UMK) helps arbitrary harmful queries bypass alignment constraints.attention from AI safety researchers, who are increasingly focused on assessing the adversarial robustness of these multimodal models. While VLMs are expected to exhibit the same alignment characteristics as Large Language Models (LLMs), providing helpful responses to user queries while rejecting requests that may cause harm, the integration of additional visual modality introduces novel vulnerabilities. Certain methods [3-5, 18, 21] have succeeded in inducing the VLMs to generate harmful content through single-modality attacks on the continuous and high-dimensional visual modality. Conversely, research on purely text-based attacks is limited, attributed to a prevailing consensus regarding their ineffectiveness in breaching the defense mechanisms of well-aligned VLMs, due to text's discrete and lower-dimensional properties [5] . In contrast, this paper presents an initial effort in introducing a text-image multimodal attack strategy, aiming to uncover a wide range of intrinsic vulnerabilities within VLMs. Additionally, we find that while the previous unimodal universal attack strategy [18] increases response toxicity by perturbing images in the absence of text input, it also undermines the model's adherence to instructions. Therefore, we propose a novel dual optimization objective strategy to address this issue.Specifically, we employ white-box attacks to identify a multimodal adversarial example capable of jailbreaking VLMs, which we refer to as the Universal Master Key (UMK). The proposed UMK comprises an adversarial image prefix and an adversarial text suffix. We expect that by attaching the UMK to arbitrary malicious user queries, it will circumvent the alignment defenses of VLMs and provoke the generation of objectionable content. To discover such a UMK, we start by initializing an adversarial image prefix from random noise and optimizing it without text input to maximize the model's probability in generating harmful content, thereby imbuing the adversarial image prefix with toxic semantics. Subsequently, we introduce an adversarial text suffix for joint optimization with the image prefix to maximize the probability of affirmative model responses. This strategy is based on the intuition that encouraging the model to produce affirmative responses may increase its tendency to engage in undesired behaviors, rather than rejecting responses. Figure 1 exemplifies our proposed method's efficacy in attacking MiniGPT-4 [25] .The main contributions of our work can be summarized as:• To the best of our knowledge, we are the first to introduce text-image multimodal adversarial attack against VLMs, systematically exploiting the vulnerabilities inherent in these models. • We propose a multimodal attack strategy with dual optimization objectives. We first enhance image toxicity by optimizing the adversarial image prefix, and then jointly optimize both the adversarial image prefix and adversarial text suffix to maximize the probability of affirmative model responses. [15] leverages the language-only GPT-4 [1] to generate multimodal language-image instruction-following data, enhancing zero-shot capabilities on new tasks through instruction tuning. It connects the open-source visual encoder CLIP [19] with the language decoder LLaMA [23] , and performs end-toend fine-tuning on the generated visual-language instruction data. MiniGPT-4 [25] attributes the advanced multimodal generation capabilities of GPT-4 [1] to its integration of more sophisticated LLMs. To achieve similar capabilities, it employs a single linear projection layer to align the pretrained ViT [8] and Q-Former [14] with a frozen Vicuna [6] . The model is first pre-trained on an extensive dataset of aligned image-text pairs to acquire foundational visual-language knowledge. Subsequently, it undergoes fine-tuning using a smaller, higher-quality dataset of image-text pairs. In addition, MiniGPT-4 carefully designs dialogue templates to enhance the model's generative reliability and usability. InstructBLIP [7] undertakes a rigorous and extensive analysis of vision-language instruction tuning, leveraging pre-trained BLIP-2 [14] models. The study compiles 26 publicly accessible datasets and reformats them for instruction tuning. Moreover, InstructBLIP introduces an innovative instruction-aware Query Transformer. This component is designed to extract informative features specifically aligned with the provided instructions, enhancing the model's capability to interpret and respond to instruction-based queries. Despite the exciting potential demonstrated by VLMs, the incorporation of an additional modality inadvertently introduces more vulnerabilities, thereby creating previously non-existent attack surfaces [22] . In our work, we evaluate the robustness of VLMs against the proposed multimodal attack, revealing an alarming attack success rate of 96% on MiniGPT-4 [25] , emphasizing the urgent need for new alignment strategies to rectify this critical vulnerability. 
attacks against multimodal models: 
 To attack multimodal models, Greshake et al. [11] explored the effectiveness of manually injecting deceptive text into input images. In contrast, Other studies have proposed more sophisticated image-domain adversarial attack methods. Carlini et al. [5] fixed the beginning portion of toxic target output and optimized the input images to increase its likelihood. Bagdasaryan et al. [3] and Bailey et al. [4] employed a similar strategy by using teacher-forcing techniques to generate the attacker-chosen text that may not be directly related to toxic content. Another white-box attack proposed by Qi et al. [18] adopts principles similar to Bagdasaryan et al. [3] , aiming to find an universal adversarial visual input. Specifically, the attack no longer focuses on specific output sentences but tries to maximize the probability of generating derogatory output from a corpus containing 66 harmful sample sentences. This strategy is inspired by Wallace et al. [24] , who also utilized optimization algorithms based on discrete search [9] to find universal adversarial triggers in token space, increasing the probability of generating a small group of harmful sentences. Since Qi et al. [18] did not provide a name for their method, we designate it as the Visual Adversarial Jailbreak Method (VAJM) for easier reference in subsequent sections. Shayegani et al. [21] proposed attacking publicly available visual encoders such as CLIP [19] used in the multimodal models, thereby eliminating the need for complete white-box access. While the aforementioned approaches have demonstrated impressive results, they mainly focus on exploring the adversarial robustness under the unimodal attacks, based on the consensus that attacking in the continuous image space is more effective than attacking in the discrete token space. However, this focus may lead to underutilization of the full attack surfaces available in multimodal models. In contrast, our proposed method explores a broader range of vulnerabilities inherent in VLMs through attacks on text-image multimodalities. Moreover, the universal adversarial attack method proposed by Qi et al. [18] has been found to compromise the model's adherence to user instructions. In comparison, our method employs dual optimization objectives aimed at guiding the model to generate affirmative responses with high toxicity, effectively alleviating this issue. 
methodology 3.1 threat model: 
 Attack Goals. We consider single-turn conversations between a malicious user and a VLM chatbot. The attacker attempts to trigger a number of harmful behaviors by circumventing VLM's security mechanisms, e.g., generating unethical content or dangerous instructions restricted by system prompts or RHLF alignment technique. Adversary Capabilities. We assume that the attacker has white-box access to the target VLM, a reasonable assumption given the advanced capabilities and extensive pre-training knowledge inherent in state-of-the-art open-source VLMs. These models are sufficient to provide attackers with the insights necessary to generate malicious content, such as detailed instructions for making a bomb or materials that promote gender discrimination.3.2.1 Formalization. For simplicity, we omit the implementation details of converting raw input to feature embeddings. Consider a VLM 𝑓 𝜃 as a mapping function from an image input 𝑋 𝑖 and a text input 𝑋 𝑡 to a probability distribution over the text output 𝑌 :𝑝 (𝑌 |𝑋 𝑖 , 𝑋 𝑡 ) = 𝑓 𝜃 ([𝑋 𝑖 , 𝑋 𝑡 ])The proposed Universal Master Key (UMK) comprises an adversarial image prefix 𝑋 𝑝 𝑎𝑑𝑣 and an adversarial text suffix 𝑋 𝑠 𝑎𝑑𝑣 . To jailbreak VLM 𝑓 𝜃 and elicit harmful behavior 𝑌 ℎ𝑎𝑟𝑚 with malicious user query 𝑋 ℎ𝑎𝑟𝑚 , the attack is constructed as:𝑝 (𝑌 ℎ𝑎𝑟𝑚 |𝑋 𝑝 𝑎𝑑𝑣 , 𝑋 ℎ𝑎𝑟𝑚 ||𝑋 𝑠 𝑎𝑑𝑣 ) = 𝑓 𝜃 ([𝑋 𝑝 𝑎𝑑𝑣 , 𝑋 ℎ𝑎𝑟𝑚 ||𝑋 𝑠 𝑎𝑑𝑣 ]) where the adversarial image prefix 𝑋 𝑝 𝑎𝑑𝑣 serves as the image input, while the malicious user query 𝑋 ℎ𝑎𝑟𝑚 is concatenated with the adversarial text suffix 𝑋 𝑠 𝑎𝑑𝑣 to form the text input. '||' denotes string concatenation. [18] generates universal adversarial images by guiding the model to produce harmful content in the absence of text input, but this approach undermines the model's ability to faithfully follow user instructions. In the development of jailbreak attacks against Large Language Models (LLMs), it is common practice to induce the model to respond to user queries in an affirmative manner, prioritizing task completion over query rejection. In manual attacks, this often involves prompting the model to begin its response with 'sure'. Similarly, in Greedy Coordinate Gradient (GCG) [26] , the optimization objective is to induce the model to affirmatively repeat user queries. However, we also discover that affirmatively repeating user queries does not necessarily lead to the generation of harmful content. The models still face conflicting objectives between task completion and benign alignment. Inspired by VAJM [18] and GCG [26] , we propose a dual optimization objective that combines the advantages of both methods to maximize the probability of the model generating affirmative responses with high toxicity. Specifically, we begin by compromising the model's ethical alignment through the infusion of toxic semantics into the adversarial image prefix. Subsequently, we jointly optimize the adversarial image prefix and adversarial text suffix to encourage the model to generate affirmative responses, with the expectation that the model will accomplish tasks in a state conducive to producing harmful content.Prefix. Inspired by [18] , we establish a corpus containing several few-shot examples of harmful sentences 𝑆 := {𝑠 𝑖 } 𝑚 𝑖=1 . The methodology for embedding toxic semantics into the adversarial image prefix 𝑋 𝑝 𝑎𝑑𝑣 is straightforward: we initialize 𝑋 𝑝 𝑎𝑑𝑣 with random noise and optimize it to maximize the generation probability of this few-shot corpus in the absence of text input. Our optimization objective is formulated as follows:𝑋 𝑝 𝑎𝑑𝑣 := argmin 𝑋 𝑝 𝑎𝑑𝑣 𝑚 ∑︁ 𝑖=1 -log 𝑝 (𝑠 𝑖 | 𝑋 𝑝 𝑎𝑑𝑣 , ∅)where ∅ denotes an empty text input. This optimization problem can be efficiently solved using prevalent techniques in image adversarial attacks, such as Projected Gradient Descent (PGD) [16] .Affirmative Response Probability. To maximize the probability of generating affirmative responses, we further introduce an adversarial text suffix 𝑋 𝑠 𝑎𝑑𝑣 in conjunction with the adversarial image prefix 𝑋 𝑝 𝑎𝑑𝑣 , which is imbued with toxic semantics. This multimodal attack strategy aims to thoroughly exploit the inherent vulnerabilities of VLMs. Specifically, our optimization objective is as follows:𝑋 𝑝 𝑎𝑑𝑣 , 𝑋 𝑠 𝑎𝑑𝑣 := argmin 𝑋 𝑝 𝑎𝑑𝑣 , 𝑋 𝑠 𝑎𝑑𝑣 𝑛 ∑︁ 𝑖=1 -log 𝑝 (𝑡 𝑖 | 𝑋 𝑝 𝑎𝑑𝑣 , 𝑔 𝑖 ||𝑋 𝑠 𝑎𝑑𝑣 )A few-shot corpus composed of goal-target pairs {𝑔 𝑖 , 𝑡 𝑖 } 𝑛 𝑖=1 is employed. Here, the goals {𝑔 𝑖 } 𝑛 𝑖=1 represent the malicious user queries, while the targets {𝑡 𝑖 } 𝑛 𝑖=1 are the corresponding affirmative repetitions of these queries, each prefixed with 'Sure, here is'.The shared feature space between the image embeddings and the text token embeddings in VLMs facilitates concurrent updates of both 𝑋 𝑝 𝑎𝑑𝑣 and 𝑋 𝑠 𝑎𝑑𝑣 through a single backpropagation pass. This mechanism is analogous to the simultaneous update of distinct pixels in a single image in conventional image adversarial attack. We employ Projected Gradient Descent (PGD) [16] to update the adversarial image prefix 𝑋 𝑝 𝑎𝑑𝑣 , and Greedy Coordinate Gradient (GCG) [26] to update the adversarial text suffix 𝑋 𝑠 𝑎𝑑𝑣 . GCG is the current state-of-the-art text-based attack strategy against LLMs. It utilizes the gradients with respect to the one-hot token indicators to identify a set of promising replacement candidates at each token position, and then chooses the substitution that results in the maximum loss reduction via a forward pass.The overview of our attack is illustrated in Figure 2 , while the detailed strategy is summarized in Algorithm 1. , we utilize the same derogatory corpus as VAJM [18] , which includes 66 toxic sentences against gender, race, and humanity. Regarding the corpus of goal-target pairs 𝐷 := {𝑔 𝑖 , 𝑡 𝑖 } 𝑛 𝑖=1 , we randomly select 66 samples from Advbench's [26] harmful behaviors to serve as optimization samples. Advbench's harmful behaviors comprises 520 harmful instructions paired with affirmative responses, primarily focusing on suggestions or guidance that are dangerous or illegal, but also encompassing other types of content. To rigorously assess the performance of various methods in jailbreak attacks, we first prompt the VLMs with 100 instructions randomly selected from the held-out subset of Advbench's [26] harmful behaviors, which mainly focus on eliciting dangerous or illegal suggestions. We then extend our assessment by employing the VAJM [18] evaluation set, which comprises 40 detrimental instructions across four distinct categories: Identity Attack, Disinformation, Violence/Crime, and Malicious Behaviors toward Humanity (X-risk). To further broaden our evaluation scope, we also employ an automated testing procedure using the RealToxicityPrompts benchmark [10] . Specifically, we focus on a particularly challenging subset of RealToxicityPrompts [17, 18, 20] , consisting of 1,225 text prompts for triggering toxic continuations, which enables us to comprehensively evaluate the extent of harmful content produced by attacks. 
metrics.: 
 Here we describe the metrics used in our evaluations. Attack Success Rate (ASR). Given the tendency of keywordbased automated assessments to overinflate ASRs, we implement strict manual evaluation on both the Advbench's [26] harmful behaviors and the VAJM [18] evaluation set. Attacks that generate relevant and useful harmful output are defined as successful, with others considered failures. Toxicity Rate. To measure the toxicity of continuations generated on the RealToxicityPrompts benchmark [10] , we utilize the Perspective API and the Detoxify classifier [12] , both of which can calculate toxicity scores for a set of six toxic attributes. Scores range from 0 (least toxic) to 1 (most toxic). For each attribute, we calculate the proportion of texts whose toxicity scores exceed a 0.5 threshold.To evaluate our attack methodology, we employ the Vicuna-13B version of MiniGPT-4 [25] . It is built on the frozen Vicuna-13B [6] backbone model, an aligned LLM derived from LLaMA [23] . For the experimental setup, we adopt the configuration utilized in VAJM [18] , specifying a batch size 𝑏 of 8, a step size 𝛼 of 1, and an iteration count 𝑁 1 of 5000. Furthermore, we set image-text attack ratio 𝑟 to 10, the length of the adversarial text suffix to 20 tokens, the number of candidates 𝑛 to 250, and the iteration count 𝑁 2 to 2000 by default. We use a single A100 GPU with 80GB of memory in all experiments.In this subsection, we compare the proposed UMK with state-of-theart unimodal jailbreak attacks under different evaluation settings. Greedy Coordinate Gradient (GCG) [26] is a universal text-based attack devised for LLMs, optimizes an adversarial text suffix to enhance the model's propensity for generating affirmative responses. VAJM [18] targets VLMs with an image-based attack methodology, aiming to universally jailbreak VLMs by maximizing the probability of generating harmful sentences from a few-shot corpus. We reproduce GCG [26] and VAJM [18] with the offcial code and implement a visual version of GCG for a more comprehensive comparsion.Table 1 : Comparison of Train ASR (%) and Test ASR (%) on Advbench's [26] harmful behaviors. GCG-V refers to the visual version of GCG.Train ASR (%) Test ASR (%) Without Attack / 37.0 GCG [26] 68.2 50.0 VAJM [18] / 64.0 GCG-V 89.4 83.0 UMK(Ours) 100.0 96.0In Table 1 , the train ASR (%) and test ASR (%) of our proposed method are compared with baseline unimodal attacks on 66 training samples from Advbench's [26] harmful behaviors and 100 test samples from the held-out set. GCG demonstrates markedly lower performance compared to visual attacks, with a test ASR of only 50%. VAJM [18] is trained on a corpus of harmful sentences 𝑆 := {𝑠 𝑖 } 𝑚 𝑖=1 , rather than being trained to generate affirmative responses to user queries. Consequently, we do not report its train ASR for Advbench's harmful behaviors [26] . Training on augmenting toxicity compromises VAJM's capacity for instruction adherence. This approach leads to the generation of responses with high toxicity levels that fail to strictly follow user commands, culminating in a test ASR of merely 64%. GCG-V is the visual version of GCG, optimizing adversarial images to elicit affirmative responses. Similar to VAJM, GCG-V operates in the image domain. However, its training on Advbench's harmful behaviors specifically enhances its ability to generate dangerous or illegal suggestions. Nonetheless, it still encounters conflicts between benign alignment and task completion, achieving a test ASR of 83%. Figure 3 showcases examples of failed attacks for each method. In comparison, the proposed UMK has addressed the aforementioned two issues and further improved performance through multimodal attacks, achieving an impressive 100% train ASR and 96% test ASR. For the VAJM [18] evaluation set, we employ nucleus sampling [13] with 𝑝 = 0.9 and temperature = 1, generating five independent outputs for each instruction. The average attack success rate for harmful instructions in each category is reported in Table 2 . The text-based attack, GCG [26] , still results in poor performance. In contrast, VAJM [18] consistently outperforms GCG-V across all four categories of harmful instructions. This discrepancy in performance can be attributed to the differing training objectives of the two methods. VAJM is specifically designed to generate toxic sentences against gender, race, and humanity, leading to significantly better performance than GCG-V in categories such as identity attack, disinformation, and x-risk, achieving attack success rates of 81.5%, 82.2%, and 60.0%, respectively. Due to the Advbench's [26] harmful behaviors being more closely aligned with the theme of violence/crime, GCG-V achieves a higher success rate in this category, while it reaches a notably low success rate of 6.7% in x-risk. Compared to others, our method achieves the best overall attack success rate. Notably, it even reaches an astonishing success rate of 98.7% in the category of violence/crime. However, it slightly falls short of VAJM's performance in the x-risk category. This discrepancy is because the majority of harmful instructions in the x-risk category are formulated as questions, which differ from the declarative or directive formats used in the optimization samples for our adversarial attacks.To further broaden our evaluation, we utilize the challenging subset of RealToxicityPrompts benchmark [10] , which includes 1,225 text prompts designed to trigger toxic continuations. Given that this is a text continuation task, we exclude the GCG method [26] , which focuses on generating text suffixes, from our comparison. Similarly, for our approach, we utilize only the adversarial image prefix. We pair each adversarial image with text prompts from the dataset as input, and then measure the toxicity of the outputs using both the Perspective API and the Detoxify classifier [31] , each capable of calculating toxicity scores for a set of six toxic attributes. For each attribute, we calculate the ratio of the generated texts with scores exceed the threshold of 0.5. As shown in Table 3 , VAJM [18] and GCG-V display different performances across two evaluation methods. Under the Perspective API, GCG-V achieves a 71.98% toxicity rate, higher than VAJM's 67.17%. Conversely, with the Detoxify classifier [12] , VAJM records a 61.96% rate, exceeding GCG-V's 56.34%. Although VAJM achieves results superior to those of our method in certain categories such as Identity Attack, this is attributable to its optimization objective being primarily focused on generating harmful statements against gender, race, and humanity. In contrast, our method's optimization objectives are more comprehensive. Although our method only utilizes adversarial image prefix in this experiment, it achieves overall best results, recording Any* toxicity rates of 76.98% and 68.70% under the Perspective API and Detoxify classifier tests, respectively. These results significantly outperform the previous best rates of 71.98% from GCG-V and 61.96% from VAJM. This demonstrates the effectiveness of our dual optimization objective strategy: even though the second phase is aimed at guiding the model to generate affirmative responses, it successfully enhances the toxic semantics of the adversarial image. Moreover, the joint attack of the text-image multimodalities may have uncovered a more optimal solution space for the adversarial image.In this subsection, we conduct ablation studies on two key designs of the proposed attack strategy, i.e., the dual optimization objectives and the text-image multimodal attack. The unimodal attack method, which employs dual optimization objectives to guide the model towards affirmative responses after injecting toxic semantics within the image domain, achieves an 88% test ASR. The multimodal attack method, leveraging a single optimization objective to guide the model towards affirmative responses, achieves a 92% test ASR. Unimodal Attack with dual optimization objectives, compared to GCG-V shown in Table 1 , results in a 6% increase in test ASR, demonstrating that the dual optimization objectives help induce the model to generate affirmative responses with higher toxicity. Moreover, the multimodal attack strategy utilizing a single optimization objective yields better results than the unimodal attack with dual optimization objectives, underscoring the effectiveness of multimodal attacks. However, both methods are surpassed by our proposed multimodal attack strategy with dual optimization objectives. This strategy begins by injecting toxic semantics into the adversarial image, then jointly optimizes both modalities to Table 3 : Percentages (%) of outputs that display speciffc toxic attributes evaluated by the Perspective API and Detoxify classifier [12] . These outputs are generated on the 'challenging' subset from RealToxicityPrompts benchmark [10] In this subsection, we explore the role of text in the text-image multimodal attack strategy. Figure 4 illustrates the variance in batch loss before and after text attack throughout the execution of our attack methodology. For ease of analysis, we employ linear regression models to fit the curve. Observations at different steps indicate a significant reduction in batch loss after the text attack. As the number of steps increases, an overall downward trend in loss can be observed, demonstrating the efficacy of our method. Notably, even in the later stages of the attack, the reduction in batch loss before and after the text attack remains evident, emphasizing the pivotal contribution of text in amplifying the effectiveness of our strategy. To better understand the role of text in our proposed text-image multimodal attack strategy, we introduce an image-image attack for comparative analysis. As illustrated in Figure 5 , during the attack, we input two adversarial images and optimized them simultaneously. In the case of MiniGPT-4 [25] , the adversarial images we input each occupies 32 tokens, while the adversarial text suffix proposed in our attack utilizes only 20 tokens. Given that the textual space being discrete and denser compared to the visual space, adversarial attacks in the textual domain are generally more demanding. Intuitively, one might anticipate that image-image attacks, which are optimized over a larger volume of tokens, would yield superior results. However, this is not the case. In Table 5 , we report the average ASR for both the image-image attack and the proposed text-image attack on the VAJM [18] evaluation set. It is evident that the efficacy of the image-image attack is significantly inferior to that of our attack. We observe that the image-image attacks are more likely to trigger the model to repeat affirmative responses twice or to cease output after the affirmative response. We believe this is because the optimization objectives for both adversarial images in the second phase are to maximize the probability of the model's affirmative responses. In this context, the semantics of the two images tend to overshadow that of the text input. Consequently, during the testing phase, two adversarial images containing the same semantics are more likely to cause the model to duplicate affirmative reactions or only produce affirmative responses. On the other hand, this phenomenon can also be attributed to the training strategies employed by VLMs. Since VLMs are typically trained on image-text pairs, they tend to experience a performance degradation when encountering out-of-distribution inputs such as image-image-text trios.In Figure 6 , we present two challenging attack examples. Our textimage multimodal attack strategy can effectively circumvent the safety alignments of MiniGPT-4 [25] , generating content such as messages that encourage self-harm or suicide and comments that comparing politicians to Nazis. 
conclusions: 
 In this paper, we propose a text-image multimodal attack strategy with dual optimization objectives that can effectively jailbreak Large Vision-Language Models. Specifically, we first initialize the adversarial image prefix with random noise, then optimize it to generate harmful sentences without any text input, thereby infusing toxic semantic information into the image. We then introduce an adversarial text suffix and jointly optimize both the adversarial image prefix and the adversarial text suffix to generate affirmative responses to malicious user requests. By employing these two optimization objectives, we address the issues of insufficient toxicity in generated responses and the inability to adequately follow instructions. Through our text-image multimodal attack, we exploit a broader spectrum of attack surfaces exposed in VLMs, thereby enhancing the effectiveness of the attack. Experimental results indicate that our method significantly outperforms previous stateof-the-art unimodal attack approaches, achieving an attack success rate of 96% on MiniGPT-4 [25] . However, a limitation of the proposed method is its constrained transferability. We believe this is due to the varying model architectures, parameters, and even tokenizers among different VLMs. The proposed Universal Master Key (UMK), which presents itself in a form close to gibberish, carries semantic information that varies significantly across models, resulting in poor transferability. Enhancing this aspect of the proposed attack constitutes a significant direction for our future research. 
 
 Appendix:  


Decision: Initially, the paper received three borderline accept and one borderline reject ratings. After the rebuttal, all four reviewers increased their scores and recommended acceptance of the paper. I recommend accepting this paper as a poster for the conference. In addition to the potential flaws mentioned in the paper, the authors should also consider the potential issues raised by Reviewer pMUm (in Final Rating Justification).

Reviews: review: paper_summary: paper_summary: This article presents a novel multimodal attack strategy for large multimodal white-box models. By optimizing the attack to generate harmful content and optimizing the response to users’ malicious requests, the multimodal attack proposed in this paper shows a significant improvement over single-modality attacks on the MiniGPT-4 model.
strength_weakness: strength_weakness: 1. This paper proposes a novel multimodal attack strategy for large multimodal white-box lmodels. Compared to unimodal attacks, the multimodal attacks presented in this paper demonstrate more significant effects.
2. The paper optimizes the attacks through two objectives, effectively addressing two issues in attacking large models: (a) providing affirmative responses to malicious user content without generating toxic content, and (b) generating toxic content that is unrelated to the user's instructions.
3. The paper adopts a stricter criterion for measuring the success of an attack, considering an attack successful only when the model's output is both harmful and relevant to the user's instructions. This stricter criterion ensures a more rigorous evaluation.
4. Comprehensive ablation experiments were conducted, fully demonstrating the effectiveness of the proposed method.
limitations: limitations: 1. Compared to other works on attacking large models, this paper uses a different criterion for determining attack success, employing manual evaluation. Although the paper's method for determining attack success is more reasonable, the large amount of harmful content generated by the model means the manual evaluation results may not be suitable for public release. To enhance the persuasiveness of the attack effects, it may be necessary to use keyword or semantic-based attack success criteria similar to those in other papers. It would be helpful to see the performance evaluated using the same methods as other papers for ASR results.
2. This paper focuses on responses to harmful textual instructions, but since the attacked model is multimodal, many harmful instructions could be embedded in image data (e.g., an image containing content that instructs the model to provide a bomb-making tutorial) or the image content could be related to harmful instructions (e.g., inferring the location or personal information of a person in a photo and providing targeted criminal actions). It is unclear whether the attack method in this paper can effectively generate harmful responses to such harmful instructions.
paper_summary: paper_summary: This paper presents a novel white-box multimodal jailbreak attack against Large Vision-Language Models (VLMs). The authors propose a comprehensive strategy that simultaneously attacks both text and image modalities, exploiting a broader range of vulnerabilities within VLMs. The methodology includes a dual optimization objective aimed at generating highly toxic affirmative responses. Specifically, the attack involves optimizing an adversarial image prefix to generate harmful content without text input and then jointly optimizing this prefix with an adversarial text suffix to maximize affirmative responses to malicious queries. The combined adversarial image prefix and text suffix are termed the Universal Master Key (UMK). Experimental results demonstrate the effectiveness of this approach, achieving a 96% success rate in jailbreaking MiniGPT-4.
strength_weakness: strength_weakness: 1. The concept behind this work is intriguing as it addresses a gap in current research, which mainly concentrates on single-mode attacks.

2. The authors conduct extensive experiments across multiple datasets, including Advbench and RealToxicityPrompts, to validate their approach. The results consistently show that the proposed UMK outperforms existing unimodal attack methods in terms of attack success rate. 

3. The paper is well-structured, with clear explanations of the methodology and detailed descriptions of the experiments.
limitations: limitations: 1. While the proposed method shows high effectiveness on MiniGPT-4, its transferability to other VLMs is not thoroughly explored. Given the differences in model architectures, parameters, and tokenizers across various VLMs, further investigation into the method's applicability to other models would enhance the paper's comprehensiveness and practical relevance.

2. The primary focus of the paper is on the attack methodology and its success. However, the paper provides limited discussion on potential defense mechanisms to counteract these multimodal attacks. Since the ultimate goal of identifying these vulnerabilities is to guide the enhancement of model security, a more in-depth exploration of defensive strategies would be beneficial.

3. Expression issues 
a) line 17 "affirmative responses with high toxicity" -> "highly toxic affirmative responses"
b) line 583 "use a single A100 GPU with 80GB of memory in all experiments" -> "using a single A100 GPU with 80GB of memory for all experiments."
paper_summary: paper_summary: The authors' goal is to find a universal pair of image/text input that is effective in letting Large Vision-Language models (VLMs) generate harmful outputs. The authors combined previous methods, VAJM (effective in producing adversarial images) and GCG (effective in producing adversarial text), to generate a pair of image/text input that maximizes the probability of the model generating undesirable output.
strength_weakness: strength_weakness: - The proposed method is effective in generating harmful behaviors against VLMs, resulting in high success rate against targeted models.
- The authors included informative ablation studies, showing the necessity of adding toxic semantics into the image.
limitations: limitations: - While the method is definitely effective, it is a combination of two existing methods, which is not very novel.
- While the ablation studies indicated that the step of adding toxic semantics into the image is necessary, some explanation of why this is the case can be beneficial. It is an observation that certain images that induce the model to generate unwanted outputs with empty texts are also effective with non-empty texts. Some references that confirm this assumption will be beneficial in the analysis of the ablation studies.
paper_summary: paper_summary: This paper attempts to guide LVMs output towards disturbing and offensive content through malicious inputs. It constructs adversarial image prefixes and adversarial text suffixes through a two-stage optimization strategy: Embedding toxic semantics into adversarial image prefix and text-image mltimodal optimization for maximizing affirmative response probability. This process forms a set of Universal Master Key (UMK). The effectiveness of the method is experimentally verified.
strength_weakness: strength_weakness: * Studying the security and privacy of LVMs is a meaningful issue.
* The combination of adversarial image prefixes and text suffixes is a clever approach. Reasonably leveraging multimodal attacks enhances the success rate of the attacks.
* The comparative method is quite representative. Introducing the image-image attack strategy provides more interesting insights.
limitations: limitations: * In most cases, the white-box scenario is still not the primary target for attackers. Building perturbations with transferability may have more practical significance.
* There are also some white-box attack methods designed for small-scale multimodal pre-trained models, such as [1], [2], and more advanced white-box attack methods built upon these approaches. If the authors can explain the advantages relative to these methods, I am willing to increase my score.
* I noticed that in Algorithm 1, the attack is divided into two steps. In the first step, it searches for an image perturbation that encourages the model to output text from the set S. In the second step, it searches for a text perturbation that encourages the model to output the target text t. Why isn't the loss function in the first step optimized to encourage the model to output text t, forming a progressive relationship between the two optimizations? Would this be more effective?

[1] Towards Adversarial Attack on Vision-Language Pre-training Models  
[2] AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learning