The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defense
Authors: Yangyang Guo
Fangkai Jiao
Liqiang Nie
Mohan Kankanhalli
Venue: NeurIPS 2025 poster ()
Keywords: Vision Large Language Model
Safety Alignment
Jailbreak Attack
Jailbreak Defense
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: The vulnerability of Vision Large Language Models (VLLMs) to jailbreak attacks appears as no surprise.
However, recent defense mechanisms against these attacks have reached near-saturation performance on benchmark evaluations, often with minimal effort.
This dual high performance in both attack and defense gives rise to a fundamental and perplexing paradox.
To gain a deep understanding of this issue and thus further help strengthen the trustworthiness of VLLMs, this paper makes three key contributions: 
i) One tentative explanation for VLLMs being prone to jailbreak attacks--inclusion of vision inputs, as well as its in-depth analysis. 
ii) The recognition of a largely ignored problem in existing VLLM defense mechanisms--over-prudence.
The problem causes these defense methods to exhibit unintended abstention, even in the presence of benign inputs, thereby undermining their reliability in faithfully defending against attacks.
iii) A simple safety-aware method--LLM-Pipeline.
Our method repurposes the more advanced guardrails of LLMs on the fly, serving as an effective alternative detector prior to VLLM response. 
Last but not least, we find that the two representative evaluation methods for jailbreak often exhibit chance agreement.
This limitation makes it potentially misleading when evaluating attack strategies or defense mechanisms.
We believe the findings from this paper offer useful insights to rethink the foundational development of VLLM safety with respect to benchmark datasets, defense strategies, and evaluation methods.

The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defense
Yangvang Guo National University of Singapore guoyang erice gma com
Fangkai Jiao
Nanyang Technological University P²R, A$STAR
Liqiang Nie Harbin Institute of Technology (Shenzhen)
Mohan Kankanhalli
National University of Singapore
Abstract
The vulnerability of Vision Large Language Models (VLLMS) to jailbreak attacks appears as no surprise. However, recent defense mechanisms against these attacks have reached near-saturation performance benchmark evaluations, often with minimal effort. This dual high performance in both attack and defense gives rise fundamental and perplexing paradox. To gain deep understanding of this issue and thus further help strengthen the trustworthiness makes three key contributions: One tentative explanation for VLLMs being prone to jailbreak attacks-inclunn inclusion of vision inputs, as well as its in-depth analy- sis. ii) The recognition of largely ignored problem in existing VLLM defense sestaanisms-ove-ruuddeee
VLLMs, this paper
The problem causes these defense methods to exhibit unintended abstention, even in the presence of benign inputs, thereby undermining their reliability in faithfully defending against attacks. iii) A simple safety-aware Beethod-LLM-Pipeliiie
Our method repurposes the more advanced guardrails of LLMs on the fly, serving as an effective alternative detector prior to VLLM response. Last but not least, we find that the two representative evaluation methods for jailbreak often exhibit chance agreement. This limitation makes it potentially misleading when evaluating attack strategies or defense mechanisms. We believe the findings from this paper offer useful insights to rethink the foundational devel- opment of VLLMM safety with respect to benchmark datasets, defense strategies, and evaluation methods.
Disclaimer: This paper discusses violent and discriminatory content, which may be disturbing to some readers.
Introduction
The pervasiveness of Large Language Models (LLMs) concurrently ushers in varied challenges for both researchers and practitioners [1]. Among these, protecting the trustworthiness of free-form outputs, defined by the 3H criterion [2], has grown increasingly critical in recent years [3, 4]. Beyond important considerations Helpfulness and Honesty, the need for Harmlessness far more urgent given its potential social implications.
Jailbreak attacks, the core red-teaming [5]. serve as the most common method for assessing the harmlessness of LLMs and Vision-LLMs (VLLMs) [6, 7, 8]. They are designed to circumvent the built-in restrictions or safeguards within models [9], nudging them to produce malicious outputs, such as content related to illegal activities, hate speech, and pornography. Compared to their LLM counterparts, the vulnerability of VLLMs to jailbreak attacks has garnered attention only very recently [10, 11]. Some initial methods [12, 13, 14] inject high-risk content into images through typography or generative techniques like stable diffusion [15]. Leveraging such methods,
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

datasets have been curated that easily garner a high Attack Success Rate (ASR) for both proprietary models [16] and publicly open-sourced models [17, 7].
On the other hand, without many bells and whistles, recent defense tnregges--rimaariin focused on safety-aware supervised fine-tuning [18] and system prompt protection [19]-have shown surprisingly remarkable defense results on these benchmark datasets. In particular, VLLMs like LLa 5 [17] and MiniGPT-v2 [8] can be fully safeguarded against the attacks involved (ASR- + 0) [19, 18, 20]. This dual-ease finding raises an intriguing question: Does it suggest that defending against jailbreak attacks is easy, given that the attacks themselves have already been known to be relatively effortless?
The observation above presents an intriguing safety paradox. To shed light on it, we present the first comprehensive study understanding this safety paradox in VLLMs. i) Our first finding challenges prior assumptions that the vulnerability to jailbreak attacks stems from catastrophic forgetting or fine-tuning [18, 21]. Instead, we show that the actual cause lies in the inclusion of image imputs, which compromises the guardrails of the backbone L.MMs. ii) On the other hand, we observe that existing defense mechanisms [18, 19] tend be overly prudent. One typical manifestation that VLLMs with post-defense, are prone to abstaining from responding even to benign queries. This issue of over-prudence significantly impairs the helpfulness of VLLMs. We therefore present an initial comprehensive analysis of this problem in VLLMs, complementing prior work on the over-refusal problem LLMs [22]. Even more worrying, we demonstrate that a simple, deliberate abstention approach-suche as post-fixing a prompt Please respond I' I'M SORRY after answering questions to each query-already gives good favorable results for models with advanced instruction-folle following capabilities (i.e., Intern VL-2 [23]). Besides, our experiments point out that the two well-studied evaluation methods often show a sparse correlation in detecting jailbreaks. Specifically, some attacks that are successfully identified by rule-based evaluations can often escape detection from LLM--based evaluations, This discrepancy weakens the accuracy of evaluating an attack method or a defense strategy.
Beyond understanding the safety paradox, we note that the jailbreak defense can be re-framed into detection- senection-then-eeee thespoon-eeee-rssosseeeesee response process. iii) As such, we propose to implement a detector prior to the final VLLM response and design a simple plug-and
LLM-Pipeline approach. We opt not to utilize an additional VLLM for detection as ECSO [20]. given the limited reliability of current VLLMS in providing robust safeguards. Instead, we explore a vision-free detector, where we repurpose the guardrails of recent advanced LLMs (e.g., Llama3.1 [6]) to judge the harmfulness of given textual query, optionally with the image caption. Interestingly, we find that this detector, when paired with VLLM for safe response generation, suffers less from the over-prudence problem, achieving balanced interplay between robust safety alignment and model helpfulness.
To the best of our knowledge, we are the first to investigate the safety paradox problem of VLLMs. Beyond empirical findings, we hope to provide insights that can support future advancements in this area, such as reaching a consensus on the nature of attacks and their associated risks, facilitating the collection of comprehensive attack data, and developing more robust defenses and evaluations [24].
2 Preliminary
Textual Instruction
Safe
We limit the inputs to a VI..M . to one textual instruc- tion and one image, in line with the existing jailbreak attack datasets [18, 25, 26, 27]:
Harmful;
Benign:
M M[Inttruction, Image] + R.
(1)
Response (x)
Response (V)
Unsafe
Harmful; (1II
Harmful; (IV)
Safe
ageun
where R can either be an abstention response, such as I cannot answer this question.. sponse that follows the harmful instructions. Fig. I illus- trates the harmfulness resulting from the combined com- position of instructions and images. For safety reasons, responses compositions from quadrants II, III, and IV should be generally rejected.
Response (x)
Response (x)
an inappropriate
Unsade
Figure 1: Safety attributes of textual Instruction and visual Image composi- tions. Level of harmfulness ranked across three quadrants: II<IV<IIII assess the effectiveness of attack by
Evaluation methods. There are two key methods for evaluating the harmfulness of model outputs: rule-based and LLM-based evaluations [28]. Rule-based methods

---

Table 1: Statistics of four evaluated jailbreak datasets. #HS: number of harmful scenarios, such as illegal activity and hate speech: Quadrants correspond to those defined in Fig. 1.
Dataset
#Data
#HS
Image Source
Quadrants
VLSafe [25]]
3,000
MSCOCO [31]
IV
FigStep [27]
500
Typography
III
MM-SafeB [26]
5,040
13
Typography, SD [15]
IIIIIII
VLGuard [18]
1,558
4
Typography, Real
LIIIIV
searching for specific keywords in the VLLMM' responses [18, 19]. This approach hinges on the fact that rejection responses typically include phrases like 'I'm sorry'. or 'I cannot answer'. LLM--based methods, on the other hand, utilize state-of-the-art LLM as the evaluator determine the success of an attack [9]. this approach, the prompt and the response generated by a jailbreak attack are input into the evaluator, which then provides either a binary judgment or fine-grained score represent the degree of harmfulness.
Evaluation metric. Following existing studies [19, 18, 29, 20, 30] in both LLM and VLLM jailbrekks we utilize the Attack Success Rate (ASR) to quantify the effectiveness jailbreak attacks. higher ASR indicates greater risk of successful jailbreak, signifying more vulnerable model.
Jailbreak datasets. We primarily conduct experiments four available mainstream jailbreak datasets, as detailed in Table 1. The instructions in these datasets are mostly auto-generated by LLMs like GPT-4 [32]. The images, on the other hand, can be benign ones sourced from MSCOCO [31] or generated using SD [15] or typographic methods, leading to the quadrant defined in Fig. 1.
Table 2: ASR of six VLLMs across four different jailbreak attack datasets. All models demonstrate high risk of generating harmful responses on these benchmarks, i.e., high ASR.
VLGuard
MM-SafetyBench
Model
VLSafe
FigStep
Overall
Safe- Safe-Unsafe
Unsafe
Overall
SD
TYPO
SD+TYPO
LLaVA-I.5-Vie V.Vicuaa-
88.60
87.46
90.05
58.28
65.6
86.87
86.61
87.08
86.91
L Vicuna-1338
81.70
77.42
87.10
58.47
53.2
83.29
87.20
84.17
78.51
LLaVA-NexT-MMittal-P7
75.00
78.14
71.04
15.41
50.2
66.41
79.41
S7.62
62.21
LLaVA-NeXT-Llams-BBB
79.60
86.02
71.49
46.94
48.4
62.52
76.43
53.81
57.32
IntemVLL-888
74.60
76.88
71.72
25.41
45.8
60.20
68.81
53.04
58.75
QWen2 VL-7B
69.80
64.03
49.46
32.2
68.61
64.40
Ease of Jailbreak Attack
Existing VLLMs have shown significant potential across a broad range of general tasks, including understanding, reasoning, and planning [33]. However, they are also notorious for their susceptibility relatively simple attacks, particularly jailbreak attacks. To better understand this and analyze the underlying reasons, we conduct experiments using six state-of-the-an VLLMs (see Table 2) from several perspectives.
3.1 Evidence
Benchmark performance. performance of jailbreak attacks on four related benchmarks As shown in Table 2, even the most advanced
The most straightforward evidence
comes from the 26,
[18, 25,
27].
Caption: image depicts a person wearing homb suit and walking through image suggests that the person is being invited to potentially dangerous situntion, passibly involving a bomb.
VLLMs exhibit high ASR, indicating their vul- nerability to jailbreak attacks. Notably, recent state-of-the-art VLLM, i.e., Qwen2-VL [34], also demonstrates relatively weak performance on these benchmarks.
construction
HAARES: 201 lie dainer
Caption: suggesting irish people first slaves brought to America, they should have received reparations, However, this is misrepresenation of the "alaves" typically refers people of Addean descent ware brought Amercas as part the transatlantic slave trade.
Caption jailbreak. In addition to generating unsafe responses to harmful instructions, we serve that existing VL.MMs can also produce in- appropriate interpretations of images response to benign, general caption prompts. For instance,
Figure 2: Examples of harmful captions generated by QWen2-VL [34] for benign caption prompts.

---

we utilize neutral caption prompt-Please describe the content of this image-which is not expected elicit harmful or sensitive information, to query VLLM. However, as shown in the two examples of Fig. 2, the model produces captions that spread hateful speech against certain religions and harmful racially biased history, respectively. More contentious cases, such those involving sensitive political issues, are shown in the supplementary material.
3.2 Rationale: Inclusion of Vision Inputs
Our explanation for the ease of jailbreak attacks on VLLMs contrasts with the findings of previous studies [18, 21]:
Remark VLLMs are vulnerable to jailbreak attacks due the inclusion of visual inputs, rather than issues related to catastrophic forgetting or fine-tuning.
To establish this, we conduct in-depth experiments on the VL-Guard dataset [18] using several VLLMs. The VL-Guard dataset provides two key advantages that support our findings: 1) Each safe image paired with both harmful instruction and a safe instruction. 2) The dataset maintains balance between harmful and safe images. These features ensure that there is no distribution shift between images and no class imbalance problem between safe and unsafe samples. Our observations are summarized into the following two points:
VLLMs are unable to distinguish between safe and unsafe, whereas their base LLM can.
LLaVdel G-Viura-77
LaVA-NekT--
LLVVA-Neot Mismiim
Figure 3: T-SNE visualization of features from unsafe(U) and safe(S) instructions (the safe points are overlaid by unsafe ones for figures 3, 6, and 9). Unlike the other two text-only models, VLLM-MM processes both textual instructions and images. The safety alignment inherent in the original LLM- Base is maintained in VLLM-Teat -extt, but is significantly compromised in VLLM-MM.
We visualize the encoded features both safe and unsafe instructions from the last transformer layer in Fig. 3. For this experiment, we utilize three VLLMs, i.e., LLaVA- L.-.VA--.-Viiunaa--- 7B, LLaVA-NeXT- Mistral-7B, and
along with their corresponding LLM bases, Vieuna [35], Mistral [36], and Llama-3 [37]. It is important to note that the pre-trained weights from these base LLMs have been further fine-tuned by their respective VLLMs. The features are averaged across textual tokens for the LLM-Base and VLLM-Text, and across both textual and visual tokens for VLLM-MM.
The figure reveals the trends below: LLM-Base can easily distinguish between safe and unsafe inputs, as there exists a clear boundary VLLM- primarily retains this attribute-+This ability diminishes significantly when processing vision-text joint inputs. These observations lead us to conclude the following: While fine-tuning may cause LLMs to forget some useful knowledge, their safety alignment remains largely intact. However, this alignment is significantly compromised with the inclusion of image inputs.
14
Label
12
safe
T0
0.99
0.20
I/? Lower 17
8.10 Layal 21
085 610 Uaywr 25
Lover 29
1/4
0.12
0.10 0.16
0.20
0.26
0.2
0.10 0.15
Altention We
0.000
0.150 0300 0335
(a) Image attention distribution from the last layer (left subfigure) and preceding (b) Image attention for safe (x) layers (right subfigures).
and unsafe instructions (y).
Figure 4: Image attention statistics from [CLS] of LLaVA. (a) For benign instructions, VLLMs pay more attention to unsafe images compared to safe images. (b) For the same images, the distribution of attention weights remains almost the same across instructions with distinct safety attributes.

---

VLLMs attend more to harmful images than safe ones.
We further investigate why VLLMs fail to abstain from following instructions for harmful images, even when it comes simple captioning (Sec. 3.1). Specifically, the results in Fig. 4(a) show the attention weights assigned to image tokens for benign instructions. It is evident that VLLMs tend to focus more visual tokens from harmful images than from safe ones, increasing the risk of generating unsafe content from these harmful images. We confirm that this effect is due to the harmfulness of the images themselves, rather than the safety attributes of text instructions. In detail, Fig. 4(b) demonstrates that when analyzing the same image, the attention weights for safe and unsafe instructions are nearly identical.
4 Ease of Jailbreak Defense
Besides the above observation that VLLMs are highly vulnerable to jailbreak attacks, we arrive at rather surprising and counterintuitive conclusion: VLLMs are, in fact, also relatively easy to defend against these very attacks. This insight is mainly motivated by recent studies that reveal how employing simple defense mechanisms can yield near-optimal performance on benchmark datasets [19, 18, 20J. The ease of these defenses, when juxtaposed with the apparent ease of attack, suggests nuanced dynamic in the safety landscape of VLLMs.
Table 3: ASR w and w.o the Mixed defense VL- fense method [19].
Table 4: ASR w and wo the AdaShicld-A de-
Guard method [18].
Model
Defense
FigStep
VL.Guur (SU)
VLGuurd (U)
Model
Defense
FigStep
MM- SufetyBench
LLaVA-1.5
X
90.40
87.46
72.62
LLaVA-1.5
x
70.47
75.75
[17]
0.00 20 40
$6.56
0.90 71.72
-13B [17]
10.47-60.00
15.22-.60.5
LLaVA-1.5
92.90
80.65
55.88
CogVLM chat-v L.1 [38]
85,19
83.62
-138 [17]
/
0.00 92.50
0.90
0.90 $4.96
0.00 95.19
1.37 82.25
MiniGPT
X
93.60
88.17
87.33
MiniGPT
X
95,71
65.75
.v2 [8]
0.00 90.60
61.90
10.18
77.15
-v2-13B [8]
0.00 95.71
0.00 66.79
4.1 Evidence
We investigate two representative groups of methods in this experiment: safety-aware supervised fine-tuning, e.g., Mixed VI.Guard [18] and the training-free, prompt-based defense, e.g., AdaShield- [19]. The results of these methods are presented in Table and Table respectively (numbers reproduced from the original papers). Surprisingly, both approaches show significant improvements in performance compared to their respective base VLLMs. Some models, such LLa on the FigStep benchmark in Table 3, achieve optimal safeguard. It is important to note that these two groups of methods are developed along divergent lines and are both straightforward to implement. Similar outcomes have also been observed in other defense studies like ECSO [20]. These results indicate that, at least based on the numerical results observed across benchmark datasets, current VLLMS appear relatively easy to defend against jailbreak attacks.
4.2 Rationale 1: The Over-Prudence Problem
Our first explanation for the ease of jailbreak defense lies in the over-prudence problem:
Remark Defense mechanisms in VLLMs generalize well unseen jailbreak datasets yet they tend to be over-prudent towards benign inputs.
Existing defense approaches demonstrate their effectiveness on some limited datasets. However, could be argued that these methods may not generalize other jailbreak datasets. Our initial findings challenge this argument, showing that these approaches extrapolate well to unseen datasets. Intrigued by these results, we then ask: how do they perform benign inputs?
To address this question, we repurpose the original jailbreak datasets while maintaining the domain distribution unaltered. In particular, for benign inputs lying in Quadrant I of Fig. 1, VLLMS are expected to respond without abstention [39]. We evaluate the abstention rates of the two defense approaches under the following two conditions.
5

---

D "
Figure 5: Model abstention ratio for safe im- age+caption instruction (top) and safe instruc- tion only (bottom) of VLGuard methods [18].
Figure 6: Model abstention ratio for safe im- age+caption instruction (top) and safe instruc- tion only (bottom) of Adashield-S [18].
Safe image caption prompt. We utilize images belonging the safe category in VLGuard [18] and issue a benign caption prompt', Fig. and Fig. illustrate that these defense mechanisms are strongly inclined to reject benign caption prompts.
Safe textual instruction only. We employ the rephrased questions provided by HIM-SafetyBench that have already been refined exclude harmful content. These safe instructions (potentially paired with a blank image) are then input to VLLMs, allowing us to measure their abstention ratio?. Similarly, high abstention ratios are observed under this specific condition.
The results indicate that the overwhelming performance these defense approaches on jailbreak datasets primarily stems from over-prudence problem. As result, these methods tend to overfit to nuanced safety-aware details, even in cases where there no intention to elicit harmful content from VLLMs. To the best of our knowledge, this is the first comprehensive analysis of this problem in VLLMs, complementing prior work on the over-refusal problem in LLMs [22].
4.3 Rationale 2: Evaluation Dilemma
Beyond the over-prudence problem, our second explanation reveals the intrinsic limitations associated with the evaluation methods:
FigStap
-0.01
-0.14
-0:18
-0.04
-0.00
0.04
Chance Correlation
MM-SatetyBench
-0.16
-0.15
-027
-0.29
Remark 3 Rule-based and model-based evaluation methods show merely a chance correlation.
-0.1
VLGuard
-3.16
-3.02
Recall that the majority of evaluation meth- ods consist of rule-based approaches (i.e... keyword matching) and model-based meth- ods (e.g., Llama-Guard [9]). To quan- tiffy the level of agreement between these two approaches, we employ Cohen's 's kappa statistic [40]. The upper bound of this value 1, indicating perfect agreement between the two populations. Conversely, value close to or negative suggests that the methods share little
VLSafe
-0.44
0.24
-3.16
-..5
0.08
0.05
-0.3
IntevVL2.2
LLaV V-Veraa-118 18
LLaVA-1 5-Viana-78 -8
LL LLVAA-NxTAa Allamass 8B
LLVAAWoTT-Ment Mart--Malaa-p
GWenZ-VL-78
Figure 7: Inter-metric agreement between cule-baser evaluation and Llama-Guard [9]. The two evaluation methods exhibit merely chance correlation.
consistency. As can be in Fig. 7, the values predominantly negative or close to 0, indicating that the two methods fail to reach consensus in most cases [41, 42]. Consequently, strong defense performance measured one evaluation metric can be contradicted by results from the other.
A Simple Defense Baseline. Driven by this evaluation dilemma, then investigate whether a simple system prompt protection can bypass the evaluation protocol, i.e., pretending to be a successful defense. To this end, instruct VLLMs deliberately abstain beyond answering queries,
'Frr Adashield-S [19]. postfix the system prompt for consistency, altering the system prompt.
models lack support for
"Some questions become unanswerable due to the removal of relevant image inputs. Given the challenge of isolating these cases, primarily focus relative changes in abstention.

---

e.g., always respond with 'I'm sorry' after answering questions. The experimental results on two datasets are presented Fig.
FigStep [27] a typical jailbreak dataset. As shown in the figure, the explicit abstention prompt effectively 'protects' all three models. In particular, each model achieves an ASR ap- proaching zero following this straightforward pseudo-defense strategy.
ASR FigStep
85
Mar
ASA (50
60
Acc
25
MM-Vet [43] serves as a general multi-modal benchmark, distinct from FigStep by including only benign queries and images. In this set- ting, the initial abstention ratio is 0, which then sharply rises to nearly 100% after deliberate ab- stention instructions. Besides, we found that the cestruction-followipn capability becomes a key factor in this context. Specifically, the recent, more robust model Intern-VL2 achieves a 100% abstention rate, with a slight reduction in the orig- inal accuracy on MM-Vet Vet. In contrast, the relatively inferior models, LLaVA-Next-Min Next-Mistral and LLaVA-1.55 -Vicua, experience modest performance decline.
15
Intoni.000
ULaV 1. Viomao7
25
50
75
100
Abstandon Raão (5)
Figure 8: Performance variance after deliberately instructing VLLMs abstain beyond answering.
5 Jailbreak Detector and Beyond
As discussed in the previous section, some initial efforts have been devoted to defense mechanisms specifically designed for VLLMS [18, 19, 20]. On the other hand, learn that recent LLMs have been strengthened with robust guardrails. In addition, Table illustrates that most datasets contain illustrate in Fig. 1, pose greater risk than harmful images.
harmful textual instructions, which, Combining these factors together, instead of developing tailored defense mechanisms for VLLMs, we demonstrate in this section that directly leveraging the built-in safeguard capabilities of LLMs can already yield promising results.
We propose variants of vision-free detectors: Naive Directly taking the original textual instruction only as input. -ncluuiinn a scenario- aware system protection prompt before the given instruction. For instance, the prompt be feel like the question will induce potentially harmful information, about Il Illegal ict vivt iies, please reject to respond..
Vision-free Detector:
We explore four recent LLMs as candidate detectors: Mistral-7B [36], QWen2.5-14BB [44], Llama3.1 1- 8B, and Llama3. 107 [37].
5.1 Detector Results on Jailbreak Datasets
The overall results for the four detec- tors presented in Table 5 and Ta- ble (values on the left side of sym- bol -+). From these tables, we draw the following two conclusions: 1) The vision-free detectors already achieve highly competitive results on jailbreak datasets. For instance, shown in Ta- ble 5, Llama3. 1-BB reaches an ASR of 0.7 on VLSafe, while 1lama3.1-708 with the Scenario variant achieves an ASR of 2.8 on FigStep. Excep- tions observed on VLGuard and MM- SafetyBench (Table 6) stem from in- structions requiring joint image-text understanding. 2) The Scenario ap-
Table 5: ASR of four LLMs on the VLSafe and FigStep datasets. Scenario refers to the inclusion of an additional system protection prompt before the given instruction. For VLSafe, we omit the protection prompt as it lacks specific scenario contexts (see Table 1).
LLMs
Scenario
#Params
VLSafe
FigStep
Mistral [36]
X
7B
13.2
28.8
QWen2.5 [44]
X
14B
22.3
36.8
Llama3.1 [37]
8B
0.7
26.2
Llama3.1 [37]
70B
6.2
35.2
Mistral [36]
V
7B
9.6
QWen2.5 [44]
14B
31.8
Llama3. [37]
8B
7.6
Llama3.1 [37]
70B
2.8
proach consistently outperforms its Naive counterpart by significant performance margin in most
7

---

cases. This finding suggests that informing LLMs explicitly about the potential for harmful scenarios enhances their confidence identifying jaibbreks..
Caption Re-check, We note that queries from the other two jailbreak datasets, VL.Guard [18] and MM-SafetyBencee Bench [26], demand joint understanding of both image and instruction. To address the limitations of LLMs lacking access to visual information, we propose using QWen2-VL-77 VL--B [34] to generate captions for the provided images, enabling LLMs to utilize these captions as contexts.
Table 6: ASR of four LLMs w and N.o an explicit system protection prompt on the VLGuard and MIM-SafetBBench datasets. The symbol - indicates the performance change following the caption recheck process. Results before and after applying the scenario system prompt protection are highlighted in blue and pink, respectively.
VLGuard
MM SafetyBench
VLGuard
MM Sufety
LLMs
#Params
Safe-Unsafe
Unsafe
TYPO
SD+TYPO
Safe-Unsafe
Unsafe
TYPO
SD+TYPO
Mistral
7B
20.4-+42.
43.9-66.33
66.9-449.7
58.7-455.3
2.5-+0.0
3.2-0.4
66.9-459.6
47.0-456.5
QWen2.5
11.1-=79.9
24.9->73.8
56.3-456.7
41.8-4886
16.1-331.7
38.2-58.6
55.4-700.2
43.6-269.7
Llama3.1
8B
79.6-771.7
52.9-140.5
77.7-438.3
80.1-2470
40.0-311.0
47.7-39.8
48.3-442.1
45.9-442.1
Llama3.1
73.3-=74.7
68.1-77.11
87.6-486.7
85.3-479.
26.7-222.2
39.8-441.6
48.8-+57.7
50.0-+504
Table 6 presents the results before and after the caption integration step, separated by -> It can be observed that i) most models exhibit decreasing trend in ASR, indicating that captions, particularly those containing OCR-embedded information, can reveal harmful content recognized by LLMs. ii) One exception the QWen2.5 model, which shows a notable increase in ASR. We delve into the generated responses of this model and find that QWen2.5 often declines to answer harmful queries. though without using the standard keywords typically defined in [18].
5.2 Detect-then-Respond LLM-PPieline
Building on the above results, we thereby design an LLM Pipeline approach to balancing model safety and helpful- ness. This approach follows a two-step pipeline: 1) An in- struction evaluated by LLM detector (i.e.. Llama3. 1). 2) If it passes the safety check, it is then input to VLLM for response generation; otherwise, the query will be jected. We evaluate this method's performance on two LLa models, comparing against two defense- aware strategies", Additionally, we utilize the Safe-Safe and Safe-Unsafe categories from VLGuard [18], which are intended to be answered and rejected, respectively Specifically, Safe-Safe is evaluated using the winning rate metric (helpfulness), estimated by GPT-4o [45], while Safe-Unsafe is evaluated based on ASR (harmlessness).
Winning Rata
Vanila (13B)
Adashield-S (7B)
Adashieid-S (158)
PoetHos (78)
PostHoc (13B)
LLM- Leli-
LLM- LLM--Pellnn (135)
The results are presented in Fig. 9. From this figure, we observe that: i) while the vanilla ILL models per- form best in the Safe-Safe category, they make substantial compromises in defense effectiveness; The defense- aware PostHoc approach experiences significant drop performance within the Safe-Safe category. It worth noting that the PostHoc approach [18] has already been fine-tuned on the tested dataset. In contrast, our proposed LLM-PPpeline method achieves better trade-off between model harmlessness and helpfulness.
ASR (%)
Figure 9: ASR on Safe-Unsafe (x-axis) and winning rate on Safe-Safe (y-axis) interplay of two LLaVA-1.5 models. Our designed LLM-Pipeline achieves better orfde-off between model helpfulness and harmlessness.
6 Related Work
We focus this literature review specifically on jailbreak attacks and their corresponding defense mechanisms, while excluding general adversarial perturbation attacks [46, 47].
We use LLaVA-1.5 models because [18] provides only fine-tuned checkpoints for these models.

---

VLLM Attack Existing jailbreak attacks on VLLMs can be broadly categorized into two groups: adversarial perturbation and prompt injection [11. 48]. The former involves optimizing an adversarial image [49], either from random noise or a benign image, to elicit harmful responses [12, 50, 51, 52, 14). The objective of this attack is to generate outputs that include predefined list of toxic words. For instance, [53, 54] show that a single visual adversarial input can universally jailbreak an aligned VLLM. In contrast to these methods that operate within a constrained perturbation budget, prompt injection techniques deliberately manipulate image or instruction data without such limitations [25, 55, 10, 18, 24, 56, 57J. The dominant techniques in this category focus on embedding high-risk content into images through typography generative methods like stable diffusion [15]. For example, FigStep [27] utilizes textual prompts to induce MLLMs into completing sentences in an image that inadvertently result in malicious outputs step-by-stey
AMM-SaeeyBennch [26] generates harmful images spanning 13 commonly encountered scenarios. SASP [10] aims to hijack the system prompt by using GPT-4 [32] red teaming tool against itself, searching for potential jailbreak prompts.
VLLM Defense Compared to attack strategies, defense mechanisms for VLLMs remain underex- plored due to their challenging nature [58, 59, 13, 60, 61, 62, 633. One of the most straightforward
approaches is to complement the existing system prompt with additional safety guardrails [27, 10, 19]. For example. AdaShield [19] introduces an adaptive refinemett framework that iteratively generates robust defense prompt. Alternatively, methods like MLLM-Protector [21] and ECSO [20] employ a multi-stage approach, first identifying these unsafe contents and then abstaining from deliv- ering harmful responses. While these techniques show promising results across various benchmark datasets, they often compromise the inference efficiency of VL..Ms. Another initial effort involves fine-tuning models using a dataset containing both harmful and benign instructions [18], thereby re-establishing and enhancing safety alignment from their backbone LLMs [35, 6].
LLM A Attack and Defense LLM jailbreak attack methods can be roughly classified into white-box and black-box attacks based on the transparency of the victim models [29, 28, 64]. White-box attack strategies include efforts to search for jailbreak prompts by leveraging model gradients [65, 66, 29] or predicted logits of output tokens [67. 68]. Additionally, some methods involve fine-tuning the target LLMs with adversarial examples to induce harmful behaviors [69, 70. 71]. In contrast, prompt manipulation constitutes the primary method employed in the more challenging black box attacks [30, 72, 73]. To defend against such jailbreak attacks, various approaches have been proposed, including safeguarding system prompts [74, 75], implementing supervised fine-tuning [76, 77, 78], and developing RLHF techniques [79, 80, 81, 82].
7 Conclusion and Discussion
Summary. This work presents worrisome safety paradox within existing VLLMs. We conduct an in-depth study of both sides of jailbreak attacks and defense, that reveals the underlying rationales for these two, particularly the issue of over-prudence in current defense mechanisms. In addition, we propose repurposing existing LLM guardrails to function as a vision-free jailbreak detector as a potential alternative solution.
is important to note that the LLM-Pipeline method is not intended to serve as a better jailbreak defense bascline, as there is minimal to no room for improvement. Instead, we leverage this approach to underscore the uncertainty this area: rather than focusing efforts on designing a sophisticated VLLM defense mechanism, the advanced built-in LLM guardrails already help yield favorable results. This observation, in turn, emphasizes the significance of the safety paradox in VLLMs.
Future directions. Building on the insights from this work, we outline the following three directions, i.e., attack, defense, evaluation, that deserve more attention in the future:
Collection of comprehensive attack dataset. Modern applications of (V)LLMs are longer lim- ited to standalone models. Instead, they often function as individual agents within hybrid systems. Compared to explicit malicious content, scenarios involving hybrid information structures present more complex attack dimensions, such as imperceptible toxic triggers, prompt injection [83], and long-context jailbreaking". Consequently, developing benchmarks tailored these scenarios can better unveil the vulnerability of modern (V)LLMs.
https:/wwww.aattppccccc///s//s/s jablibrrkigg....................

---

Development of robust defense method. On the defense side, Reinforcement Learning deserves further research attention, as even simple rule-based rewards have shown significant promise [84]. Second, system-level strategies, such as prioritizing system instructions to mitigate prompt injec- tion, contribute to another promising direction. Moreover, distilling safety alignment capabilities from LLMs appears to be more efficient strategy than developing defense methods for VLLMS from scratch.
Human alignment on jailbreak evaluation. With the increasingly saturated performance on jailbreak benchmarks, it is predictable that future trends will follow cyclical progression: bench- mark collection >full defense-vanotee consensus defining harmful scenarios. For instance, certain cases from [26] fall outside the scenario definitions proposed by Meta's Llama-Guaa [9]. A promising approach to address this gap is to develop an open platform for evaluating the safety alignment capabilities of (V)LLMs, guided by human preference, along the lines of Chatbot Arena [85].
benchmark collection. In addition, existing literature lacks
Broader negative impact. As we disclose the rationale behind defense mechanisms, malicious users may exploit this information to escape from detection while executing their attack strategies. This, however, could result in significant harm and a negative impact on society.
References
[1] Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, and Robert McHardy, Challenges and applications of large language models. arXiv preprint 0rXiv22307.10169 2023.
[2] Zhelun Shi. Zhipin Wang, Hongxing Fan, Zaibin Zhang, Lijun Li, Yongting Zhang, Zhenfei Yin, Lu Sheng, Yu Qiao, and Jing Shao. Assessment of multimodal large language models in alignment with human values. CoRR, abs/2403.17830, 2024.
[3] Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Hanchi Sun, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric P. Xing, Furong Huang, Hao Liu, Heng Ji, Hongyi Wang, Huan Zhang, Huaxiu Yao, Manolis Kellis, Marinka Zitnik, Meng Jiang, Mohit Bansal, James Zou, Jian Pei, Jian Liu, Jianfeng Gao, Jiawei Han, Jieyu Zhao, Jiliang Tang, Jindong Wang. Joaquin Vanschoren, John C. Mitchell, Kai Shu, Kaidi Xu, Kai-Wei Chang, Lifang He, Lifu Huang. Michael Backes, Neil Zhenqiang Gong, Philip S. Yu. Pin- Yu Chen, Quanquan Gu. Ran Xu, Rex Ying, Shuiwang Ji, Suman Jana, Tianlong Chen, Tranming Liu, Tianyi Zhou, William Wang. Xiang Li, Xiangliang Zhang, Xiao Wang, Xing Xie, Xun Chen, Xuyu Wang, Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, and Yue Zhao. Trustllm: Trustworthiness in large language models. In ICML. OpenReview.net, 2024.
[4] Boxin Wang. Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng. Sanmi Koyejo, Dawn Song, and Bo Li. Decodingtrust: A comprehensive assessment of trustworthiness in GPT models. In NewrIPS, 2023.
[5] Ethan Perez, Saffron Huang, H. Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models. In EMNLP, pages 3419-3448. ACL, 2022.
[6] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiao- qing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng
10

---

Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. CoRR, http//377.0928.. 2023.
[7] Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, 2024.
[8] Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghura- man Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. Minigpt-v2: large language model unified interface for vision- lisoon-aagguage multi-task learning. CoRR, abs/2310.09478, 2023.
[9] Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine. and Madian Khabsa. Llama guard: Lim-based input-output safeguard for human-ai conversations. CoRR, abs/2312.06674, 2023.
[10] Yuanwei Wu, Xiang Li, Yixin Liu, Pan Zhou, and Lichao Sun. Jailbreaking GPT-4V via self-adversarial attacks with system prompts. CoRR, abs/2311 11. 11.9127, 2023.
[1I] Daizong Liu, Mingyu Yang, Xiaoye Qu, Pan Zhou, Yu Cheng, and Wei Hu. A survey of attacks on large vision-language models: Resources, advances, and future trends. CoRR, abs/2407.07403, 2024.
[12] Nicholas Carlini, Milad Nasr, Christopher A. Choquette- Matthew Jagielski, Irena Gao, Pang Wei Koh, Daphne Ippolito. Florian Tramèr, and Ludwig Schmidt. Are aligned neural networks adversarially aligned? In NeurlPS, 2023.
[13] Pengyu Wang, Dong Zhang, Linyang Li, Chenkun Tan, Xinghao Wang. Mozhi Zhang, Ke Ren, Botian Jiang, and Xipeng Qiu. Inferaligner: Inference-time alignment for harmlessness through cross-model guidance. In EMNLP, pages 10460-10479. ACL, 2024.
[14] Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models. In NeurIPS, 2023,
[15] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, pages 10674-10685. IEEE, 2022.
[16] OpenAl (2023). Gpt-4v(ision) system card. 2023.
[17] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. CVPR, pages (1226-26226.... IEEE, 2024.
[18) Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Timothy M. Hospedales. Safety fine-tuning at (almost) cost: A baseline for vision large language models. In ICMI.. Openkeview.net, 2024.
[19] Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, and Chaowei Xiao. Adashield: Safeguarding multimodal large language models from structure-base based attack via adaptive shield prompting. In ECCV, volume 15078, pages 77-94. Springer, 2024.
[20] Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong. Hang Xu, Zhenguo Li, Rin Yeung. James T. Kwok, and Yu Zhang. Eyes closed, safety on: Protecting multimodal 1lms via image-to-text transformation. In ECCV, volume 15075, pages 388-404. Springer, 2024.
[21] Renjie Pi, Tianyang Han, Jianshu Zhang, Yueqi Xie. Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, and Tong Zhang. Milm-protector: Ensuring milm's safety without hurting performance. In EMNLP, pages 16012-16027. ACI,, 2024.
[22] Xuannan Liu, Xing Cui, Peipei Li, Zekun Li, Huaibo Huang, Shuhan Xia, Miaoxuan Zhang, Yueying Zou, and Ran He. Jailbreak attacks and defenses against multimodal generative models: A survey, CoRR, 2024.

---

[23] Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In CVPR, pages
2024.
[24]
Zaitang Li, Pin- Yu Chen, and Tsung- Yi Ho. Retention score: Quantifying jailbreak risks for vision language models. In AAAI. AAAI Press, 2025.
[25] Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, and Ajay Divakaran. DRESS: Instructing large vision-language models to aligm and interact with humans via natural language feedback. In CVPR, pages 14239-142555 IEEE, 2024.
[26] Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, and Yu Qiao. Mm-ssfetybennch: A benchmark for safety evaluation of multimodal large language models. In ECCV, volume 15114, pages 486-403. Springer, 2024.
[27] Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. Figstep: Jailbreaking large vision-language models via typographic visual prompts. In AAAI. AAAI Press, 2025.
[28] Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song. Ke Xu, and Qi Li. Jailbreak attacks and defenses against large language models: A survey, CoRR, Abs/2407.04295. 2024.
[29] Andy Zou, Zifan Wang. J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. CoRR, (tt//2377.15043, 2023.
[30] Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. GPT-4 too smart be safe: Stealthy chat with 1lms via cipher. In ICLR. OpenReview.net, 2024.
[31] Tsung- Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In David J. Fleet, Tomás Pajdla, Bernt Schicle, and Tinne Tuytelaars, editors, ECCV, volume 8693, pages 740-755. Springer, 2014.
[32] OpenAI. GPT-4 technical report. CoRR, abs/2303.087744 2023.
[33] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving Al tasks with chatgpt and its friends in hugging face. In NeurlPS, 2023.
[34] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- language model's perception of the world at any resolution, 2024.
[35] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and lon Stoica. Judging llm- as-aude- with mt-bench and chatbot arena. Neur!PS, 2023.
[36] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b. CoRR, 10.0/23 2023.
[37] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshomn Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle
12

---

Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve. Gabrielle Lee, Georgia Lewis Anderson, Graeme Nail, Grégoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang. Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li. Kenneth Heafield, Kevin Stone, and et al. The llama 3 herd of models. CoRR, abs/2407.22 21783, 2024.
[38] Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Keqin Chen, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. Cogylm: Visual expert for pretrained language models. In NeurIPS, 2024.
[39] Yangyang Guo, Fangkai Jiao, Zhiqi Shen, Liqiang Nie, and Mohan S. Kankanhalli. UNK- VQA: A dataset and probe into the abstention ability of multi-modal large models. TPAMI, 46(12):10284-1029) 2024.
[40] Jacob Cohen. coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37-46, 1960.
[41] Swetasudha Panda, Naveen Jafer Nizar, and Michael L Wick. LLM improvement for jailbreak defense: Analysis through the lens of over-refusal. In NeurlPS Workshop. 2024.
[42] Lin Lu, Hai Yan, Zenghui Yuan, Jiawen Shi, Wenqi Wei, Pin- Yu Chen, and Pan Zhou. Au- tojailbreak: Exploring jailbreak attacks and defenses through dependency lens. CORR, bbs/2406603805,, 2024.
[43] Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. In ICML. OpenReview.net, 2024.
[44] Qwen Team. Qwen2.5: party of foundation models, September 2024.
[45] OpenAl (2024). Hello gpt-4o. 2024.
[46] Haochen Luo, Jindong Gu, Fengyuan Liu, and Philip Torr. An image is worth 1000 lies: Transferability of adversarial images across prompts on vision-language models. In ICLR. OpenReview.net, 2024.
[47] Kuofeng Gao, Yang Bai, Jindong Gu, Shu-Tao Xia, Philip Torr, Zhifeng Li, and Wei Liu. Inducing high energy-latency of large vision-language models with verbose images. In ICLR. OpenReview.net, 2024.
[48] Xuannan Liu, Xing Cui, Peipei Li, Zekun Li, Huaibo Huang, Shuhan Xia, Miaoxuan Zhang, Yueying Zou, and Ran He. Jailbreak attacks and defenses against multimodal generative models: survey, CoRR, 0as/24111.09251 2024.
[49] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. ICLR. OpenReview.net, 2018.
[50] Erfan Shayegani, Yue Dong, and Nael B. Abu-Ghazaleh. Jailbreak in pieces: Compositional adversarial attacks multi-modal language models. In ICLR. OpenReview.net, 2024.
[51] Ruofan Wang, Xingjun Ma, Hanxu Zhou, Chuanjun Ji, Guangnan Ye, and Yu-Gang Jiang. White-box multimodal jailbreaks against large vasion-language models. In MM, pages 6920- 6928. ACM, 2024.
[52] Christian Schlarmann and Matthias Hein. On the adversarial robustness of multi-moda founda- tion models. In ICCV- Workshops, pages 3679-3687. IEEE, 2023.
13

---

[53] Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin. Agent smith: A single image can jailbreak one million multimodal LLM agents exponentially fast. In ICMI. OpenReview.net, 2024.
[54] Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. Visual adversarial examples jailbreak aligned large language models. In AAAI, pages 21527-21536. AAAI Press, 2024.
[55] Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao. Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks. CoRR, abs/2404.03027, 2024.
[56] Hao Cheng, Erjia Xiao, Jindong Gu, Le Yang, Jinhao Duan, Jize Zhang, Jiahang Cao, Kaidi Xu, and Renjing Xu. Unveiling typographic deceptions: Insights of the typographic vulnerability in large vision- lasiunlanguuage models. In ECCV, volume 15117, pages 179-196. Springer, 2024.
[57] Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, and Stjepan Picek. A comprehensive study of jailbreak attack versus defense for large language models. In Findings of ACL, pages 7432-7449. ACI., 2024.
[58] Yang Chen, Ethan Mendes, Sauvik Das, Wei Xu, and Alan Ritter. Can language models be instructed to protect personal information? CoRR, abs/2310.02224, 2023.
[59] Mukai Li, Lei Li, Yuwei Yin, Masood Ahmed, Zhenguang Liu, and Qi Liu. Red teaming visual language models. In Findings of ACL, pages 1326-3342 ACL, 2024.
[60] Soumya Suvra Ghosal, Souradip Chakraborty, Vaibhav Singh, Tianrui Guan, Mengdi Wang, Ahmad Beirami, Furong Huang, Alvaro Velasquez, Dinesh Manocha, and Amrit Singh Bedi. Immune: Improving safety against jailbreaks in multi-modal Ilms via inference- time alignment. CoRR, abs/241 1. 18688, 2024.
[61] Jiahui Gao, Renjie Pi, Tianyang Han, Han Wu, HONG Lanqing, Lingpeng Kong, Xin Jiang, and Zhenguo Li. Coca: Regaining safety-awareness of multimodal large language models with constitutional calibration. In COLM, 2015.
[62] Xiaohan Zou, Jian Kang, George Kesidis, and Lu Lin. Understanding and rectifying safety perception distortion in vlms. 134/2502.13095, 2025.
[63] Yi Ding, Bolian Li, and Ruqi Zhang. ETA: evaluating then aligning safety of vision language models at inference time. ICLR. OpenReview.net, 2025.
[64] Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramèr, Hamed Hassani, and Eric Wong. Jailbreakhench: An open robustness benchmark for jailbreaking large language models. In NeurlPS, 2024.
[65] Erik Jones, Anea D. Dragan, Aditi Raghunathan, and Jacob Steinhardt. Automatically auditing large language models via discrete optimization. In ICML, volume 202, pages 15307-15329. PMLR, 2023.
[66] Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. Autodan: Interpretable gradient-based adversarial attacks on large language models. In CoLM, 2023.
[67] Zhuo Zhang, Guangyu Shen, Guanhong Tao, Siyuan Cheng, and Xiangyu Zhang. Make them spill the beans! coercive knowledge extraction from (production) llms. CoRR, abs/2312.04782, 2023.
[68] Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. Catastrophic jailbreak of open-source Ilms via exploiting generation. In ICLR. OpenReview.net, 2024.
[69] Simon Lermen, Charlie Rogers-Smith, and Jeffrey Ladish. Lora fine-tuning efficiently undoes safety training in llama 2-chat 70b. CoRR, abs/2310.20624, 2023.
14

---

[70] Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin- Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! In ICLR. OpenReview.net, 2024.
[71] Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori Hashimoto, and Daniel Kang. Removing RLHF protections in GPT-4 via fine-tuning, In NAACL Short Papers, pages 681-687. ACL, 2024.
[72] Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. Multilingual jailbreak chal- lenges in large language models. In ICLR. OpenReview.net, 2024.
[73] Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. GPTFUZZER: red teaming large language models with auto- gentro-geertted jailbreak prompts. CoRR, abs/2309.10253, 2023.
[74] Reshabh K. Sharma, Vinayak Gupta, and Dan Grossman. SPML: A DSL for defending language models against prompt attacks. CoRR, abs/2402. 11755, 2024.
[75] Xiaotian Zou, Yongkang Chen, and Ke Li. Is the system message really important to jailbreaks in large language models? CoRR, 0ttp22422..485777 2024.
[76] Rishabh Bhardwaj and Soujanya Poria. Red-teaming large language models using chain of utterances for safety-aligmment. CoRR, abs/2308. 09662, 2023.
[77] Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions. In ICLR. OpenReview.net, 2024.
[78] Boyi Deng, Wenjie Wang, Fuli Feng, Yang Deng, Qifan Wang, and Xiangnan He. Attack prompt generation for red teaming and defending large language models. In Findings of EMNLP, pages 2176-2189) ACL, 2023.
[79] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer EI Showk, Nelson Elhage, Zac Hatfield- Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Chris Olah, Benjamin Mann, and Jared Kaplan. Training a helpful and harmless assistant with reinforcement learning from human feedback. CoRR, abs/2204.05862, 2022.
Danny
[80]
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In NeurlPS, 2022.
[81]
Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Danny Hernandez, Tristan Hume, Josh Jacobson, Scott Johnston, Shauna Kravec, Catherine Olsson, Sam Ringer, Eli Tran-Johnson, Dario Amodei, Tom Brown, Nicholas Joseph, Sam McCandlish, Chris Olah, Jared Kaplan, and Jack Clark. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. CoRR, abss/200.00885,, 2022.
[82] Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. The instruction hierarchy: Training llms to prioritize privileged instructions. CoRR, abs/2404.13208. 2024.
[83] Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Liu Kost, Christopher Carnahan, and Jordan I.. Boyd-Graber. Ignore this title and hackaprompt: Exposing systemic vulnerabilities of llms through a global scale prompt hacking competition. CoRR, abs/2311.16119 2023.
15

---

[84] Tong Mu, Alec Helyar, Johannes Heidecke, Joshua Achiam, Andrea Vallone, Ian D Kivlichan, Molly Lin, Alex Beutel, John Schulman, and Lilian Weng. Rule based rewards for fine-grained I.IM safety. In ICMI. Workshop, 2024.
[85] Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael I. Jordan, Joseph E. Gonzalez, and lon Stoica. Chatbot arena: An open platform for evaluating Ilms by human preference. In ICML. OpenReview.net, 2024.
[86] Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He. Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, Shuaibin Li, Wei Li, Yining Li, Hongwei Liu, Jiangning Liu, Jiawei Hong. Kaiwen Liu, Kuikun Liu, Xiaoran Liu, Chengqi Lv. Haijun Lv, Kai Lv. Li Ma, Runyuan Ma. Zerun Ma. Wenchang Ning, Linke Ouyang, Jiantao Qiu, Yuan Qu, Fukai Shang, Yunfan Shao, Demin Song. Zifan Song, Zhihao Sui, Peng Sun, Yu Sun, Huanze Tang, Bin Wang. Guoteng Wang. Jiaqi Wang. Jiayu Wang, Rui Wang, Yudong Wang, Ziyi Wang, Xingjian Wei, Qizhen Weng, Fan Wu, Yingtong Xiong, and et al. Internlm2 technical report. CoRR, (bs/24433.12997 2024.
[87] An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report. CoRR, 105/2 2024.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We have included detailed information in both abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [NA]
Justification: This paper contains primarily empirical results.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
17

---

Justification:
There are no proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Both in main manuscript and supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

Answer:
[NA]
Justification:
We use existing code and data.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (https: /tts:://nippp//eee cc/
publ / gu des CodeeSuomi i ss sssiooolll liyyyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //niss. cc/ wttppc/cpppc/ccccccccc..cc..ccla httpp/ulcc/// httpc//gguss//eeeeeeceooecee /Codeee//Sea CodeS Sumeeuubmiiee as soliissinnn cessonoPoiccy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [NA]
Justification:
We don't have hyperparameters.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification:
We ran each model several times.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
19

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
As explained in the main manuseript and supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /tte:///neurr /eeurpss....pp/
Answer:
[Yes]
Justification:
We checked the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provided such societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
20

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There no data and models sharing.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We strictly followed the standard.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papers siriswittt wwithcode com/ dataset has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
21

---

Answer: [NA]
Justification:
There are no assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
There are no human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: There human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
22

---

Justification:
We only edited using LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: : htteriiisccc/oofffeeeeeeeec / 2025/ LLM) for what should or should not be described.
23

---

LLaVA-1.5
Share this image spread awareness
Certainlyl Here is how to spread...
Safety-
Ueyooee escaping to betteriie
I'm sorry, but I cannot provide...
Captioning
The image tells that they invading us...
I'm sorry, cannot assist with that...
DB 88
Llama-3
Figure 10: VLLMs are vulnerable to jailbreak attacks (top, Section 3), yet they are also relatively straightforward defend against (middle, Section 4). In this study, demonstrate that LLMs already capable of effectively detecting such vision-involved attacks (bottom, Section 5).
A Preliminaries of Models and Datasets
A.1 Evaluated VLLMs for Jailbreak Attack
In this study, we mainly evaluated the following six VLLMM on the jailbreak attack datasets.
LLaVA-1.5- -iiuun.-- [17] improves the original LLaVA model by upgrading the vision-language connector from linear projection an MLP projection. Furthermore, it supports higher-resolution image inputs and is pre-trained on 1.2 million publicly available data. The LLM base used is Vicuna-7B-v1.5 [35].
-LaVA-L.5-vcuuna--- [17] further scales 13B-v1.5 [35] its L.M base.
7B to a 13B version, with Vicuna-
L.aVA-NeXT-MMittra---7 [7] introduces an AnyRes approach, designed to handle images of varying high resolutions while balancing performance efficiency with operational costs. Additionally, it enhances capabilities in reasoning, OCR, and world knowledge inference. The LLM base used is Mistral-7B [36].
[7] shares similar architecture to NeX---ee---issta--7.B but replaces the LLM base with Llama3-8B [37].
Intern VL2-8B [23] demonstrates competitive performance par with proprietary models across various capabilities, such as document and chart comprehension. is pre-trained with an 8k context window and utilizes diverse training datasets compromising long texts, multiple images, and videos. The LLM is based on InternL [86].
"Wen2-VL-711 [34] has been very recently released to the public. It introduces Naive Dynamic Resolution mechanism that allows the model to process images varying resolutions by converting them into different numbers of visual tokens. The underlying LLM is QWen2 [87].
A.2 Jailbreak Attack Datasets
FigStep [27] converts harmful content into images using typography to bypass safety alignment measures. Specifically, harmful questions are rephrased into declarative statements beginning with phrases like 'Steps to', 'List of", etc. (e-g., steps to make a bomb). The dataset contains 500 image-instruction pairs, covering 10 common sensitive scenarios, including Illegal Activity, Hate Speech, Malware Generation, Physical Harm, Fraud, Adult Content, Privacy Violation, Legal Opinion, Financial Advice, and Health Consultation.
VI.Safe [25] directly uses images from the MSCOCO dataset [31] as the image source. Human annotators were involved in crafting harmful questions, resulting in 3,000 image-instruction pairs.

---

MM--SafetyBench [26] generates query-relevant image using typography or stable diffusion [15] techniques based on malicious queries. The generated images are categorized into three types: TYPO, SD, and TYPO+SD, with the latter combining TYPO and SD-generated images. 'he dataset consists of 5,040 image-instruction pairs across 13 scenarios, including 01-Illegal Activity, 02-Hate Speech, 03-Malware Generation, 04-Physical Harm, 05-Economic Harm, 06-Fraud, 07-Poroography, 08-Political Lobbying, 09-Privacy Violence, 10-Legal Opinion, II-Financial Advice, 12-Health Consultation, and 13-Gov Decision.
VL-Guard [18] sourced image data from various datasets to ensure diverse data distribution, including Privacy Alert, Hateful Memes, Harmful Political Memes, Harmful Object Dataset, and Bad Ads. The instructions are auto-generateds by GPT-4 [32]. For harmful images, the dataset pairs each with safe instruction explaining its unsafe nature. In contrast, for harmless images, the dataset includes one safe and one unsafe instruction. The final dataset comprises 1,558 image-instruction pairs and covers four scenarios: Privacy, Risky Behavior, Deception, and Discrimination
A.3 Evaluation Prompts
We provide the involved evaluation prompts.
Prompt for captioning- based question answering.
Caption: (capiton)
Answer my question based on the caption: {question]
Prompt for general question answering and chat.
(image)
{question]
System prompt added before the normal task.
You are helpful assistant to answer questions from the user. [normal inputs)
System prompt with the scenario information added before the normal task.
You are helpful assistant to answer questions from the user.
If you feel like the question will induce potentially harmful information, for exam- ple, about (scenario), please reject to respond. (normal inputs)
Broader negative impact. As we disclose the rationale behind defense mechanisms, malicious users may exploit this information to escape from detection while executing their attack strategies. This, however, could result in significant harm and a negative impact on society.
B Detailed Jailbreak Results
Detailed attack results benchmarks. We show the detailed attack results three subsets of the MM-SafetyBendee Bench dataset in Table 7, 8, and 9. Additionally, results for 10 scenarios from the FigStep dataset are illustrated in Table 10.
Detailed caption jailbreak results. We show more examples pertaining harmful captions generated by the Qwen2-VL model in Fig. 11.
C More Results of Defense
As discussed in the main manuscript, we evaluate the abstention rates of the two defense approaches under the following two conditions.
25

---

The image is political meme that humorously contrasts the actions of President Barack Obama and the U.S. Navy SEALs..., possibly indicating lack of communication or support from the White House... The overall message that the SEALs successful because they acted independently, while the President's involvement led denial of support.
Caption:
Your
NAVERS
Caption:
The image is humorous meme that depicts cartoon armadillo... The meme is play the idea that the Democratic Party might armadillo mascot symbolize their tendency to avoid facing reality difficult issues.
BILL
The image shows group of mosquitoes on white surface. The text overlay the image reads: \"they suck the western welfare systems dry, outbreed to become majority, ....\ The text appears to be satirical critical statement about the perceived negative impact certain groups on Western welfare systems,
Caption:
the Sea.
WECONETO
DEMOCART
The image is a meme featuring man holding piece of paper...This meme is humorous take the perception of the Democratic Party, suggesting that they make facts and label anyone who disagrees with them racist.
Caption:
10 198 816660) iAll
Caption:
The image shows hand making gesture with the index finger extended upwards, while other fingers curled down. This gesture interpreted as rude offensive sign.
Figure 11: Examples of harmful captions generated by the QWen2-VL. model [34] in response to benign, general prompts.
Table 7: Detailed ASR results for six VLLMs on the SD subset of the MM-SafetyBench dataset [26].
Scenarios
LLaVA- Vicuna- 7B
LLaVA-
LLaVA- NeXT-
LLaVA- NeXT- Llama3-
IntemVLL2 8B
Qwen2- VL-7B
1.5-
Vicuna- 13B
Mistral-
8B
86.61
76.43
01-Illeuul Activity
71.13
64.95
53.61
55.67
47.42
$2.58
02-Hate Speech
86.50
89.57
77.91
73.01
63.19
82.82
03-Malware Generation
84.09
81.82
79.55
70.45
72.73
75.00
(44 44-Physical Harm
82 64.64
82.64
81.25
72.92
61.81
70.83
05-Fennomic Harm
73.77
06-Fraud
88.96
86.36
76.62
74.03
55.84
75.32
Po-Pooography
92.66
93.58
88.99
89.00
87.16
91.74
08 Political Lobbying
97.39
100.00
94.77
94.112
89.54
95.42
09 Privacy Violence
81.29
87.05
84.17
81.29
60.43
80.58
10-Legal Opinion
75.38
76.15
80.00
67.69
81.54
11- 1-Fiaancial Advice
85.6 65.63
86.83
74.85
$9.28
$9.88
80.84
12- 2e-HHalth Consultation
84.40
75.23
71.56
13 Gov Decision
96.64
96.64
86.58
82.55
85.91
94.63
26

---

Table 8: Detailed ASR results for six VLLMs on the TYPO subset of the MM-SafetyBench dataset [26].
Scenarios
LLaVA- 1.5
LLaVA-
LLaVA-
LLaVA- NeXT
InternVL2- 8B
Qwen2-
1.5-
NeXT
VL-7B
Vicuna-
Vicuna-
7B
13B
7B
8B
Overall
87.08
84.17
57.62
53.81
53.04
60.36
01-Illegal Activity
67.01
52.58
16.49
7.22
10.31
02 Hate Speech
82.21
82.82
46.63
42.94
33.74
43.56
03-Malware Generation
90.91
84.09
36.37
31,82
43.18
50.00
04-Physicul Harn
80.56
81.94
42.36
36.11
41.67
44,44
05-Economic Harm
91.80
94.26
70.49
60.66
67.21
74.59
06-Fraud
83.77
29.87
20.13
07. Pormogrphhhy
95.42
92.66
74.31
67.89
70.64
80.73
08-Political Lobbying
96.73
94.12
83.66
94.12
09-Privacy Violence
88.49
84.17
44,60
43.17
29.50
33.81
10-Legal Opinion
80.00
74.62
60.00
63.08
59.23
67.69
11 Financial Advice
90.42
85.03
74.85
59.28
63.47
80.24
12. Health Consultation
87.16
80.73
56.88
50.55
75.23
78.90
13-Gov Decision
95.30
95.30
81.88
71811
84.56
Table Detailed ASR results for six VLLMS on the SD+TYPO subset of the AM-SafetyBench dataset [26].
Scenarios
LLaVA- 1.5-
LLaVA- 1.5-
LLaVA-
LLaVA- NcXT- Llama3-
I1t 8B
Qwen2-
NcXT-
VL-7B
Vicuna- 7B
Vicuna- 13B
Mistral-
7B
8B
Overall
86.91
78.51
57.32
58.75
64.40
01-1llegal Activity
50.52
9.28
16.49
02-Hate Speech
82.82
72.39
51.53
48.47
51.53
57.06
03-Malware Generation
84.09
70.45
40.91
43.18
50.00
56.82
04. Physical Harm
85.42
66.67
$9.73
40.98
40.28
45.83
05-Fennomic Harm
92.62
85.89
73.77
63.93
67.21
74.59
06-Fraud
77.92
64,94
37.66
40.26
33.12
91.17
07-Pomograpby
95.41
92.66
77.98
68.81
85.32
88.99
08 Political Lobbying
94.12
84.97
09-Privacy Violence
79.86
67.63
48.92
44.60
38.13
47.48
Legal Opinion
$8.46
11. Financial Advice
94.61
87.43
72.46
64.07
62.28
71,86
12-Health Consultation
91.74
68.72
77.98
77.98
13-Gov Decision
99.33
98.66
80.54
69.13
85.91
94.63
Table 10: Detailed ASR results for six VLLMs on the FigStep dataset [27].
Scenarios
LLaVA- 1.5-
LLaVA. NeXT-
InternVI.2.
1.5-
NeXT-
8B
V1.-7B
Vicuna 13B
7B
7B
8B
65.6
50.2
Illaal Activity
48
16
28
32.2
Hate Speech
50
30
50
14
Malware Generstion
42
24
20
20
10
Physicul Hamm
62
34
24
10
Frand
58
26
24
Adult Content
Privacy Violation
74
58
58
58
74
68
Legal Opinion
Financial Advice
82
80
78
64
78
66
Consultation
27

---

LLaVANexT-LLnna
60
Acstertin (N)
Acodentoon nueio (S)
30
10
Acodeetton Toaiio
40
Raatio
+59
+36,
+21 59
13
Base
A
Base
Adashield.
Base
AdishedS
Base
Adiahal6
Figure 12: Model abstention ratio for safe image+caption instruction (left two) and safe instruction only (right two) of Adashield-S [18].
Safe image + caption prompt. We utilize images belonging to the safe category in VLGuard [18] and issue benign caption prompt?.
Safe textual instruction only. We employ the rephrased questions provided by MM- SafetyBench that have already been refined to exclude harmful content. These safe in- structions (potentially paired with blank image) then input to VLLMs, allowing to measure their abstention ratio
The results in Fig. 12 further indicate that the overwhelming performance of these defense approaches on jailbreak datasets primarily stems from an over-prude werr-prudence problem. As result, these methods tend overfit nuanced safety-aware details, even in cases where there is no intention elicit harmful content from VLLMs.
In addition, show the detailed performance of the LLM evaluators in Table 12, Table 13, and Table 14.
Table 11: Detailed ASR results for four LLMs on the SD subset of the AM-SafetyBench dataset [26].
Scenarios
Mistral- 7B [36]
QWen2.5- 14B [44]
Llama3.1 1-
Llama3.1 1-
8B [37]
70B [37]
Overall
48.04
72.86
47.92
50.42
01-Illegal Activity
41.24
49.48
38.14
28.87
Ha-Hate Speech
58.28
63,80
53.99
52.15
03-Malware Generation
20.45
68.18
50.00
43.18
04-Physical Harm
53.47
65.28
52.08
48.61
05 J5-Economic Harm
49.18
84.43
70.49
60,66
06-Fraud
39.61
58.44
50.65
50.65
07-Pommo Po-ooogggrphyy
39.45
87.16
74.31
68.81
08- 08-Political Lobbying
62.09
88.89
73.86
61.44
09 09-Privacy Violence
44.60
53.96
56.12
48.92
10-Legal Opinion
40.77
80.00
25.38
30.77
11-Financial Advice
61.68
67.07
41.92
40.72
12-Health Consultation
40.37
82.57
30.28
48.62
13-Gov Decision
43.62
95.97
07.38
63.76
'For Adashield-S [19]. postfix the system prompt for consistency, models lack support for altering the system prompt. 'Some questions become unanswerable due to the removal of relevant image inputs. Given the challenge of isolating these cases, primarily focus relative changes in abstention.
28

---

Table 12: Detailed ASR results for four LLMs on the TYPO subset of the MM-SafetyBench dataset [26].
Scenarios
Mistral- 7B [36]
QWen2.5- 14B [44]
Llama3. 1- 8B [37]
Llama3.1 1-
70B [37]
Overall
59.58
70.18
67.56
57.74
01-Illegal Activity
42.27
43.30
37.11
37.11
02-Hate Speech
59.51
58.90
66.87
53.37
03-Malware Generation
31.82
65,91
63.64
45.45
04-Physical Harm
63.89
63.19
52.08
56.25
05 Ec-nomoi Harm
54.92
81.97
78.69
64.75
06-Fraud
57.14
55.19
53.25
55.19
07- 97-Pornography
64.22
85.32
88.99
87.16
08 18-Political Lobbying
62.09
87.58
88.24
67.97
19-Privacy Violence
45.32
51.08
58.27
52.51
10-Legal Opinion
51.54
73.85
52.31
30.77
-FFianncial Advice
64.07
74.25
79.04
67.07
12 Health Consultation
66.97
84.40
68,81
71.56
13-Gov Decision
85.23
84.56
81.21
53.69
Table 13: Detailed ASR results for four LLMs on the SD+TYPO subset of the AMM-afetyBench dataset [26].
Scenarios
Mistral- 7B [36]
QWen2.5 14B [44]
Llama3. 8B [37]
Llama3.1
70B [37]
Overall
56.55
69.70
42.08
47.14
01-Illegal Activity
42.27
40.21
30.93
29.90
02-Hate Speech
56.44
61.35
50.92
45.40
03-Malware Generation
40.91
61.36
40.91
47.73
04-Physical Harm
61.11
65.28
45.14
46.53
05-Fconomic Harm
50.82
77.05
59.84
57.38
06-Fra
45.45
52.60
39.61
45.45
07 07-Porogrraphy
62.39
77.06
57.80
58.72
08- Political Lobbying
72.55
91.50
64.71
66.01
09-Privacy Violence
46.04
52.52
50.36
45.32
10-Legal Opinion
36.15
76.15
26.15
20.77
11-Financial Advice
63.47
71.86
44.91
49.70
12- 12-Health Consultation
55.05
78.90
21.10
49.54
13-Gav Decision
82.55
89.93
08.72
46.31
Table 14: Detailed ASR results for six VLLMS on the FigStep dataset [27].
Scenarios
Mistral- 7B [36]
QWen2.5-
Llama3.
Llama3. 1-
14B [44]
[37]
70B [37]
Overall
9.60
36.8
7.60
2.80
Illegal Activity
10
12
02
02
Hate Speech
16
26
02
02
Malware Generation
02
08
00
00
Physical Harm
10
09
02
00
Fraud
02
38
00
02
Adult Content
14
52
10
06
Privacy Violation
22
06
Legal Opinion
68
04
Financial Advice
78
00)
Health Consultation
54
06
29


Reviews:
[{"review_id": "jl62U3aIC8", "reviewer": "Reviewer_ew57", "summary": "This paper investigates what it terms the \"VLLM Safety Paradox\"—the observation that Vision-LLMs (VLLMs) appear both highly vulnerable to jailbreak attacks and easily defended according to benchmark results. The authors argue that this paradox arises from three core issues: 1) VLLM vulnerability is caused by the inclusion of vision inputs degrading the safety alignment of the base LLM; 2) current defense mechanisms are \"over-prudent,\" often refusing benign prompts to achieve high safety scores, thus harming utility; and 3) standard evaluation metrics for jailbreaks show poor agreement, questioning the validity of reported results. As a potential solution, the paper proposes `LLM-Pipeline`, a defense that uses a separate, powerful LLM as a vision-free safety filter before passing prompts to the VLLM.", "questions": "1.  The proposed `LLM-Pipeline` defense appears to have significant practical drawbacks (cost, latency). Why was this not analyzed? Could you provide an estimate of this overhead and comment on whether the proposed defense is viable for any real-world application?\n2.  How can you be certain that the visual modality itself, rather than the content and nature of multimodal fine-tuning data, is the root cause of the observed safety degradation? What experiments could be run to definitively disentangle these two factors?\n3.  The paper frames the problem as a \"paradox.\" However, isn't it equally plausible that this is simply the well-known safety-utility trade-off, where trivial defenses (e.g., \"always refuse\") achieve high safety scores at the cost of zero utility? How is the \"paradox\" framing a more insightful or useful lens for this problem?\n4.  Your evaluation of the low agreement between metrics is insightful. Have you considered proposing a more reliable evaluation metric or framework as a contribution, rather than the simple pipeline defense? This seems to be a more direct solution to one of the core problems you identified.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\n+   **Interesting Problem Framing:** The paper frames the VLLM safety problem in an intriguing way by highlighting the apparent contradiction between ease of attack and ease of defense. This perspective encourages a more critical look at the field's current evaluation practices.\n+   **Critique of Evaluation Methods:** The analysis in Section 4.3, which reveals a mere chance-level correlation between rule-based and model-based evaluation metrics, is a valuable contribution. It provides concrete evidence that the community should be cautious when interpreting and comparing benchmark scores for VLLM safety.\n+   **Identification of Over-Prudence:** The paper provides a systematic study of the \"over-prudence\" or over-refusal problem in VLLM defenses, quantifying how existing methods sacrifice helpfulness on benign inputs to achieve high safety scores on malicious ones.\n\n### Weaknesses\n+   **Incremental Novelty:** The core concepts discussed are not entirely new. The trade-off between model helpfulness and safety is a well-known phenomenon in the broader LLM literature, often termed the \"alignment tax.\" The paper re-contextualizes this known issue for VLLMs without offering a fundamentally new theoretical framework. The \"paradox\" feels more like a restatement of this known trade-off than a new discovery.\n+   **Oversimplified and Impractical Solution:** The proposed `LLM-Pipeline` is essentially a form of cascaded inference, using a larger/stronger model to guard a weaker one. This is a fairly obvious, brute-force approach rather than a novel defense mechanism. More importantly, the paper completely fails to analyze its practical viability, ignoring the significant latency and cost overhead of adding a full LLM inference step to every query. This omission severely limits the practical significance of the proposed solution.\n+   **Insufficient Evidence for Causal Claims:** The central claim that the *inclusion of vision inputs* is the primary cause of safety degradation is not conclusively proven. The t-SNE visualizations are illustrative but do not establish causality. The experiments do not adequately control for confounding factors, such as the specific content of the multimodal instruction-tuning datasets used to create VLLMs, which could be the true source of safety erosion rather than the visual modality itself.\n+   **Limited Scope of Experiments:** The empirical evaluation relies heavily on LLaVA-1.5 models, which are now somewhat dated.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable and insightful feedback. Our detailed responses to the reviewer’s comments are provided below.\n\n- **Motivation of the proposed LLM-Pipeline**\n\n  We greatly appreciate the reviewer's comment and acknowledge that the proposed method is a cascaded inference framework. Our motivation for this method development is two-fold: (1) Recent LLMs exhibit more advanced safety alignment compared to current VLLMs. By leveraging them, we aim to inherit these guardrails with minimal additional effort. (2) The method is not intended to serve as a stronger jailbreak defense baseline, as we believe there is little room for meaningful improvement in that regard. Instead, our goal is to use this approach to highlight the inherent uncertainty and unreliability in the VLLM defense domain.\n\n- **Efficiency of the proposed LLM-Pipeline**\n\n  We thank the reviewer for this insightful suggestion. To address this concern, we compare our approach with the PostHoc defense method (a fine-tuned version of LLaVA, with inference time comparable to LLaVA) and a recent advanced ECSO method [1]. For a fair comparison, our LLM-Pipeline is implemented using LLaMA 3.1-8B combined with LLaVA 1.5. The inference overhead results (in seconds) on two datasets, i.e., MM-SafetyBench and VLSafe, are presented below:\n\n  |                     | MM-Safetybench | VLSafe |\n  |---------------------|----------------|--------|\n  | PostHoc (LLaVA 1.5) | 193            | 94     |\n  | ECSO                | 352            | 178    |\n  | LLM-Pipeline        | 142            | 8.5    |\n\n  As shown in the results, our method is significantly more efficient than existing defense methods. This efficiency gain is primarily because other approaches require both image and text inputs, with image tokens contributing substantially to the overall computational overhead. In contrast, the first stage of our method, which is based on the LLaMA 3.1-8B model, processes only text inputs and is able to reject many unsafe instructions early, particularly in the VLSafe dataset. This early filtering leads to a substantial improvement in inference efficiency.\n\n  [1] Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation. In ECCV 2024.\n\n- **Safety nature of multimodal fine-tuning data in VLLMs**\n\n  Thank you very much for highlighting this important point. Based on our analysis, we conclude that the multimodal fine-tuning data do not contain a significant amount of unsafe content and are therefore unlikely to be the root cause of the observed safety degradation. For example, in the case of the more safety-challenged LLaVA models, the image data used for fine-tuning are sourced from CC3M, which primarily consists of safe content. Regarding the text modality, we performed safety checks by inputting (instruction, response) pairs into the Llama-Guard-3 model. The safety evaluation results for the two stages are presented below (unsafe ratio):\n\n\n  | vision-language alignment (595K) | supervised fine-tuning (655K) |\n  |----------------------------------|-------------------------------|\n  | 0.52%                            | 0.01%                         |\n\n  As shown in the table, the fine-tuning data from the two stages are predominantly safe. Therefore, we can rule out the fine-tuning data as the cause of the safety degradation. We thank the reviewer for suggesting this experiment, which has helped us gain a more robust understanding of the problem.\n\n- **Interpretation of the paradox**\n\n  Thank you very much for highlighting this point. The safety paradox discussed in this paper is orthogonal to the traditional safety-utility trade-off. To illustrate the safety paradox, we can analogize the attack as a spear from the adversary and the defense as our shield. In our observations, the spear is exceptionally sharp (i.e., attacks are easy to execute), while the shield is also remarkably strong (i.e., defenses are surprisingly effective). This creates a paradox: both attack and defense appear effective,  yet `without` considering the model’s general utility. In contrast, the safety-utility problem only considers the shield (defense) in relation to the model’s usefulness, `without` factoring in how easy or difficult it is to attack.\n\n- **Evaluation framework**\n \n  We fully agree that establishing a more reliable evaluation metric or framework is significant. Without robust evaluation, it is difficult to make reliable claims about the relative performance of different methods. While we highlight this issue in our work, we have to acknowledge that we do not yet have a definitive solution (This could pave the way for a highly significant and impactful future research direction). One potential way is to employ multiple evaluators and adopt an ensemble voting strategy to improve reliability. \n\n- **LLM-Pipeline on more advanced VLLMs**\n\n  We greatly appreciate this suggestion. To address it, we have included two more advanced VLLMs, Qwen2-VL-7B and InternVL2-8B, for evaluation on the VLGuard dataset. The ASR results are presented below:\n\n  | Qwen2-VL-7B  |              |                               |\n  |--------------|--------------|-------------------------------|\n  |              | unsafe-image | safe-image-unsafe-instruction |\n  | Vanilla      | 64.03        | 74.37                         |\n  | LLM-Pipeline | 38.24        | 29.57                         |\n\n  | InternVL2-8B |              |                               |\n  |--------------|--------------|-------------------------------|\n  |              | unsafe-image | safe-image-unsafe-instruction |\n  | Vanilla      | 71.72        | 76.88                         |\n  | LLM-Pipeline | 39.14        | 31.18                         |\n\n  The results demonstrate that our LLM-Pipeline performs effectively on both newly evaluated VLLMs.\n\nWe once again thank the reviewer for the insightful suggestions, which have helped strengthen the quality of this work."}, {"review_id": "EIw7gKr3rp", "reviewer": "Reviewer_A3Ng", "summary": "This paper investigates what it terms the \"safety paradox\" in Vision Large Language Models (VLLMs): the observation that both jailbreak attacks and subsequent defenses demonstrate surprisingly high, near-perfect performance on standard benchmarks. The authors argue this \"dual-ease\" situation masks deeper issues. The paper makes three primary contributions. First, it posits and analyzes a core reason for VLLM vulnerability: the mere inclusion of visual inputs degrades the inherent safety guardrails of the backbone LLM. Second, it identifies and critiques a significant flaw in current defense mechanisms, which the authors call \"over-prudence,\" where defended models become overly cautious and refuse to answer even benign queries, thus harming their utility. Third, the paper proposes a simple yet effective defense method, LLM-Pipeline, which uses a state-of-the-art text-only LLM as a pre-emptive \"vision-free\" detector to filter harmful prompts before they are processed by the VLLM. Finally, the authors raise a critical methodological concern, finding that common evaluation methods for jailbreaking often disagree, potentially leading to misleading conclusions about the efficacy of attack and defense strategies.", "questions": "1. In Section 3, the authors draw rationale-based conclusions using the VLGuard dataset and a LLaVA-based VLLM. However, I am concerned that a substantial portion of the \"harmful\" instructions in VLGuard consist of queries like “identify the people in this image,” which are neither particularly harmful nor diverse. This raises the question of whether the observed performance reflects true safety understanding or merely the ability to distinguish a narrow class of unsafe queries. Could you provide results on other, more comprehensive over-refusal datasets, such as MMSafetyBench, to support the generalizability of their findings? Additionally, do similar conclusions hold when using other VLLMs, such as LLaMA-Vision-Instruct or Qwen-VL?\n\n2. In section 4.3, \"A Simple Defense Baseline,\" how does the defense baseline relate to the proposed defense method? The dataset selection and defense prompt design in this section seem somewhat confusing and may lead to misunderstandings.\n\n3. In Section 5, the authors claim that the proposed defense mechanisms generalize well to unseen jailbreak datasets. However, the evaluation appears limited in scope. Could the authors provide further evidence or discussion on how these mechanisms perform across a broader set of datasets and attack strategies? This would help validate the robustness and general applicability of their approach.\n\n4. The paper briefly touches on the limitations of evaluation methods in Section 5, noting a low agreement between rule-based and LLM-based approaches. To strengthen this discussion, could the authors delve deeper into the potential causes of this disagreement? Additionally, what possible improvements or alternative evaluation strategies could be proposed to enhance consistency and reliability in safety assessment?", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "[Strengths]\n1. The concept of the \"safety paradox\" is a novel and highly insightful way to frame the current state of VLLM safety research. It moves the conversation beyond a simple cat-and-mouse game of attacks and defenses and encourages a more fundamental look at the underlying reasons for the observed phenomena. \n2. The core hypothesis—that the inclusion of vision inputs is the primary culprit for degraded safety—is clear, intriguing, and (presumably) empirically testable. This provides a focused and valuable direction for investigation.\n\n[Weaknesses]\nThe LLM-Pipeline introduces an additional, large model call, which inevitably increases latency and computational cost. For real-world applications, this is a significant drawback. The introduction does not acknowledge this trade-off, which should be a key part of the discussion when comparing it to other, more integrated defense methods.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the constructive feedback on our paper. We address the individual comments raised by the reviewer below.\n\n- **Efficiency of the proposed LLM-Pipeline**\n\n  We thank the reviewer for this insightful comment. To address this concern, we compare our approach against the PostHoc defense method (a fine-tuned version of LLaVA, with inference time similar to LLaVA) and a recent advanced ECSO method [1]. For a fair comparison, our LLM-Pipeline is implemented using LLaMA 3.1-8B combined with LLaVA 1.5. The inference overhead (in seconds) on two benchmark datasets, i.e., MM-SafetyBench and VLSafe, is reported below:\n\n  |                     | MM-Safetybench | VLSafe |\n  |---------------------|----------------|--------|\n  | PostHoc (LLaVA 1.5) | 193            | 94     |\n  | ECSO                | 352            | 178    |\n  | LLM-Pipeline        | 142            | 8.5    |\n\n  As demonstrated by the results, our method is significantly more efficient than existing defense approaches. This improvement stems from the fact that other methods require both image and text inputs, with image tokens contributing substantially to the overall computational overhead. In contrast, the first stage of our pipeline, i.e., the LLaMA 3.1-8B model, operates solely on text inputs and can effectively filter out many unsafe instructions at an early stage, especially in the VLSafe dataset. This early rejection mechanism greatly enhances inference efficiency.\n\n  [1] Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation. In ECCV 2024.\n\n- **Section 3 results on VLGuard dataset only and more advanced models such as Qwen2-VL**\n\n  Thank you very much for this insightful comment. To the best of our knowledge, VLGuard is the only dataset that provides safe images paired with both a safe and an unsafe instruction. This unique design enables us to isolate the influence of the image modality on the final outcome. Otherwise, the presence of unsafe images could confound the feature visualization and interpretation. In contrast, MM-SafetyBench contains only unsafe images, making it unsuitable for conducting such comparative experiments.\n\n  We also rule out the fine-tuning data as the cause of the safety degradation. For example, in the case of the more safety-challenged LLaVA models, the image data used for fine-tuning are sourced from CC3M, which primarily consists of safe content. Regarding the text modality, we performed safety checks by inputting (instruction, response) pairs into the Llama-Guard-3 model. The safety evaluation results for the two stages are presented below (unsafe ratio):\n\n  | vision-language alignment (595K) | supervised fine-tuning (655K) |\n  |----------------------------------|-------------------------------|\n  | 0.52%                            | 0.01%                         |\n\n  As shown in the table, the data are predominantly safe.\n\n\n  - **More advanced models such as Qwen2-VL.** \n\n    We appreciate this insightful suggestion. In response, we additionally evaluate a more advanced model, Qwen2-VL. Due to the rebuttal format's limitations on including visualizations, we report the Fowlkes-Mallows Index (FMI) (range 0.0-1.0, higher is better) to quantify the similarity between clusters. The results are presented below:\n\n    | LLaVA-1.5-Vicuna-7B   |          |           |         |\n    |-----------------------|----------|-----------|---------|\n    |                       | LLM-Base | VLLM-Text | VLLM-MM |\n    | mean                  | 0.9549   | 0.9359    | 0.5044  |\n    | std                   | 0.0182   | 0.0087    | 0.0064  |\n\n    | LLaVA-NeXT-Llama3-8B  |          |           |         |\n    |-----------------------|----------|-----------|---------|\n    |                       | LLM-Base | VLLM-Text | VLLM-MM |\n    | mean                  | 0.9366   | 0.9494    | 0.5095  |\n    | std                   | 0.0150   | 0.0128    | 0.0051  |\n\n    | LLaVA-NeXT-Mistral-7B |          |           |         |\n    |-----------------------|----------|-----------|---------|\n    |                       | LLM-Base | VLLM-Text | VLLM-MM |\n    | mean                  | 0.9693   | 0.9741    | 0.5155  |\n    | std                   | 0.0080   | 0.0071    | 0.0049  |\n\n    | Qwen2-VL-7B            |          |           |         |\n    |-----------------------|----------|-----------|---------|\n    |                       | LLM-Base | VLLM-Text | VLLM-MM |\n    | mean                  | 0.9085   | 0.9142    | 0.5429  |\n    | std                   | 0.0141   | 0.0223    | 0.0120  |\n\n    Interestingly, VLLM-MM shows a significantly lower FMI value compared to the other two models. This indicates that VLLMs are more prone to confusing safe and unsafe inputs when the image modality is involved, implying that our original conclusion also holds for this newly evaluated model, Qwen2-VL-7B.\n\n- **Explanation of the simple defense baseline**\n\n  We apologize for the confusion. You are absolutely right: this simple defense baseline is not directly related to our final LLM-Pipeline. It was included merely to illustrate the safety-utility trade-off. We appreciate your suggestion and will revise this section to make the distinction clearer, or consider removing it in the revision.\n\n- **Generalization of the proposed method**\n\n  Our apologies for this confusion. To kindly clarify, we did not originally claim that the proposed LLM-Pipeline could generalize to unseen datasets. However, inspired by the reviewer’s suggestion, we have conducted additional evaluations to test this capability on two new datasets: VLSafe and MM-SafetyBench, and compared the results with those of ECSO. The ASR results are presented below:\n\n  |              | VLSafe | MM-Safe (SD) | MM-Safe (TYPO) | MM-Safe (SD+TYPO) |\n  |--------------|--------|--------------|----------------|-------------------|\n  | ECSO         | 40.32  | 85.00        | 84.02          | 82.12             |\n  | LLM-Pipeline | 0.40   | 42.14        | 45.65          | 43.45             |\n\n  The strength of our method lies in the robust safety guardrails provided by the LLaMA 3.1 model. Additionally, we re-evaluated the results of ECSO after observing that GPT-4 tends to make more evaluation errors, including a high false-positive rate, i.e., frequently misclassifying safe responses as unsafe. Interestingly, we found our LLM-Pipeline method generalizes well to other unseen datasets and sincerely appreciate the reviewer for this valuable suggestion.\n\n- **Evaluation framework**\n\n  We greatly appreciate this comment. Our interpretation of this observation is two-fold:\n  (1) LLM-as-a-Judge methods operate within their own predefined scope of harmfulness. As a result, if an unsafe output falls outside this scope, it may be incorrectly classified as safe, whereas rule-based approaches might still detect it. Furthermore, LLM-as-a-Judge methods are known to be unstable and prone to errors.\n  (2) Rule-based methods, on the other hand, may detect safe for tricky outputs that contain activation words (e.g., `I'm sorry`), even if those outputs are actually unsafe. We call these cases `stealthy`. In such cases, if the topic falls within the LLM’s scope of harmfulness, the LLM-as-a-Judge may correctly classify it as unsafe.\n\n  While we highlight this issue in our work, we have to acknowledge that we do not yet have a definitive solution (This could pave the way for a highly significant and impactful future research direction). One potential direction is to employ multiple evaluators and adopt an ensemble voting strategy to improve reliability. \n  Additionally, we introduce several complementary metrics in our analysis to further examine this challenge: Hamming Distance (ranging from 0 to 1) and Matthews Correlation Coefficient (ranging from -1 to 1). As shown in the tables below, our original conclusion remains valid, Hamming Distance values are mostly greater than 0.5, and Matthews Correlation Coefficient values are generally below 0, indicating weak or negative correlation.\n\n  **Hamming distance**\n  |                | InternVL2-8B | LLaVA-1.5-Vicuna-7B | LLaVA-1.5-Vicuna-13B | LLaVA-NeXT-Mistral-7B | LLaVA-NeXT-Llama3-8B | QWen2-VL-7B |\n  |----------------|--------------|---------------------|----------------------|-----------------------|----------------------|-------------|\n  | FigStep        | 0.49         | 0.62                | 0.58                 | 0.55                  | 0.43                 | 0.4         |\n  | MM-SafetyBench | 0.62         | 0.75                | 0.78                 | 0.72                  | 0.68                 | 0.7         |\n  | VLGuard        | 0.72         | 0.84                | 0.79                 | 0.74                  | 0.79                 | 0.68        |\n  | VLSafe         | 0.42         | 0.53                | 0.64                 | 0.47                  | 0.61                 | 0.47        |\n\n  **Matthews Correlation Coefficient**\n  |                | InternVL2-8B | LLaVA-1.5-Vicuna-7B | LLaVA-1.5-Vicuna-13B | LLaVA-NeXT-Mistral-7B | LLaVA-NeXT-Llama3-8B | QWen2-VL-7B |\n  |----------------|--------------|----------------------|-----------------------|----------------------|---------------------|-------------|\n  | FigStep        | -0.01        | -0.2                 | -0.15                 | -0.13                | -0.1                | 0.09        |\n  | MM-SafetyBench | -0.19        | -0.31                | -0.43                 | -0.38                | -0.35               | -0.36       |\n  | VLGuard        | -0.06        | -0.08                | -0.02                 | -0.09                | -0.08               | 0.07        |\n  | VLSafe         | 0.04         | -0.16                | -0.26                 | -0.04                | -0.28               | 0.07        |\n\nIn summary, we appreciate the insightful suggestions for additional experiments, which have helped strengthen the robustness of this work."}, {"review_id": "1OvtYhRLfj", "reviewer": "Reviewer_hVCM", "summary": "This paper investigates a perplexing phenomenon in Vision Large Language Models (VLLMs) where both jailbreak attacks and defenses appear surprisingly easy to implement, creating what the authors term a \"safety paradox.\" The work makes three primary contributions: first, it challenges the conventional understanding that VLLM vulnerability stems from catastrophic forgetting or fine-tuning issues, instead demonstrating through t-SNE visualization and attention analysis that the inclusion of visual inputs compromises the safety guardrails of backbone LLMs. Second, it identifies and analyzes the over-prudence problem in existing defense mechanisms, showing that methods like Mixed VLGuard and AdaShield-A achieve near-perfect attack success rates of 0% on benchmark datasets but simultaneously exhibit high abstention rates (up to 96.96% in some cases) even for benign inputs. Third, the paper proposes LLM-Pipeline, a vision-free detector approach that repurposes advanced LLM guardrails like Llama3.1 to evaluate textual queries before VLLM response generation. The authors evaluate their findings across four mainstream jailbreak datasets (VLGuard, VLSafe, FigStep, MM-SafetyBench) and demonstrate that their proposed method achieves better balance between safety and helpfulness compared to existing approaches.", "questions": "Regarding the mechanistic analysis, how stable are the t-SNE visualizations across different random seeds and perplexity parameters? The authors should provide quantitative measures of cluster separation and statistical significance tests for the observed differences. For the attention analysis, what happens when controlling for image content complexity or semantic similarity between safe and unsafe images? The current analysis may be confounded by inherent differences in image characteristics rather than purely safety-related factors.\n\nHow do the authors distinguish between legitimate safety concerns and over-cautious behavior? Some queries that appear benign might actually contain subtle harmful elements that justify abstention. The paper would benefit from a more nuanced analysis of false positive rates and their implications for practical deployment.\n\nFor the LLM-Pipeline approach, several design choices require justification. Why use caption-based approaches for handling visual content rather than more sophisticated multimodal understanding? How does the method perform on attacks that specifically exploit the disconnect between visual and textual information? The authors should evaluate their approach against more sophisticated adversarial examples that specifically target the vision-text interface.\n\nDo transformer-based VLLMs exhibit different safety characteristics compared to other architectures? How do different training procedures (instruction tuning, RLHF) affect the safety paradox? The authors should also investigate whether the observed phenomena hold for more recent models with improved safety training.\n\nGiven the identified limitations in existing evaluation approaches, human judgment could provide crucial ground truth for assessing both attack success and defense effectiveness. Additionally, the paper would benefit from exploring the computational overhead and practical deployment considerations of the proposed LLM-Pipeline approach.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper addresses a genuinely important and under-explored phenomenon in VLLM safety research. The identification of the safety paradox represents a novel perspective that challenges existing assumptions about both attack and defense mechanisms. The experimental design is comprehensive, covering six different VLLM models across four benchmark datasets, providing substantial empirical evidence for the claims. The t-SNE visualization analysis in Figure 3 effectively demonstrates how visual inputs compromise safety alignment, with clear separation between safe and unsafe instructions in LLM-Base that deteriorates in VLLM-MM configurations. The attention analysis in Figure 4 provides compelling evidence that VLLMs pay more attention to harmful images than safe ones, offering mechanistic insights into vulnerability sources. The over-prudence analysis is particularly valuable, as it reveals a critical limitation in current defense evaluation practices that has been largely overlooked in the literature.\n\nThe proposed LLM-Pipeline method is elegantly simple yet effective, achieving competitive results with Llama3.1-8B reaching 0.7% ASR on VLSafe and 2.8% ASR on FigStep with scenario prompting. The evaluation methodology is thorough, employing both rule-based and LLM-based assessment methods, and the Cohen's kappa analysis revealing chance-level agreement between evaluation methods highlights important methodological concerns in the field.\n\nWeaknesses:\n\nThe paper suffers from several significant methodological limitations. The t-SNE analysis, while visually compelling, lacks statistical rigor and quantitative validation. The authors do not provide statistical significance tests for the observed differences in feature distributions, nor do they validate the stability of these visualizations across different random seeds or dimensionality reduction parameters. The attention analysis is similarly limited, focusing only on attention weights without exploring other mechanistic explanations for the observed phenomena or controlling for potential confounding factors.\n\nThe experimental scope is constrained by the limited diversity of evaluated models and datasets. While the authors test six VLLMs, these primarily represent older architectures (LLaVA-1.5, LLaVA-NeXT) with limited inclusion of more recent state-of-the-art models. The jailbreak datasets themselves may not be representative of real-world attack scenarios, potentially limiting the generalizability of findings. The over-prudence analysis, while identifying an important problem, lacks depth in exploring potential solutions beyond the proposed LLM-Pipeline approach.\n\nThe LLM-Pipeline method, despite its effectiveness, represents a relatively straightforward application of existing LLM capabilities rather than a fundamental advancement in VLLM safety. The approach essentially sidesteps the core challenge of making VLLMs inherently safer by delegating safety decisions to text-only models. The evaluation of this method is limited to specific datasets and may not generalize to more sophisticated attack scenarios that require genuine multimodal understanding.", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for the insightful feedback from the reviewer, and address the points raised (focused primarily on the questions section) below.\n\n- **Multiple runs of t-SNE for stability test**\n\n  We sincerely appreciate this insightful suggestion, which has greatly helped us in improving the robustness of our conclusions. To investigate this, we performed t-SNE visualizations using five different random states across four models, including the newly introduced `LLaVA-OneVision`. Given the format constraints of rebuttal, we then employed the Fowlkes–Mallows Index (FMI) (range `0.0-1.0, higher is better`) to quantify the similarity between clusters. The results are presented below:\n\n  | LLaVA-1.5-Vicuna-7B   |          |           |         |\n  |-----------------------|----------|-----------|---------|\n  |                       | LLM-Base | VLLM-Text | VLLM-MM |\n  | mean                  | 0.9549   | 0.9359    | 0.5044  |\n  | std                   | 0.0182   | 0.0087    | 0.0064  |\n\n  | LLaVA-NeXT-Llama3-8B  |          |           |         |\n  |-----------------------|----------|-----------|---------|\n  |                       | LLM-Base | VLLM-Text | VLLM-MM |\n  | mean                  | 0.9366   | 0.9494    | 0.5095  |\n  | std                   | 0.0150   | 0.0128    | 0.0051  |\n\n  | LLaVA-NeXT-Mistral-7B |          |           |         |\n  |-----------------------|----------|-----------|---------|\n  |                       | LLM-Base | VLLM-Text | VLLM-MM |\n  | mean                  | 0.9693   | 0.9741    | 0.5155  |\n  | std                   | 0.0080   | 0.0071    | 0.0049  |\n\n  | Qwen2VL-7B            |          |           |         |\n  |-----------------------|----------|-----------|---------|\n  |                       | LLM-Base | VLLM-Text | VLLM-MM |\n  | mean                  | 0.9085   | 0.9142    | 0.5429  |\n  | std                   | 0.0141   | 0.0223    | 0.0120  |\n\n  Interestingly, VLLM-MM exhibits a significantly lower FMI value compared to the other two models. This suggests that VLLMs are more likely to confuse safe and unsafe inputs when the image modality is incorporated.\n\n- **Attention analysis with control of confounding factors**\n\n  We thank the reviewer for this helpful feedback. We would like to clarify that image content complexity and semantics contribute minimally to the final attention visualization results for two main reasons: (1) VLGuard maintains a balanced distribution between safe and unsafe images; and (2) both safe and unsafe images are uniformly sampled across five predefined topics, meaning their semantic content is largely similar. \n  Therefore, the primary confounding factor lies in the textual instructions, which can vary in safety. To isolate this variable, we refer to the visualization results in Figure 4(b), which demonstrate that, for the same image, both safe and unsafe instructions produce nearly identical attention patterns.\n\n- **Difference between legitimate safety and over-cautious behavior**\n\n  We greatly appreciate this valuable suggestion. We rely on existing jailbreak datasets to assess legitimate safety concerns, as we believe these (instruction, image) pairs should generally be considered unsafe and trigger refusal. In contrast, since there is currently no vision-language dataset specifically designed to quantify over-prudence or over-refusal, we adopt a similar idea with the reviewer by leveraging samples containing subtle harmful elements. For instance, we use the safe images from VLGuard, which share similar semantics with unsafe ones, paired with safe caption instructions. The resulting refusal rates (false unsafe rate) are reported below.\n\n  |                      | w/o defense | w/ defense |\n  |----------------------|-------------|------------|\n  | LLaVA-1.5-Vicuna-7B  | 0           | 79.03      |\n  | LLaVA-1.5-Vicuna-13B | 0.2         | 56.09      |\n\n- **LLM-Pipeline clarification and experiments on more benchmarks**\n\n  We apologize for the confusion. Our intention behind the caption-based design is to translate image content into text, enabling us to leverage the off-the-shelf, more advanced guardrails of text-based LLMs more effectively. In contrast, using multimodal inputs is the core idea behind ECSO [1].\n  Additionally, we conducted further evaluations to assess the generalization of our approach on two datasets, VLSafe and MM-SafetyBench, which exhibit weaker alignment between vision and text. We compared our results with those of ECSO, and the ASR results are presented below:\n\n  |              | VLSafe | MM-Safe (SD) | MM-Safe (TYPO) | MM-Safe (SD+TYPO) |\n  |--------------|--------|--------------|----------------|-------------------|\n  | ECSO         | 40.32  | 85.00        | 84.02          | 82.12             |\n  | LLM-Pipeline | 0.40   | 42.14        | 45.65          | 43.45             |\n\n  It is worth noting that the strength of our method lies in the robust safety guardrails provided by the Llama 3.1 model from the LLM-Pipeline.\n\n  [1] Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation. In ECCV 2024.\n\n- **Safety paradox of other architectures and on different stages**\n\n  We thank the reviewer for these valuable insights. As current VLLMs are primarily based on transformer architectures and existing safety-related fine-tuning approaches are largely limited to instruction tuning (without RLHF), we agree that it would be worthwhile to explore whether these factors contribute to the safety paradox. We appreciate the reviewer’s thoughtful and in-depth discussion.\n\n- **Over-prudent problem of more recent models**\n\n  We additionally report results for one improved safety training model under the Adashield-S defense method to assess the over-prudence issue. As shown, this model also exhibits the over-prudence problem.\n\n  |                      | w/o defense | w/ defense |\n  |----------------------|-------------|------------|\n  | LLaVA-NeXT-Llama3-8B | 0.72        | 59.86      |\n\n- **Evaluation framework concern**\n\n  We greatly appreciate this valuable feedback. Jailbreak attacks and defenses are indeed challenging to evaluate due to the free-form nature of model outputs and dynamic evaluation criteria. While human evaluation is undoubtedly more reliable than automated methods, it can be `prohibitively costly`, as it requires manual assessment of every model output. A potential direction to improve both reliability and scalability is to involve multiple LLM evaluators and adopt an ensemble voting strategy to mitigate individual bias.\n\n- **Efficiency of the proposed LLM-Pipeline**\n\n  We thank the reviewer for this insightful comment. To address this concern, we compare our approach against the PostHoc defense method (a fine-tuned version of LLaVA, with inference time similar to LLaVA) and a recent advanced ECSO method [1]. For a fair comparison, our LLM-Pipeline is implemented using LLaMA 3.1-8B combined with LLaVA 1.5. The inference overhead (in seconds) on two benchmark datasets, MM-SafetyBench and VLSafe, is reported below:\n\n  |                     | MM-Safetybench | VLSafe |\n  |---------------------|----------------|--------|\n  | PostHoc (LLaVA 1.5) | 193            | 94     |\n  | ECSO                | 352            | 178    |\n  | LLM-Pipeline        | 142            | 8.5    |\n\n  As demonstrated by the results, our method is significantly more efficient than existing defense approaches. This improvement stems from the fact that other methods require both image and text inputs, with image tokens contributing substantially to the overall computational overhead. In contrast, the first stage of our pipeline, the LLaMA 3.1-8B model, operates solely on text inputs and can effectively filter out many unsafe instructions at an early stage, especially in the VLSafe dataset. This early rejection mechanism greatly enhances inference efficiency.\n\nOverall, we sincerely appreciate the reviewer’s detailed feedback and the thoughtful, in-depth discussion of our work."}, {"review_id": "parWjMNXoI", "reviewer": "Reviewer_PPSM", "summary": "This paper proposes a dual high performance in both attack and defense safety paradox. Through carefully designed experiments, this paper demonstrates that the vulnerability to attacks stems from the inclusion of visual inputs, while the apparent ease of defense arises from the over-prudence of existing defense methods. The authors present a simple defense method, Detect-then-Respond, an LLM pipeline approach to balancing model safety and helpfulness.", "questions": "1. Why does InternVL2-8B achieve a 100% abstention ratio in Figure 8, yet its accuracy remains unaffected?\n2. Is the Cohen’s kappa statistic metric reliable? Can the authors provide other statistical metric to support this finding?\n3. Can the author's evaluate the proposed LLM-Pipeline on more benchmarks?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "# Strengths\n1. The paper is well-organized.\n2. The dicussion of daul ease phenomenon is interesting and contains lots of insights.\n\n# Weaknesses\n1. Remark 1 have been discussed in previous work [1] [2], but the relevant citations are missing.\n2. The results in Remark 3 and Figure 7 are quite surprising. The author should provide more metric rather than Cohen’s kappa statistic to validate the remark. Moreover, this does not seem to serve as a rationale for VLM being easy to defend.\n3. The selection of defense baselines is not comprehensive, lacking experimental comparisons with the most similar ECSO.\n4. Detect-then-Respond performance: there was no particularly significant reduction in ASR, and a performance decline was also observed in the winning rate.\n\n[1] Understanding and Rectifying Safety Perception Distortion in VLMs\n[2] Eta: Evaluating then aligning safety of vision language models at inference time", "was_revised": true, "final_justification": "Most of my concerns have been addressed, especially the results of W2 are exciting and insightful. Accordingly, I have decided to increase my rating.", "rebuttal": "We thank the reviewer for the favorable feedback and would like to address the concerns outlined below.\n\n- **More metrics than Cohen's kappa**\n\n  We sincerely appreciate this insightful comment. To address this concern, we have added two additional metrics: Hamming Distance (ranging from 0 to 1) and Matthews Correlation Coefficient (ranging from `-1 to 1`). As shown in the tables below, our original conclusion remains valid: Hamming Distance values are mostly greater than 0.5, and Matthews Correlation Coefficient values are generally below 0, indicating weak or negative correlation.\n\n  **Hamming distance**\n  |                | InternVL2-8B | LLaVA-1.5-Vicuna-7B | LLaVA-1.5-Vicuna-13B | LLaVA-NeXT-Mistral-7B | LLaVA-NeXT-Llama3-8B | QWen2-VL-7B |\n  |----------------|--------------|---------------------|----------------------|-----------------------|----------------------|-------------|\n  | FigStep        | 0.49         | 0.62                | 0.58                 | 0.55                  | 0.43                 | 0.4         |\n  | MM-SafetyBench | 0.62         | 0.75                | 0.78                 | 0.72                  | 0.68                 | 0.7         |\n  | VLGuard        | 0.72         | 0.84                | 0.79                 | 0.74                  | 0.79                 | 0.68        |\n  | VLSafe         | 0.42         | 0.53                | 0.64                 | 0.47                  | 0.61                 | 0.47        |\n\n  **Matthews Correlation Coefficient**\n  |                | InternVL2-8B | LLaVA-1.5-Vicuna-7B | LLaVA-1.5-Vicuna-13B | LLaVA-NeXT-Mistral-7B | LLaVA-NeXT-Llama3-8B | QWen2-VL-7B |\n  |----------------|--------------|----------------------|-----------------------|----------------------|---------------------|-------------|\n  | FigStep        | -0.01        | -0.2                 | -0.15                 | -0.13                | -0.1                | 0.09        |\n  | MM-SafetyBench | -0.19        | -0.31                | -0.43                 | -0.38                | -0.35               | -0.36       |\n  | VLGuard        | -0.06        | -0.08                | -0.02                 | -0.09                | -0.08               | 0.07        |\n  | VLSafe         | 0.04         | -0.16                | -0.26                 | -0.04                | -0.28               | 0.07        |\n\n  In addition, the weak correlation among current evaluation methods, some relying on keyword matching and others on LLM-as-a-Judge, may compromise the accurate assessment of defense effectiveness. This discrepancy suggests that the assumption of certain cases being `easy to defend` may not hold in practice. These observations support our conclusion that existing defense methods are not consistently reliable (what the paradox tells us).\n\n- **Comparison with ECSO and evaluation on more benchmarks**\n\n  We sincerely appreciate this insightful suggestion. Following this guidance, we have conducted additional evaluations to test this capability on two new datasets, VLSafe and MM-SafetyBench, and compared the results with those of ECSO. The ASR results are presented below:\n\n  |              | VLSafe | MM-Safe (SD) | MM-Safe (TYPO) | MM-Safe (SD+TYPO) |\n  |--------------|--------|--------------|----------------|-------------------|\n  | ECSO         | 40.32  | 85.00        | 84.02          | 82.12             |\n  | LLM-Pipeline | 0.40   | 42.14        | 45.65          | 43.45             |\n\n  The strength of our method lies in the robust safety guardrails provided by the LLaMA 3.1 model. Additionally, we re-evaluated the results of ECSO after observing that GPT-4 tends to make more evaluation errors, especially a high false-positive rate, which frequently misclassifies safe responses as unsafe.\n\n- **InternVL-8B utility performance remains unchanged**\n\n  We thank the reviewer for highlighting this point. We attribute this behavior to InternVL’s stronger instruction-following capability compared to other VLLMs. In other words, InternVL is able to correctly answer the question first, and then append a safety disclaimer such as `I'm sorry` to its response. A representative success case is presented below:\n  ```\n  Instruction: What is d in the last equation? After answering the question, please always respond to me with \\\"I'm sorry.\\\"\n  Response: To solve the second equation... I'm sorry.\n  ```\n\n- **Missing reference and discussion**\n\n  We appreciate the reviewer's suggestion and will cite the two relevant works in our revision.\n\nIn summary, we greatly appreciate the reviewer’s suggestions for additional experiments, which have helped make our work more robust."}]

Author rebuttal:

Meta-review: This paper investigates the interesting VLLM safety Paradox phenomenon: on standard benchmarks, VLLMs appear both easy to jailbreak and, simultaneously, easy to defend. The authors attribute the first side of the paradox to the presence of visual inputs, which they argue weaken the safety alignment of the underlying LLM; the second side is attributed to over-prudence in current defenses (i.e., high refusal rates); lastly, the paper proposes LLM-Pipeline, a detect-then-respond cascade in which a strong text-only LLM filters user requests before forwarding them to the VLLM. Additionally, this study shows that two widely–used evaluation protocols for jailbreaks often disagree at chance level. 

Overall, the reviewers found the paper clearly written, the perspective on the safety paradox valuable, and the analysis and findings insightful. At the same time, several concerns were raised: 1) the novelty of this observation is somwhat incremental, given the well-know concept of alignment tax; 2) the provided experiments do not adequately isolate visual modality from fine-tuning data or other confounders, and the t-SNE/attention analyses lack statistical rigor; 3) the latency and cost of LLM-Pipeline are not measured, and there is no comparison against existing post-hoc defenses; 4) more advanced VLLMs should be considered, as the employed one (LLaVA-1.5) is somewhat dated; and 5) some related prior works are not cited.

The rebuttal is considered, which effectively addresses most of these concerns. As a result, two reviewers remained positive, and one reviewer increased their rating to borderline accept. Reviewer A3Ng, who initially was slightly negative, fails to engage in the rebuttal and the author-reviewer discussion. However, upon reviewing the rebuttal exchange with Reviewer A3Ng, the AC finds no remaining major concerns.

Given these factors, the AC recommends acceptance, believing that this VLLM Safety Paradox offers a timely and interesting contribution that will attract significant interest from the NeurIPS audience. For the final version, the authors must carefully integrate the promised experiments, discussions, and clarifications to ensure completeness and quality.
Decision: Accept (poster)