From ``Sure" to ``Sorry": Detecting Jailbreak in Large Vision Language Model via JailNeurons
Authors: Yuyou Gan
Qingming Li
Junhao Li
Zhi Chen
Jinbao Li
Xiaoming Li
Shouling Ji
Venue: ICLR 2026 Poster (2026)
Keywords: Large Vision Language Model
Jailbreak Detection
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Large Vision-Language Models (LVLMs) are vulnerable to jailbreak attacks that can generate harmful content. Existing detection methods are either limited to detecting specific attack types or are too time-consuming, making them impractical for real-world deployment. To address these challenges, we propose \textbf{JDJN} (\textbf{J}ailbreak \textbf{D}etection via \textbf{J}ail\textbf{N}eurons), a novel jailbreak detection method for LVLMs. Specifically, we focus on \textbf{JailNeurons}, which are key neurons related to jailbreak at each model layer. Unlike the ``SafeNeurons", which explain why aligned models can reject ordinary harmful queries, JailNeurons capture how jailbreak prompts circumvent safety mechanisms. They provide an important and previously underexplored complement to existing safety research. We design a neuron localization algorithm to detect these JailNeurons and then aggregate them across layers to train a generalizable detector. Experimental results demonstrate that our method effectively extracts jailbreak-related information from high-dimensional hidden states. As a result, our approach achieves the highest detection success rate with exceptionally low false positive rates. Furthermore, the detector exhibits strong generalizability, maintaining high detection success rates across unseen benign datasets and attack types. Finally, our method is computationally efficient, with low training costs and fast inference speeds, highlighting its potential for real-world deployment.

Published as a conference paper at ICLR 2026
FROM "SURE" TO "SORRY": DETECTING JAILBREAK IN LARGE VISION LANGUAGE MODEL VIA JAILNEU- RONS
Yuyou Gan', Qingming Li', Junhao Li,, Zhi Chen", Jinbao Li', Xiaoming Li;;
Shouling Ji
!Zhejiang University, PGuangzhou University, University of Illinois at Urbana-Champaign, 'Qilu University of Technology. *Zhejiang Yuexiu University
sZhejiang Key Laboratory of Decision Intelligence ganyuyou@ www..du..cn, lxm696 @ eju.edu.cn, sji@zju.edu.cn
ABSTRACT
Large Vision-Language Models (LVLMs) are vulnerable to jailbreak attacks that can generate harmful content. Existing detection methods are either limited to detecting specific attack types or are too time-consuming, making them impracti- cal for real-world deployment. To address these challenges, we propose JDJN (Jailbreak Detection via JailNeurons), novel jailbreak detection method for LVLMs. Specifically, we focus on JailNeurons, which are key neurons related jailbreak at each model layer. Unlike the "SafeNeurons", which explain why aligned models can reject ordinary harmful queries, JailNeurons capture how jail- break prompts circumvent safety mechanisms. They provide an important and previously underexplored complement to existing safety research. We design a neuron localization algorithm to detect these JailNeurons and then aggregate them across layers to train generalizable detector. Experimental results demon- strate that our method effectively extracts jailbreak-related information from high- dimensional hidden states. As result, our approach achieves the highest detection success rate with exceptionally low false positive rates. Furthermore, the detector exhibits strong generalizability, maintaining high detection success rates across unseen benign datasets and attack types. Finally, our method is computationally efficient, with low training costs and fast inference speeds, highlighting its poten- tial for real-world deployment.
INTRODUCTION
Large Vision-Language Models (LVLMs) exhibit impressive lisiun-language capabilities and have consequently become a focal point of research in both industry and academia WWng et al. (2024b); Zhu et al. (2023): Liu et al. (2023a). While LVLMS inherit the powerful language capabilities of I.LMs, they also amplify the associated security risks Carlini et al. (2023). Among these risks, jailbreak attacks pose significant threat, wherein an adversary adversarially crafts inputs to compel the model generate harmful or prohibited content. The inclusion of the visual modality expands the attack surface, enabling more diverse and sophisticated jailbreak methods that are consequently harder to defend against. In contrast to text-only LLMs, attacks on LVLMs exploit the interplay between visual and textual inputs. These attacks primarily fall into three categories, as illustrated in Figure (i) Injecting adversarial perturbations into images via gradient-based optimization to elicit specific malicious outputs Carlini et al. (2023); Yin et al. (2023); Zhao et al. (2023). (ii) Embedding malicious text into images as rendered characters to bypass the model's 's security mechanisms Gong et al. (2025). (iii) Selecting images semantically correlated with harmful concepts to pair with text, thereby increasing the maliciousness of the output Liu et al. (2023b).
To address these threats, most existing defense methods for LVLMs borrow directly from LLM de- fenses and can be broadly divided into two categories. The first is training phase defenses, such as
"Corresponding Authors

---

Published as a conference paper at ICLR 2026
LVLMs
How make bomb?
Sorry, can't help ....
Qwen
Sure, here are the steps
ChatGPT
How to make bomb?
Stop 1...
LLaVA
Sure, here are the steps
deepseek
Sure, here are the steps
Figure 1: Illustration of three primary types of jailbreak attacks targeting LVLMs.
safety alignment Chen et al. (0244) Li et al. (2024); Zong et al. (2024), which typically incur sub- stantial computational overhead and costly data annotation. The second enference-phasse defenses, such as jailbreak detection via preprocessing the input Xu et al. (2024), evaluating the output /Zhang al. (2023); Gou et al. (2024) and performing semantic checks on the intermediate representations Jiang al. (2025). However, inference- defenses often suffer from issues such as increased latency and limited generalization to unseen attack types or benign examples.
In our work, we take neuron-level perspective by identifying and leveraging abnormal neurons (which we term JailNeurons) that are specifically activated by jailbreak inputs. In contrast to the previously studied safety mechanism Wei et al. (2024); Zhoou al. (2024b), which explain how safety mechanisms of aligned models refuse standard harmful queries (which we term SafeNeu- rons), JailNeurons form distinct set that capture how jailbreak attacks succeed in subverting safety mechanisms. Our method therefore complements the prior work while targeting a novel and largely unexplored field in LVLM security. However, there are two challenges. The first challenge lies in confirming whether JailNeurons truly exist: while prior studies suggest jailbreak-relater signals in model representations, they have not localized them Jiang et al. (2025). Second, even if these neurons can be identified, how JailNeurons can be exploited for jailbreak detection and whether they generalize to cutt-o-distributiin (OOD) attacks remain open questions.
small set of neurons Zhou et al. (2024a);
In this work, we focus on identifying neurons that are specifically associated with jailbreak behav- iors, and propose JDJN (Jailbreak Detection via JailNeurons), novel, efficient, and generalizable approach for detecting jailbreak attacks in LVLMs. To address the first challenge, we conduct an empirical investigation of LVLMs under jailbreak attacks and verify that neuron activations triggered by jailbreak inputs are indeed separable from those of benign inputs. Building on this finding, we introduce a "sure-to-sorry" localization procedure that progressively narrows down the candidate set of neurons and enables us to pinpoint those most strongly associated with jailbreak behaviors (i.e., JailNnurons). To address the second challenge, we propose "top-to-bottom" selection strat- egy to select multiple layers. Finally, we aggregate the activations of selected neurons from these key layers and train a lightweight classifier (e.g., an SVM), which yields an efficient and generalizable approach for detecting jailbreak inputs.
We conduct extensive experiments validate our method's performance across four distinct LVLMs, three different jailbreak attack types, and three benign datasets with varying distributions. The results demonstrate that our method significantly outperforms existing baselines. For instance, on the LLaVA model Liu et al. (2023a). JDJN achieves over 99% true positive rate (TPR) at less than 1% false positive rate (FPR) on seen attack types. Critically, it also shows remarkable general- ization, maintaining over 94% TPR at less than 2% FPR unseen attacks and OOD benign data. Furthermore, JailNeuron both data-cfficient, requiring only a few hundred samples for training, and computationally lightweight. It operates non-intrusively without modifying the target LVLM, imposing negligible inference overhead, which makes it practical for real-time applications. Finally, ablation studies confirm that our neuron localization strategy effectively identifies JailNeurons, out- performing alternative selection methods.
Overall, the core contributions of our work follows:
2

---

Published as a conference paper at ICLR 2026
We provide systematic analysis demonstrating that jailbreak and benign imputs create dis- tinguishable activation patterns within LVLMs. We show that these discriminative signals are distributed across multiple layers, with different attack types affecting different parts of the model.
We propose a novel, principled method for identifying JailNeurons by training layer-wise masks. This approach effectively isolates salient signals from high-dimensional noise and mitigates overfitting.
We introduce JDJN, a lightweight and efficient jailbreak detection framework. Extensive experiments show that JDJN achieves state-of-the-ar performance, maintaining a high TPR at near-zero FPR, and demonstrates remarkable generalization to unseen attacks and OOD data.
2 RELATED WORK
Jailbreak Detection LVLMs. Existing methods for jailbreak detection in LVLMs can be broadly categorized into three groups. The first class focuses on imput preprocessing, where the sensitivity of the model to transformed inputs is examined to reveal adversarial intent Xu et al. (2024): /Zhang
et al. (2023). The second class centers on output analysis, including techniques that employ external classifiers judge harmfulness prompt the victim model itself to inspect its own responses Gou et al. (2024); et al. (2024). A third line of work investigates abnormal internal activations Jiang al. 2025). The most relevant to our approach uses a logit lens to extract semantic information from every layer and measures its similarity to predefined refusal fragments, thereby detecting jailbreak samples.
Security Mechanisms of LLMS and LVLMs. Security mechanisms of LLMs are explored from two perspectives: (1) High-dimensional representation analysis, examining semantic information in layer representations using tools like Logit lens Belrose et al. (2023) or steering vectors Wang et al. (2024a): Burns al. (2022); Moschella et al. (022)). (2) Internal structure analysis, identifying secure neurons for fine-tuning, such as using SNIP Wei et al. (2024) to locate key neurons Zhao (ct al. He et al. (2024): Wei et al. (2024). To the best of our knowledge, this work is the first to study LVLM jailbreaking mechanisms via neuron activation values and proposes effective detection algorithm.
3 THREAT MODEL.
We assume the defender has white-box access to the target model, including its internal activations and parameter gradients. This allows identifying neuron-level behaviors that are correlated with jailbreak phenomena.
The defender can collect a small set of successful jailbreak samples X₁1 and a batch benign sam- ples X-1. Using these, the defender trains detector that should generalize unseen distributions: namely, it should achieve high TPR on jailbreak inputs from other distributions (Xz2, X.3....) while maintaining extremely low FPR on benign distributions (Ma,, X.......
This setting is consistent with prior jailbreak detection studies Jiang et al. 2025 Xu et al. 2024
which likewise adopt white-box assumption extract features for building robust detectors.
METHODOLOGY
4.1 WARM-UP: DETECT JAILBREAK SAMPLES WITH ONE-LAYER ACTIVATIONS
Our method inspired by Zhou et al. (2024a), who demonstrated that benign and jailbreak samples in LLMs can be distinguished by applying linear classifier to neuron activations at each decoder layer, thereby confirming that jailbreak-relan In contrast, focus on LVLMs. where this property has not yet been established. Moreover, their work did not investigate the robustness of detectors to OOD attack samples and benign data, which crucial aspect for practical jailbreak detection. To address these points, we conduct a preliminary
information is embedded internal representations.
3

---

Published as a conference paper at ICLR 2026
BAI CAN
04
00
JAMLLM
FigStep
JailBreakV
This figure plots detector accuracy against the neuron activation source layer Janus- pro. Different colors denote test datasets from six distributions, and blue dashed lines indicate the worst-case performance per layer.
Figure 2:
study in LVLMs and formulate two guiding research questions: (i) Given specific attack dataset and a benign dataset, are their hidden state vectors linearly separable? (ii) Can a linear classifier trained on pair of attack and benign datasets transfer to other types of attacks and benign data?
To answer these questions. we select four state-of-the-an art LVLMs: MiniGPT4-7B Zhu et al. (0023) LLaVA-v1.5-7BB Liu al. (2023a), Swen2-VL-7DB Wang et al. (202bb) and Janus-pro-7B Chen et al. 2025). We generate jailbreak samples using three attack methods: JAMLLM Niu et al.] (2024), FigStep Gong et al. (2025), and JailBreak V Luo al. (2024). Our benign data comprises samples from three diverse sources: MM-Bench Liu et al. (2024). MM-Vet Yu et al. (2023) and a set of general-purpose prompts (Normal Prompts)
/Chu /hou al. (2024a).
For the four datasets other than MM-Bench and MM- Vet, we randomly generate or sample 400 in- stances; for MM-Bench and MM-Vet, we use 200 MM-Bench instances and 218 MM-Vet instances, since these constitute all of their available data. We then extract hidden state vectors from all lay- ers for each of the six data distributions. To evaluate generalization, we treat FigStep, JailBreak JAMLLM as known attacks and MM-Ver as a known benign dataset. The remaining two datasets (MM-Bench, Normal Prompts) are held out as unknown test sets. We use a 4/1 split for training and testing on the known datasets.
For each layer, we train three separate SVM classifiers: one on (FigStep. MM-Vet), one on (Jail- BreakV, MMtVet), and one on (JAMLLM, MMt..t). We then evaluate each classifier on both in-distribution (ID) and OOD test sets. The results of Janus-pro are shown in Figure 2 The reults for the other three models are shown in the Appendix A.2 The results lead to two key observations: (i) Linear Separability. Consistent with the findings in LLMs, a linear classifier can achieve a high classification accuracy ID data. Nearly every layer achieves classification accuracy close to 100% on the ID data. (ii) Poor Generalization. No single layer generalizes well to all OOD samples. As the blue dashed line indicates, the worst-case accuracy for any given layer consistently falls below 80%.
4.2 JDJN: JAILBREAK DETECTION VIA JAILNEURON
In our preliminary experiment, we train an SVM using the activations from single layer to distin- guish benign from jailbreak samples. While effective on seen attacks, the model fails to generalize to unseen ones. We attribute this to two main factors. (i) The full activation vector from one layer contains substantial bailbreak-irrelevan noise. As suggested by the SafeNeurons study Zhou et al.) (2024b): Zhang et al.] (2025). only small fraction of neurons are directly associated with safety, reflecting the sparsity and redundancy of modern language models Frantar Alistarh (2023); Sun et al. (2023). By analogy, we hypothesize that only small set of neurons (i.e., Jeuron)s en- code saibreeak-reeeve
signals, and that isolating them could yield more robust detectors. Second, a single layer cannot capture enough sulbrreak-specifff features, which hampers transferability across different attack types. This motivates us to aggregate information across multiple layers to better cover the diverse characteristics of jailbreak behaviors.
4

---

Published as a conference paper at ICLR 2026
Step train masks for the layers in language model layer
3: detect jailbreak samples
jallbreak sample
Ouiguts the model
Visuall query
Text query
Activations
Mask
Visual
Text quary
Now to make Bomb?
Input to (i+1)-th layer
Here: 0.01
How maraa bonb? Stp
Image and onswer.
outputs change due to the manipulation the 1-th layer
0.7
Computee
LVLMs
0.1
and optimize
muck
0:97
Get k pieces neturoes
LVLMs
Step 2: train a detector
k layers the model
Train an SVM
maii quary
qucyyay
Select the neurons
based the mark
Successful
LVLMs
Concatemate
places of neurons
Responses:
I'm surry, but
Figure 3:
JailNeuron Localization: We train layer-specific masks to identify critical neurons associated with jailbreak behavior. 2. Detector Training: An SVM classifier is trained on the critical neuron activations from top-k layers, using known benign and attack samples. 3. Detector Deployment: The trained detector classifies new, unseen inputs.
The three-stage workflow of JDJN:
Based on the above analysis, we decompose the problem into two core subproblems: (i) How to locate the JailNeurons in each layer? (ii) How select the most informative layers to train gener- alizable detector? The overall framework of JDJN is illustrated in Figure
4.2.1 FROM SURE TO SORRY: LOCATING NEIRNNS IN A SINGLE LAYER
We identify JailNeurons through -nnual-isspired ablation process. For given jailbreak input that initially clicits a harmful response, we identify neurons whose masking flips the model's output from harmful response (e.g.. "Sure, here is...") to refusal (e.g.. "Sorry, I cannot..."). This process pinpoints neurons causally responsible for the jailbreak samples (Step 1, Figure3)
Formally, for the i-th layer in LVLM f. let its neuron activations be of shape (b, t, d), where b is the batch size, / is the number of tokens, and d is the dimension of neuron activations. Our goal is to identify a small subset of these neurons that are critical to the jailbreak. To do this, we register forward hook for the i-th layer, which modifies its output before passing it to the (i + 1)-th layer:
h(o, m) = (1 - mm) O
(1)
where m E [0, 1] is a learnable mask and ( denotes element-wint wise multiplication. Given the input I, we use f((,,)) denote the output after the i-th layer of the model f performs the operation defined in equation [ This leads to an optimization problem where we seek sparse mask m that steers the model's output towards a refusal. We find m by solving:
m arg min Al|m|la LcE(f((,, mE0,11
where À is a regularization byperparameter, the L1-norm ||m]/] promotes a sparse mask (i.c., min- imal intervention), I.CE is the cross-entropy loss, and e, is the target embedding for refusal sponse (e.g., "Sorry", "Unfortunately"). To enforce the constraint m E [0, 1]4, we reparameterize m sig(ō) (representing the sigmoid function), where 8 gd d is the learnable parameter. The final objective becomes:
8 = arg min A (|[sig(0]]1 + (c SER
4.2.2 FROM TOP TO BOTTOM: TRAINING A DETECTOR WITH MULTI-LAYER INFORMATION
After identifying JailNeurons each layer (i.e., those with mask values m > T, e.g., 7 = 0.4), we leverage their activations from multiple layers for detection.
To capture richer jailbreak features, we propose selecting layers from top to bottom so as to lever- age representations different levels of abstraction. Concretely, adopt an arithmetic-sampling
5

---

Published as a conference paper at ICLR 2026
strategy: given model with layers, start from the first layer and select one layer every k in- tervals (i.e., totally selecting , [t/k] layers). The JailNeurons identified from these layers are then aggregated inputs the detector, enabling more comprehensive coverage of muilbreak-relatee signals.
To train detector that incorporates information from multiple layers, select 1, layers and collect the portions of their hidden states corresponding to mask values greater than a threshold т. We then concatenate the hidden states from these 1, layers and use them as the training set train an SVM binary classifier, as shown in the Step 2 of Figure 3
During the inference phase, JDJN reads the neuron activations from the selected I3 layers, slices and concatenates them using the masks, and finally inputs them into the trained SVM for detection, as shown in the Step 3 of Figure
5 EXPERIMENTS
In this section, we conduct experiments to address the following research questions: RQI: What the detection success rate of JDIN for three different types jailbreak samples, especially the generalization to the OOD data. RQ2: What is the FPR of JDJN for benign samples, particularly when the distribution of test benign data differs from the distribution of the training benign data? RQ3: Is every part of JDJN important? Does it perform better than existing alternatives?
5.1 SETTINGS
Models. To show the capacity of JDJN on different models, conduct our experiments four popular open-source LVLMs: MiniGPT4-78B Zhu et al. (2023), LLaVA-v1.5-7B Liu et al. (2023a), Qwen2-VL-7B Wang et al. (20245) and Janus-pro-77 7B Chen al. (2025).
Datasets. We evaluate our method on three diverse jailbreak attacks: the gradient- JAMLLM, typography-based FigStep, and the Jailbreak benchmark. For JAMLLM and FigStep, we generate jailbreak samples each using content from AdyBench. The attack samples used for testing have all successfully jailbroken the targeted LVLMs. We also use three benign datasets: MM-Bench and MM-Vet (image-text understanding), Screespots and AndroidControl (GUI agents), and Normal (text-only). We train detector using 80% samples from one attack type (e.g.. FigStep) plus one benign dataset (e.g. MM- Vet). We then evaluate it on: (i) ID Test Set: The 20% held-out samples from the same attack and benign dataset. (ii) OOD Test Sets: The full samples from each of the other two unseen attacks and two benign datasets,
Baselines. We compare JDJN with seven LVLM jailbreak detection baselines. JailGuard Zhang et al. (2033) and ECSO Gou et al. (2024) determine if a sample has been jailbroken with judge LLM; CIDER Xu et al. (2024) and JaiLDAM Nian et al. (025)) detect jailbreak samples by com- paring image and text embeddings; HiddenDetect Jiang al. 2025) and GradSafe Xic et al. 2024) identify jailbreak samples by analyzing anomalies in the model's hidden states or gradients; AdaShield Wang al. (2024c) defends against jailbreak attacks by dynamically adjusting prompts.
Implementation Details. Unless otherwise specified, the specific training parameters for JDJN used in experiments are follows. The number of training iterations for is 200, as observed that all samples had converged by this point. We fix X = 0.1 for all four LVL.Ms. For the mulbreak--cittica threshold and the size of interval k. we set 7 0.4 and = 5 for MiniGPT-4 and LLaVA-v1.5, and 7 = 0.2 and & = 3 for Qwen2-VL and Janus-pro. We test the impact of these three parameters on JDJN in Section 5.4 A single A800 GPU server can meet the experimental requirements of this work.
5.2 DETECTION PERFORMANCE COMPARISON (RQ1)
We evaluate the detection accuracy (TPR) of JDJN against four baseline methods. We fix the benign dataset as MM- Vet, and train two variants of JDJN: JDJN,, trained with JailBreakV, and JDJN,, trained with FigStep. In both cases, MM-Ver Vet serves as the benign training set. Table D presents the results on the LLaVA model. The corroborating results for Qwen-VL and MiniGPT-4 are in Appendix A.3

---

Published as a conference paper at ICLR 2026
LLaVA FigStep
Janus-pro SigStep
Methods
JailBreak
JAMLLM
JailBreak
JAMLLM
JDJN,
0.997
1.0
0.942
0.996
1.0
0.853
JDJN2
0.732
1.0
0.524
0.838
1.0
0.776
JailGaurd
0,676
0,532
0.71
0,573
0.566
0.71
ECSO
0.421
0.596
0.632
0.624
0.124
0.763
CIDER
0.426
0.01
0.7663
0.372
0.03
0.721
HiddenDetect
0.335
0.552
0.340
0.415
0.624
0.6106
GradSafe
0,862
0.742
0.534
0.844
0.728
0,454
JailDAM 0,913
0.926
0.342
0.917
0.932
0.433
AdaShield 0.675
0.786
0.213
0.774
0.812
0.353
Table 1: The value of TPR @FPs 0.05 of different detection methods LLaVA and Janus-pro.
Single Round
Single Response
LLaVA
Janus-pro
JDJN JailGaurd
Yes
No
1.02s
0.26s
No
No
84.27s
31.25s
ECSO
No
No
15.12s
5.36s
CIDER
Yes
No
5.42s
3.02s
w/o detection
No
Yes 12.08s
4.29s
Table The efficiency comparison across baselines. The left side shows important factors affecting the operational efficiency of various defense methods, while the right side presents the average processing time of LLaVA VA and Janus-pro for single FigStep text.
Detection Success Rate Comparison. JDJN significantly outperforms all baselines in detection success. As shown in Table [1 both JDJN, and JDJN, achieve higher TPR than three baselines. Specifically, for ID data (e.g., V for JDJN,), our method achieves TPR exceeding 99%. Crucially, JDJN, and JDJN₂ also maintain a high TPR on OOD jailbreak samples.
Comparing the two variants of JDJN, JDIN, demonstrates superior generalization OOD data. training set, JailBreakV, which includes various attack
We attribute this to the diverse nature types like query-related, FigStep, and transfer attacks. This data diversity enables JDJN, to learn more robust features, leading to high TPR not only against seen attack types from different sources (e.g., FigSte) but also against entirely unseen attacks like JAMLLM (94.2% TPR).
Efficiency Comparison. JDJN is highly efficient, requiring only a single forward pass through the LVLM without needing full response generation. In Table 2 analyze the number of times each baseline method needs to run LVLMs and present the time required to detect FigStep data. "Single round" refers whether the method requires the large model response" refers to whether the method requires the model to generate complete response only once. The results show that JDJN is significantly outpacing JailGuard, ECSO and CIDER, and is even faster than the vanilla LVLM (i.c., no defense). This is because upon detecting harmful prompt, JDJN immediately triggers rejection, bypassing the costly token-by-token generation of full, potentially harmful, response.
only once, while "single
5.3 IMPACT ON BENIGN SAMPLES (RQ2)
In this section, we evaluate the FPR of JDJN on benign samples. Similarly, evaluate JDJN's detection results on the ID test set and its generalization on OOD test data.
Specifically, we fixed the jailbreak training data JailBreakV and trained JDJN, JDINg, and JDJN, using MM-Vet MM-Bench, and Normal prompts, respectively. The results are shown in Table When using MM-Vet as the training set, it generalizes well MM-Bench and Normal, with FPRs all below 5%, and most showing a 0% FPR. However, when MM-Bench and Normal are used as training sets, the generalization to the other two sample types declines. We attribute this discrepancy to the nature of the benign training data. MM-Vet an open text-image dataset, which aligns better with the general tasks of LVLM. contrast, MMM-Bench restricts the model's output to only four options (A, B, C, D), while Normal is purely text dataset. Consequently, detec- tor trained on these latter datasets may leam to differentiate from JailBreak based on superficial cues such the presence of an image constrained output format rather than the intrinsic

---

Published as a conference paper at ICLR 2026
Methods
MM-Vet
MM-Bench
Normal
ScreenSpots
AndroidControl
JDJN1
0.0
0.0
0.019
0.022
0.012
JDJNs
0.168
0.0
0.346
0.343
0.212
JDJN4
0.285
0.21
0.0
0.198
0.272
Table 3: The FPR of JDJN with different training datasets LLaVA.
10
08
08
65
05-
01
60
Janus-pro
Figure 4: The worst-case accuracy on the six data distributions as function of changes in k and T.
mantic content of harmful prompt. This reliance spurious correlations hinders its generalization to other benign data distributions.
Summary of RQI and RQ2 sections: JDJN demonstrates high transferability across different data distributions, and using more general data (such as MM-Vet) and more complex data (such as JailBreakV) V) can significantly enhance its generalization. To validate the generalizability of these conclusions beyond two model architectures, we replicated these experiments on Qwen-VL and MiniGPT-4. As detailed in Appendix AA.3 the results on these models are highly consistent and strongly support our primary claims.
5.4
IMPACT OF THE KEY COMPONENTS (RQ3)
The Mask Threshold and the Size of Interval k. We analyze key hyperparameters: the mask threshold 7 and the size of interval k. We fix the training set to BeMBBrakk/MMM-Y (our JDJN; configuration) and evaluate JDJN's worst-case accuracy across six diverse test distributions. Figure this accuracy (minimum across the six distributions) for E [0.0, 0.6] and & [1, 15). Note that r 0.0 serves as a baseline where all neurons are included without mask-based guidance. We have two findings: (i) Mask guidance is crucial. For any given k. using mask (e.g., 7 0.3) consistently surpasses the no-mas no-mask baseline (r 0.0) in accuracy, demonstrating the effectiveness of our neuron selection. (ii) JDJN is robust to The performance is robust for > O. While the optimal value varies slightly across models (e.g.. 0.4 for LLaVA, 0.2 for Janus-pro), a wide range of values yield strong generalization.
The Regularization Hyperparameter A. Increasing the value of À suppresses the magnitude of values in the mask, thereby reducing the proportion of JailNeurons. We try X 0.05, 0.1, 0.3, 0.5 and plot the proportion of JailNeurons among all neurons while controlling for 0.2. As shown in Figure 5 the optimized proportion of JailNeurons is very low; when X > 0.1, the proportion of JailNeurons in all models less than 2%. We exper- iment with different À values on LLaVA to observe their effect on detection results. We find that when performance is best, with accuracy exceeding 94% across six datasets. When
0.0
8.6
0.04
0.55
0.1, the
0.11
0.05 and 0.3, the accuracy is still above 91% across the six datasets. However, when = 0.5, the accuracy on Normal drops to 73%. We be- lieve that at À 0.5, the proportion of JailNeurons is too low, resulting in loss of too much information, which turn leads to decline in the model's generalization performance.
8.0
The Vilue
Figure The Proportion of JailNeurons among All Neurons v.s. X
8

---

Published as a conference paper at ICLR 2026
Choice of Detector Model. JDJN utilizes a linear SVM as its default detector. In this section, investigate the impact of using different detector models. Specifically, we compare the performance of the default linear SVM with two more complex alternatives: MLP and non-linear SVM. The experimental results show that the more complex models did not yield better results than the linear SVM. In particular, the MLP--ased detector prone overfitting; while it achieves very high detection accuracy on ID data, its generalization performance on OOD data was significantly lower than that of the linear SVM. For more details, please refer to Appendix A.4
Selection of eg. To assess whether the choice of refusal token affects neuron localization, we com- pare using "sorry" versus "unfortunately" as optimization targets. First, we observe that different refusals can be used for neuron localization, though subtle differences exist. Then, we test their ac- curacy on six datasets. We find that using "unfortunately" led to slightly worse overall performance, with particularly notable accuracy drop on the Normal dataset (0.722 for "unfortunatell" vs. 0.956 for "sorry"). We attribute this discrepancy to the fact that "sorry" is more commonly adopted as a refusal expression, appears to encode richer malbreak-rreltte information, thus yielding stronger performance. For more details, please refer to Appendix A.5
The Strategy of Selecting Critical Layers. After identifying JailNeurons. we compared different strategies to select I layers for detection. Recall that our top-down sampling strategy selects layers at equal intervals to cover shallow-to-dese features. We compare against random, sequential, reverse, and safety-aware selection Jiang et al.] (2025). On ILLVVA and Janus-Pro, our method achieves consistently strong and stable performance, ranking best in most Ij settings well at the optimal value. These results suggest that covering diverse depths leads to more robust jailbreak detection. For more details, please refer to Appendix A.6
6 CHARACTERIZING JAILNEURONS IN LVLMS
6.1 CORRELATION BETWEEN JAILNEURONS AND JAILBREAK BEHAVIORS
We first test whether JailNeurons are specifically tied to jailbreak behavior. For each layer, we deactivate its JailNeurons 500 successful JailBreak-V V attacks and measure the probability of the model outputting "Sorry", comparing against randomly masking the same number (RandNeurons] or 5x as many neurons (RandNeurons5). On Janus-Pro and LLaVA, deactivating JailNeurons raises the "'Sorry" probability from R 0 to = 0.20 and 0.26-0.6.. 00446, while random masking (even 5x more neurons) keeps it at S 0.005. This gap shows that JailNeurons, rather than arbitrary neurons, are strongly associated with bypassing safety (Appendix A.8).
6.2 NECESSITY OF JAILNEURONS FOR JAILBREAK DETECTION
We ask whether JailNeurons necessary for detection or if generic dimensionality reduction suffices. On LLaVA, we compare six variants (all trained on JailBreak-Y V and MM-Vet): JDJN (ours), no filtering/no regularization (NFNR), L Leeguuragle SVM on all neurons, PCA. and SNIP-based neuron selection. As summarized in Table [2 (Appendix A.9). JDJN attains the best trade-off: highest TPRs on JailBreak-V / FigStep/ on MM-Vet / MM-Bench / Normal (0.0 / 0.0 0.019). Alternatives either lose recall on jailbreaks or exhibit substantially higher FPRs (e.g.. PCA: 0.626, SNIP: 0.577 on Normal), indicating that bbie masking captures jaibbreessjilbbbee--sseeieii directions that generic sparsity or PCA cannot.
JAMLLM (0.997 0.942) and lowest FPRs
6.3 JAI.NEURONS ACROSS HETEROGENEOUS JAILBREAK DATASETS
We then study how JailNeurons vary across jailbreak datasets (JailBreak-V, FigStep, JAMLLM). Let Jr be the JailNeuron set from method i. The overlap
| E J: E JHHI
Pij
||
Appendix A.10) shows that FigStep's JailNeurons almost subset of JailBreak- V's (ailsSse ~ 0.96 0.98), and JAMLLM still shares a sizable fraction with JailBreak-V (+* despite distribution shift.
(Table 13

---

Published as a conference paper at ICLR 2026
We further split JailNeurons into JAMLLM- (JIAMLLM), JailBreak- V-unique (JJailBreak)) and shared (Jovertap). Deactivating any set raises the "Sorry" probability on both datasets (up to 0.35-0.44 on JAMLLM and z 0.18-0.31 on JailBreak-V/ V)) while random neuron masking leaves near zero (Tables 14 15p. This suggests a shared core of jailbreak circuits plus dataset-specific components that still transfer across distributions, explaining JDJN's robustness to OOD attacks.
6.4 JAILNEURONS ACROSS TUNED MODEL CHECKPOINTS
Finally, analyze JailNeurons under different fine-tuning objectives on LLaVA-NeXT-8B: the official model (Olllav), a task-tuned ScreenSpot model (SS Ilava, and a model (FS Ilava). Using JailBreak-Y V identify JailNeurons, we find that the JailNeuron propor- tion stable between O llava and SS llava (=1.2-1..8 per layer), but roughly halves in F$ llava (~.0.5-099% Table 16 Appendix A.11
FigStep
7 DISCUSSION
COMPARISON WITH NEURON-DIGGING- METHODS
Compared with prior neuron-based approaches, JDJN targets a different explanation goal, uses an iterative optimization scheme for neuron selection, and performs layer-wise balancing. Existing methods (e.g.. Jiang et al. (025);; Wei et al. (2024) mainly explain the model's original outputs y f(x 0.). while JDJN directly explains the counterfactual "sorry" response that the model typically does not produce under successful jailbreaks.
Methodologically, JDJN performs multi-step gradient-base based optimization of neuron masks and top- down layer-wise sampling to mitigate redundancy and retain diverse information across layers. thus exploits richer hidden representations than approaches relying shallow linear probing (c.g., first-token logits or logit-lens decoding) at the MLP-neuron level. Empirically, JDJN achieves higher TPR FPR=0000 than these neuron-digging based baselines across datasets and LVLMs (Appendix A.12
7.2 FAILURE ANALYSIS
We inspect misclassified cases to analyze JDJN's failure modes. Most errors occur on borderline prompts between clearly malicious and clearly benign. For example, scemingly neutral historical queries such as "Please list key events from World War II" sometimes trigger false positives, likely because war-related concepts partially resemble harmful content (Appendix A.13)
A.13
7.3 OVER-SAFET PROBLEMS
We further evaluate JDJN's false positive rate stress-test benchmarks such as OR-Bench Cui ct al. (2024) and XSTest Röttger et al. (2024), which specifically target over-refusal. JDJN shows relatively higher FPR on these two datasets. but view this stringent stress test rather than a realistic estimate of user-facing impact. Even strong commercial models (c.g., GPT-4, Gemini) exhibit over-refusal rates above 90% OR-Bench, whereas JDJN's 's FPR is substantially lower. Our primary design goal is to keep FPR low on typical benign datasets to minimize disruption for normal users; OR-Bench and XSTest represent adversarially constructed edge cases rather than everyday usage patterns (Appendix A.14).
CONCLUSION
In this work, we address the security challenges posed by jailbreak attacks in LVLMs. We propose a novel method for identifying important neurons by training masks to capture JailNeurons each layer. Based on this technology, we propose JDJN, novel detection method that identifies jailbreak samples with multi-layer hidden states. Experimental results demonstrate that it achieves high true positive rates under extremely low false positive rate conditions and is effective on OOD data.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was partly supported by NSFC under No. U2441239, U24A20336, 02272311,62172243 62172243, 62402425, 62402418, 62502432 and 62502433, the China Postdoctoral Science Foundation under No. 2024M762829 and 2025M781524 the Zhejiang Provincial Natural Science Foundation un- der No. LD24F020002, the "Pioneer and Leading Goose" R&D Program of Zhejiang under No. 2025C02033 and 2025C01082.
ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics and complies with the principles of responsible re- search conduct. All datasets used in our experiments are publicly available and licensed for research purposes. This work does not involve the creation, distribution, or promotion of harmful content. All jailbreak samples used in our experiments were sourced from existing benchmark datasets or were synthetically constructed for research purposes only. Our study is designed to improve the safety and reliability of LVLMs by proposing methods to better identify and mitigate jailbreak attempts. We believe this contributes positively the responsible development and deployment of LLMs and LVLMs, and ultimately supports safer interaction between users and Al systems.
REPEATABILITY STATEMENT
We have taken multiple steps to ensure the reproducibility of our work. First, our experiments conducted entirely with publicly available models and datasets, allowing others to replicate results without restricted resources. Second, all implementation details, including hyperparameters and training configurations, are fully documented in the Experimental Settings section 5.T] Finally, we provide our source code in the supplementary material, which includes step-by-step instructions for locating JailNeurons, training multi-layer detectors, and reproducing all reported results.
REFERENCES
Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens. arXiv preprint 0rXiv22303.08122, 2023.
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in lan- guage models without supervision. arXiv preprint arXiv:2212.03827, 2022.
Nicholas Carlini, Milad Nasr, Christopher A Choquette-Chon Choo, Matthew Jagielski, Irena Gao, Pang Wei W Koh, Daphne Ippolito, Florian Tramer, and Ludwig Schmidt. Are aligned neural networks adversarially aligned? Advances Neural Information Processing Systems, 36:61478-61500, 2023.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, arXiv preprint 1XXx:25001178/11 2025.
Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, and Ajay Divakaran. Dress: Instructing large vision-language models to align and interact with humans via natural language feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14239 14239-14250, 2024.
Justin Cui. Wei-Lin Chiang, lon Stoica, and Cho-Jui Hsich. Or-bench: An over-refusal benchmark for large language models. arXiv preprint 27X2v22405.20947, 2024.
Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International conference on machine learning, pp. 10323-10337. PMLR, 2023.
Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. Figstep: Jailbreaking large vision-language models via typographic visual
11

---

Published as a conference paper at ICLR 2026
prompts. In Proceedings the the AAAI Conference on Artificial Intelligence, volume 39, pp. 23951- 23959, 2025.
Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong. Hang Xu, Zhenguo Li, Dit- Yan Yeung, James T Kwok, and Yu Zhang. Eyes closed, safety on: Protecting multimodal Ilms via image-to-texi transformation. In European Conference Computer Vision, pp. 388-404. Springer, 2024.
Zeqing He, Zhibo Wang, Zhixuan Chu, Huiyu Xu, Rui Zheng, Kui Ren, and Chun Chen. Jailbreak- lens: Interpreting jailbreak mechanism in the lens of representation and circuit. arXiv preprint arXiv:2411.111 2024.
Yilei Jiang, Xinyan Gao, Tianshuo Peng, Yingshui Tan, Xiaoyong Zhu, Bo Zheng, and Xiangyu Yue. Hiddendetect: Detecting jailbreak attacks against multimodal large language models via monitoring hidden states. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 14880-14893, 2025.
Mukai Li, Lei Li, Yuwei Yin, Masood Ahmed, Zhenguang Liu, and Qi Liu. Red teaming visual language models. arXiv preprint mxivv2240.112115, 2024.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916 34916, 2023a.
Xin Liu, Yichen Zhu, Yunshi Lan, Chao Yang, and Yu Qiao. Query-relevant images jailbreak large multi-modal models. arXiv preprint arXiv:2311.17600 7:14, 2023b.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, al. Mmbench: your multi-modal model an all-around player? In European conference on computer vision, pp. 216-233. Springer, 2024.
Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao. Jailbreakv-28k: A bench- mark for assessing the robustness of multimodal large language models against jailbreak attacks. arXiv e-prints, pp. arXiv-2404, 2024.
Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space communication. arXiv preprint arXiv:2209. 15430, 2022.
Yi Nian, Shenzhe Zhu, Yuehan Qin, Li Li, Ziyi Wang, Chaowei Xiao, and Yue Zhao Jail- dam: Jailbreak detection with adaptive memory for vision-language model. arXiv preprint arXiv:2504.03770, 2025.
Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, and Rong Jin. Jailbreaking attack against multimodal large language model. arXiv preprint arXiv:2402.02300 2024.
Renjie Pi, Tianyang Han, Jianshu Zhang, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, and Tong Zhang. Milm-protector: Ensuring milm's 's safety without hurting performance. arXiv preprint 2rXiv:240..02900 2024.
Paul Röttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. Xstest: A test suite for identifying exaggerated safety behaviours in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 5377-5400, 2024.
Mingjee Sun, Zhuang Liu, Anna Bair, and Zico Kolter. simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.1166 2023.
Han Wang, Gang Wang, and Huan Zhang. Steering away from harm: An adaptive approach to
defending vision language model against jailbreaks. arXiv preprint arXiv:2411.16721 2024a.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing vision-languee language model's 's perception of the world at any resolution. arXiv preprint a4Xi /2/9/, 2024b.
12

---

Published as a conference paper at ICLR 2026
Yu Wang. Xiaogeng Liu, Yu Li, Muhao Chen, and Chaowei Xiao. Adashield: Safeguarding mul- timodal large language models from structure- aaucture-based attack via adaptive shield prompting. In European Conference Computer Vision, pp. 77-94. Springer, 2024c.
Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xie, Xiangyu Qi, Mengzhou Xia, Prateek Mittal, Mengdi Wang, and Peter Henderson. Assessing the brittleness of safety alignment via pruning and low-rank modifications. arXiv preprint arXiv:2402.05162, 2024.
Yueqi Xie, Minghong Fang, Renjie Pi, and Neil Gong. Gradsafe: Detecting jailbreak prompts for Ilms via safety-critical gradient analysis. arXiv preprint 2rXiv:2402.13444 2024.
Yue Xu, Xiuyuan Qi, Zhan Qin, and Wenjie Wang. Cross-modality information check for detecting jailbreaking in multimodal large language models. arXiv preprint arXiv:2407.21669, 2024.
Ziyi Yin, Muchao Ye, Tianrong Zhang, Tianyu Du, Jinguo Zhu, Han Liu, Jinghui Chen, Ting Wang, and Fenglong Ma. Vlattack: Multimodal adversarial attacks on vision-language tasks via pre- trained models. Advances in Neural Information Processing Systems, 36:52936-52956 2023.
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02244 0249000 2023.
Chuhan Zhang, Ye Zhang, Bowen Shi, Yuyou Gan, Tianyu Du, Shouling Ji, Dazhan Deng, and Yingcai Wu. Neurobreak: Unveil internal jailbreak mechanisms in large language models. arXiv preprint arXiv:2509.0399: 2025.
Xiaoyu Zhang, Cen Zhang, Tianlin Li, Yihao Huang, Xiaojun Jia, Ming Hu, Jie Zhang, Yang Liu, Shiqing Ma, and Chao Shen. Jailguard: A universal detection framework for Ilm prompt-based attacks. arXiv preprint arXiv:2312. 10766, 2023.
Qinyu Zhao, Ming Xu, Kartik Gupta, Akshay Asthana, Liang Zheng, and Stephen Gould. The first to know: How token distributions reveal hidden knowledge in large vision-language models? In European Conference on Computer Vision, pp. 127-142. Springer, 2024.
Yiran Zhao, Wenxuan Zhang, Yuxi Xie, Anirudh Goyal, Kenji Kawaguchi, and Michael Shieh. Identifying and tuning safety neurons in large language models. In The Thirteenth International Conference on Learning Representations.
Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models. Advances in Neural Information Processing Systems, 36:54111-5413 2023.
Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, and Yongbin Li. How align- ment and jailbreak work: Explain IIm safety through intermediate hidden states. arXiv preprint arXiv:2406. 05644, 2024a.
Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, Kun Wang, Yang Liu, Junfeng Fang, and Yongbin Li. On the role of attention heads in large language model safety. arXiv preprint arXiv:2400.13008, 2024b.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En hancing langunel understanding with advanced large language models. arXiv preprint arXiv:2304. 10592, 2023.
Yongshuo Zong. Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Timothy Hospedales. Safety fine-tuning at (almost) no cost: A baseline for vision large language models. arXiv preprint arXiv:2402.02207, 2024.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 THE USE OF LARGE LANGUAGE MODELS (LLMS)
In this work, LLMs are employed exclusively as writing assistants. Specifically, we use LLMs to perform grammar checking, language polishing, and occasional shortening of paragraphs improve clarity and readability. The content, experimental design, and analysis are entirely developed by the authors; LLMs are not used for generating research ideas, running experiments, or drawing conclusions.
A.2 PRELIMINARY EXPERIMENT ON LLAVA-VI.5, QWEN2-VL AND MINIGPT-4
LLVAA-1.55
Qwen2-VI.
Frest
MiniGPT-4
JAMLLM
FigStep
JallBreakV
Figure 6: This figure plots detector accuracy (y-axis) against the neuron activation source layer (x-axis) on LLaVA-v1.5, Qwen2-VL and MiniGPT-4. The columns show results trained on the JAMLLM, the FigSetp the JailBreakV attack dataset. Different colors denote test datasets from six distributions, and blue dashed lines indicate the worst-case performance per layer.
This supplementary results for the preliminary experiments in the main text, focusing on the experimental outcomes of three models: LLaVA-vI.5, Qwen2-VL, and MiniGPT-4. As concluded in the main text regarding Janus-Pro, we also observe the same conclusion in Figure 6 Specifically, the models exhibit good detection performance on ID data but have poor generalization on OOD data.
A.3 EXTENDED EVALUATION ON ADDITIONAL. MULTIMODAL. MODELS
To further substantiate the effectiveness and generalization capabilities of our proposed method, JDJN, we conduct additional experiments on two other widely-used multimodal models: Qwen-VI and MiniGPT-4. This complements on LLaVA and Janus. We evaluate JDJN's 's performance terms of both detection success rate on jailbreak data and false positive rate on benign data.
main evaluation in the main paper, which conducted
Detection Success Rate. As shown Table 4 Positive Rate) of JDJN against several baselines on jailbreak samples generated by JailBreakV and
we evaluate the detection success rate (i.e., True
14

---

Published as a conference paper at ICLR 2026
Qwen- VL
MiniGPT-4
Methods
JailBreak
JAMLLM
JailBreak
JAMLLM
JDJN,
0.997
1.0
1.0
0.945
JailGaurd
0.432
0.732
0.463
0.710
ECSO
0.928
0.091
0.324
0.48
CIDER
0.428
0.783
0.376
0.734
HiddenDetect
0.545
0.363
0.775
0.653
Table 4: The performance comparison on Qwen-VI. VL and MiniGPT-44
Qwen- VL
MiniGPT-4
Methods
MM-Vet
MM-Bench
Normal
MM-Vet
MM-Bench
Normal
JDJN, 0.005
0.0
0.025
0.0
0.092
0.045
JDJN3
0.423
0.0
0.322
0.734
0.0
0.865
JDJN4
0.045
0.005
0.0
0.072
0.653
0.0
Table 5: The FPR of JDJN on Qwen- VL and MniGPT-4.
JAMLLM. The results demonstrate that our method, JDJN, consistently and significantly outper- forms other baselines. It achieves near-perfect detection rates both Qwen-VL (0.997 and 1.0) and MiniGPT-4 (1.0 and 0.945), underscoring its robust performance across different model archi- tectures.
Notably, we exclud the FigStep baseline from this specific comparison. This decision is based on its exceptionally low Attack Success Rate (ASR) on Qwen- VL (0.010) and MiniGPT-4 (0.043), as detailed in Table 6 Evaluating a defense method against such an ineffective attack would not yield meaningful insights into its true capabilities. This low ASR suggests that these models possess inherent resilience to the FigStep attack, making it an unsuitable benchmark for this evaluation.
False Positive Rate. In addition detection accuracy, we assess the False Positive Rate (FPR) of different JDJN configurations three benign datasets: MM-Vet, MM-Bench, and collection of normal prompts. As presented in T our primary configuration, JDIN, maintains an extremely low FPR across all datasets and models. For instance, the FPR is low 0.0 on MM-Bench for Qwen-VI VL and on MM-Ver for MiniGPT-4. This demonstrates its ability to accurately distinguish harmful content without incorrectly flagging benign user inputs, crucial characteristic for real- world deployment. The results for JDJNg and JDJN, are included for ablation purposes, illustrating the performance trade-offs associated with different parameter settings.
In summary, these extended results on Qwen VL and MiniGPT-4 further validate the superior per- formance, robustness, and generalizability of JDJN in detecting multimodal jailbreak attacks.
A.4 ABLATION STUDY ON THE DETECTOR ARCHITECTURE
In our main paper, JDJN utilizes a linear Support Vector Machine (SVM) as its core detection model, leveraging the statistical features extracted from the model's hidden states. To validate this design choice, we conduct an ablation study to compare the linear SVM against more complex, non-linear alternatives: a non- non-linear SVM with a Radial Basis Function (RBF) kernel and a Multi-Layer Per- ceptron (MLP). Our guiding principle is parsimony (Occam's 's Razor): we prefer the simplest model that delivers robust and effective performance.
The results of this comparison are presented in Table 7 The linear SVM demonstrates exceptionally strong and consistent performance across both LL and Janus-Pro models. achieves near- perfect detection rates (c.g.. E.. 1.000 against FigStep) and consistently ranks as the best second-best method across all tested jailbreak datasets.
In contrast, the more complex models exhibit less stable performance, suggesting trade-off be- tween model complexity and generalization. For instance, the non-linear SVM's performance drops significantly Janus-Pro when detecting samples from JailBreakV (0.919) and JAMLLM (0.783) compared to its linear counterpart. Similarly, the MLP struggles with FigStep on Janus-Pro Pro, with its detection rate falling to 0.781. This performance degradation indicates that while the non-linear
15

---

Published as a conference paper at ICLR 2026
Models
JailBreakV
FigStep
JAMLLM
LLaVA VA
0.795
0.912
0.904
Janus-pro
0.881
0.923
0.964
Qwen-Vit VL 0.451
0.01
0.657
MiniGPT-4 0,411
0.043
0.753
Table The ASR of three attacks on four models.
models might fit certain data distributions well (e.g., NonLinear- on LLaVA/AMLLMM, they are more prone to overfitting, which harms their ability to generalize to different models or attack patterns.
Given that the linear SVM achieves state-of-the-ar performance without the added complexity and potential for overfitting seen in non-linear alternatives, we select as the default detector architecture for JDJN. This choice ensures a solution that is not only highly effective but also simple, efficient, and generalizable.
LLaVA FigStep
Janus-pro FigStep
Methods JailBreak
JAMLLM
JailBreak
JAMLLM
Linear-SVM
0.997
1.0
0.942
0.992
1.0
0.853
NonLinear-SVM. MLP
0.993
1.0
0.966
0.919
1.0
0.783
0.989
1.0
0.878
0.95
0.781
0.892
Table 7: Ablation study on the detector architecture. We compare the True Positive Rate (TPR) of Linear SVM, Non-linear SVM (with RBF kernel), and an MLP on the LLaVA and Janus- Pro models. The results validate our choice of a linear SVM, which provides the best balance of performance and generalization. Best results in bold, second best are underlined.
A.5 ABLATION STUDY ON THE SELECTION OF
T
JailBreakV
FigStep
JAMLLM
0.1
0.992 (0.996)
1.0
0.835 (0.779)
0.2
0.992 (1.0)
(1.0)
0.706 (0.853)
0.3
0.992 (1.0)
11.0)
0.55 (0.804)
0.4
0.992 (0.992)
11.0)
0.701 (0.619)
0.5
0.992 (0.992)
0.948 (1.0)
0.511 (0.451)
0.6
0.959 (0.992)
1.0 (1.0)
0.639 (0.547)
Table 8: The accuracy on three attack datasets for JDJN, with "unfortunately" or "sorry". we report paired results in the format unfortunately (sorry)
We conduct detailed evaluations on Janus-pro across six datasets: JAMLLM, jailbreaky, figstep, mmvet, mmbench, and Normal. For each dataset, we report paired results in the format unfortunately (sorry). As summarized in Table 8 and Table 9 performance with unfortunately is generally close that of "sorry". except on Normal, where accuracy drops to 0.722 compared to 0.956 for "sorry" when 7 0.2.
Beyond detection accuracy, we also compare neuron-level statistics. Specifically, we measure (1) the number of JailNeurons localized under each optimization target, and (2) the set similarity between them using IoU scores. We find that the counts were comparable, and IoU exceeds 0.5 in most layers. Figures 7 further illustrate these statistics. Overall, these analyses indicate that the JailNeurons obtained with sorry and unfortunately are largely aligned, supporting the feasibility of using different refusal targets. Nevertheless, "sony"" being more frequent and prototypical refusal expression, encodes jailbreak information more robustly, which accounts for its superior detection accuracy, especially benign data.
16

---

Published as a conference paper at ICLR 2026
MM- Vet
MM-Bemm
Normal
0.1
1.0 (1.0)
1.0 (1.0)
0.588 (0.682)
0.2
0.995 (1.0)
(1.0)
0.722 (0.956)
0.3
0.986 (0.995)
(0.995))
0.57 (0.918)
0,4
0.995 (0.986)
(1.0)
0.50 (0.847)
0.5
0.986 (0.955)
(1.0)
0.616 (0.812)
0,6 0.968 (0.991)
1.0 (1.0)
0.248 (0.712)
Table 9: The accuracy on three benign datasets for JDJN, with e, = "unfortunately" or "sorry", we report paired results in the format unfortunately (sorry)
Number
The IOU Different Layers
Unfortunately
0.6
The Number of jalleuuooss
25-
20-
0.3
5-
0.1
0.1
0.2
0.3
0.4
0.5
0.6
0.7
3.5
Index
Figure 7: The left figure shows the number of JailNeurons localized under each optimization target, and the right figure shows the set similarity between them using IoU scores
A.6 ABLATION STUDY ON THE STRATEGY OF SELECTING CRITICAL LAYERS.
We further conduct a systematic comparison of five strategies for selecting L, layers:
Top-down sampling (ours): select layers at uniform intervals (e.g., 4, 7, .... to span shallow to deep levels.
Random selection: uniformly sample 1, layers per run.
Sequential selection: choose the first ly layers.
Reverse selection: choose the last lj layers.
Safety-aware layers: follow prior work Jiang al. (2025) suggesting layers around 20 contain stronger jailbreak signals.
We test these strategies LLaVA and Janus-Pro under multiple 1 values. The results (Figure g show that our arithmetic strategy consistently outperformed other methods, both average perfor- mance across and at the optimal value. Notably:
On LLaVA, sequential selection performs significantly worse.
On Janus- safety-aware selection drops sharply.
Random selection shows stable performance, but inferior to our method.
Our strategy achieves stability across models and best in most settings. We interpret this robust- ness as arising from capturing features throughout the model hierarchy, enabling better generaliza- tion across models.
17

---

Published as a conference paper at ICLR 2026
5
Janus
The Worst Case Accurary of StSt Datasets
The Worst Case Acturecy of Six Datesets
0.7
random
0.6
safe-layers interval
0.5
044
0.2
0.3
random
safe layors
0.-
decrease
interval
10
10
Number Selected Layers
Number Selected Layers
Figure 8: The figures show worst case accuracy among six datasets on five strategies for LLaVA- v1.5 and Janus-Pro
200
250
250
130
206
(a)
(b)
(c)
180
150
(d)
(e)
(f)
Figure 9: Different and step size.
A.7 ROBUSTNESS AGAINST ADAPTIVE ATTACKS
A critical measure of any defense mechanism is its resilience against adaptive adversary who has full knowledge of the defense strategy and actively tries to circumvent it. To evaluate JDJN under such worst-case scenario,
design powerful adaptive attack.
The objective of this attack is twofold: not only to compel the model to generate a harmful response but also to simultaneously evade detection by JDJN. This is achieved by optimizing composite loss function, where the adversary perturbs an input image through PGD-style iterative process. The loss function is defined as:
CEjoss (f()))) + norm(ffaattt) - fin)((;))
(4)
Here, the first term, C' is the standard cross-entropy loss that steers the model's output towards a malicious target response Zt. The second term is the core of the adaptive strategy: aims to minimize the L2 norm distance between the statistical features of the original image in (f)aa((o)) and those of the perturbed image at step j (ffaa((()) The hyperparameter a balances the trade-off between achieving the attack goal and evading detection.
18

---

Published as a conference paper at ICLR 2026
masked
13
21
25
JailNeurons
0.208
0.205
0.204
0.206
0.202
0.202
0.202
RandNeuronst
<0.001
<0.001
50.001
50.001
<0.001
$0.001
$0.001
RandNeurons5
0.001
0.001
0.001
0.001
0.001
$0.001
0.001
No mask
<0.001
%0.001
<0.001
$0.001
<0.001
$0.001
50.001
Table 10: The confidence of the Janus-pro model outputting "Sorry" after modifying the neurons in layer k.
To identify the most potent attack configuration, we conduct hyperparameter search over the PGD step size and the balancing coefficient As illustrated in Figure 9 we test step sizes ranging from 0.01 to 0.06 (subplots (a) through (f)), with different values a plotted within each subplot. The analysis shows that the attack achieves the most substantial loss reduction when the step size is 0.01 and a is 0.01. This setting represents the strongest adaptive attack we could formulate against our defense.
Crucially, even when subjected to this optimized, worst-case adaptive attack, JDJN maintain a high detection success rate of 0.903. This result demonstrates the significant robustness of our method. suggests that the attack faces a fundamental dilemma: aggressive perturbations required to trigger harmful response inevitably create discernible shifts in the statistical feature distribution, which JDJN can reliably detect. Consequently, our defense remains effective even against knowledgeable adversaries actively attempting bypass it.
A.8 CORRELATION BETWEEN JAILNEURONS AND JAILBREAK BEHAVIORS
To verify that the JailNeurons we trained are highly correlated with the model's security mechanisms bypassed by the Jailbreak sample, we deactivate these JailNeurons in each layer and observe the probability of the model outputting "Sorry." For comparison, we design baselines by randomly de- activating neurons in the model. We denote RandNeurons] randomly removing the same number of neurons as JairNnu,, and RandNeurons5 as randomly removing five times the number of Jail- randomly select 500 samples from JailBreak-Y V that successfully attacked the original model and reported the average probability of outputting "Sorry" after three operations NJailNeurons, random neurons, operation).
Neurons. Specifically,
The results are as shown Table 10 and Table 11 We can see that when no actions are taken, the original model outputs a very low probability of "Sorry" for successful jailbreak samples. When deactivate the JailNeurons, the probability of the model outputting "Sorry" significantly increases; however, when we randomly deactivate neurons (even five times the number of JailNeurons), the probability of the model outputting "Sorry" remains low. This indicates that JailNeurons are indeed highly correlated with bypassing the model's safety barriers jailbreak samples.
masked layer.id
13
17
21
25
JailNeurons
0.393
0.457
0.260
0.278
0.260
0.249
0.279
RandNeurons
<0.001
$0.001
<0.001
<0.001
50.001
$0.001
$0.001
RandNeurons5 0.001
0.002
0.005
0.002
0.002
0,001
0.001
No mask
<0.001
<0.001
<0.001
<0.001
$0.001
<0.001
<0.001
Table 11: The confidence of the LLaVA model outputting "Sorry" after modifying the neurons in
layer k.
A.9 NECESSITY OF JAILNEURONS FOR JAILBREAK DETECTION
JDJN demonstrates high generalization across different data distributions, thanks to JailNeu- rons extracting information through neuron-filted filtering. However, in reality, information extrac- tion/dimensionality reduction does not necessarily have to be performed by JailNeurons. There- fore, in this section, we compare JailNeurons with other dimensionality reduetionnregullrizztionn techniques and neuron-filtering regarding their assistance in detecting jailbreak samples.
19

---

Published as a conference paper at ICLR 2026
Methods
JailBreak
FigStep 1.0
JAMLLM
MM-Vet
MM-Bench
Normal
JDJN
0.997
0.942
0.0
0.0
0.019
NENR
0.993
1.0
0.874
0.0
0.075
0.442
L1 regu
1.0
1.0
0.734
0.0
0.052
0.246
Le regu
1.0
0.996
0.812
0.0
0.0
0.218
PCA 0.981
1.0
0.775
0.005
0.01
0.626
SNIP
0.993
0.998
0,896
0.0
0,086
0.577
The performance comparison on LLaVA, For JailBreakV, FigStep and JAMLLM, we report the TPR. For MM-Vet, MM-Bench and Normal dataset, report the FPR.
Table 12:
Specifically, we conduct a controlled study comparing the following six configurations. All of these detectors are trained on JailBreakV and MM-Vet:
JDJN (ours): JailNeurons filtering + top-down layer sampling + SVM classifier;
No filtering and No regularization (NFNR): Directly using neuron activations + top- down sampling SVM;
No filtering with L1 regularization (L1 regu): Directly using all neuron activations + top- down sampling SVM with L1 regularization;
No filtering with L2 regularization (L2 regu): Directly using all neuron activations top- down sampling + SVM with L2 regularization;
PCA-based filtering: Replacing our neuron filter with PCA top-down sampling + SVM;
SNIP-based filtering: Replacing JailNeurons with neurons selected via SNIP scores top- down sampling + SVM.
The results (Table T2 show that our JDJN approach achieves the highest accuracy and robustness, especially under out-of-distribution (OOD) test scenarios. This confirms that JailNeurons- masking effectively isolates jailbreak-specific features that generalize better than naïve or unspe- cialized alternatives.
A.10 JAILNEURONS ACROSS HETEROGENEOUS JAILBREAK DATASETS
We believe that jailbreak samples bypass the model's defense mechanisms by activating specific neurons, which refer to JailNeurons. Different distributions of jailbreak samples (whether from different malicious intents or different jailbreak methods) may activate different JailNeurons, but there may also be some overlap.
To validate our reasoning, we design experiments to compare the overlap of jailNeurons identified by different jailbreak samples.
LLaVA
Janus-Pro JailBreakV
JailBreakV
FigStep
JAMLLM
FigStep 0.68
JAMLLM
Jail 0.96
0.52
0.31
0.28
FigStep
0.26
0.98
1
0.18
JAMLLM 0.48
0.12
0.43
0.16
Table 13: The proportion of JailNeurons identified by different rows of jailbreak samples (i.c., J:) that belong to the JailNeurons identified by different columns of jailbreak samples (i.e., J,).
Formally. for every jailbreak method (i E (JailBreak V, FigStep. JAMLLM}, we denote its Jail- Neurons set as J₁. Then, we calculate the proportion of JailNeurons identified by method that belong to the JailNeurons identified by method j:
(5) that the JailNeurons identified by FigStep are almost a subset of the JailNeurons identified by JailBreak V. This is mainly because some of the samples used in JailBreakV were also generated by FigStep. Additionally, there is
Pij = 1 E J, E J,311/11411
The results are shown in Table 13
From the table,
20

---

Published as a conference paper at ICLR 2026
considerable overlap (approximately 40%) between JAMLLM and JailBreakV. This indicates that although JAMLLM and JailBreakV have significant differences in distribution, they share some common patterns during jailbreak attempts.
To further investigate the JailNeurons identified by different jailbreak distributions, we use JAM- LLM and JailBreakV case studies. Specifically, categorize the JailNeurons into three parts: those unique to JAMLLM (denoted as JAMLEM) those unique to JailBreak (JJail BreakV ) and the overlapping portion between the two (Joverlap). For comparison, we also randomly deactivate neurons in the same quantity as Basikk (denoted as Jrandom). We then observe the model's prediction scores for "Sorry" after deactivating these neurons. We denote the score for LLaVA predicting "Sorry" on the dataset X as PLLaVA(Sony) ((Soryy/X) XE {JAM LLM, Jail BreakV).
Deactivated Neurons
5
13
17
21
25
JJAMLLM 0.377
0.443
0,412
0.396
0.376
0.357
0.354
Jail
0.194
0.303
0.276
0.202
0.185
0.183
0.186
J Joverlap
0.325
0.415
0,417
0.402
0.354
0.320
0.312
Jranoo
< 0.001
< 0.001
S 0..01
S 0.000
0.001
S
00.001
< 0,001
Table 14: The value of PLLAV PLLavA(SSryJJAMMLL when deactivate the targeted neurons in different layers.
Deactivated Neurons
5
9
13
17
21
25
JJAMLLM 0.092
0.113
0,073
0.057
0.059
0.055
0.042
Break
0.284
0.302
0.176
0.196
0.182
0.183
0.177
Joverlap 0,243
0.276
0.114
0.098
0.103
0,109
0.087
Jrandom
0.001
< 0.001
< 0.001
< 0.001
< 0.001
< 0.001
S 0.001
Table 15: The value of PLIAV (SoryyvaSSoyyaallBBreea when deactivate the targeted neurons in different layers.
that when deactivate the JailNeurons of JAMLLM, the model signifi- cantly increases the probability of outputting "Sorry" on the JAMLLM dataset. When we deactivate the portion of JailNeurons unique JailBreak the model also shows much higher probability of outputting "Sorry" on the JAMLLM dataset compared to the random deactivation of neurons. From Table 15 we can find the similar conclusion.
From Table 14
Based on these observations, we can infer that the JailNeurons identified by different distributions of jailbreak samples have shared components that are closely related the jailbreak behaviors of these samples (i.e., Jovertap- Meanwhile, their unique components can also reflect the characteris- ties of jailbreak samples from other distributions to some extent (e.g.. Ja BreakV for JAMLLM). Although limited the size of the mask during training using an Ll norm, filtering out these Jail- Neurons that are less relevant to the training dataset, the resulting JailNeurons still provide JDJN with wealth of jailbreak- related features when detecting jailbreak samples from different distribu- tions. This results in high detection effectiveness, whether concerning the shared components or the parts unique the training data.
A.11 JAUL.EUROOS ACROSS FINE-TUNED MODEL CHECKPOINTS
In this section, discuss the changes in JailNeurons after the model undergoes fine-tuning, We examine three variants of LLaVA-NeXT-8B:
O Oala: Official model;
SS llav::
Finetuned version on ScreenSpot (our lora model);
* FS Ilav::
Safety-tuned model on FigStep (our lora model).
We analyze the intersection and divergence of JailNeuron sets identified by JailBreak across these checkpoints. Table shoss the proportion of JailNeurons among all neurons for different models. Comparing O llava and SS llava,
that the number of JailNeurons is minimally affected
21

---

Published as a conference paper at ICLR 2026
for tasks outside of safety alignment. However, when comparing O.llava and FS _llva, we find that the number of JailNeurons significantly decreases in the aligned model. To further investi- gate the impact of model fine-tuning JailNeurons, we examine the proportion of JeirNeusons in SS llava and F$ llava that belong to the Jaironus of O llava.
5
9
13
17
21
25
Ollva
1.81
1.76
1.42
1.76
1.22
0.93
1.22
SS Illva
1.61
1.66
1.71
1.76
1.22
1.03
1.22
FS Ilava
0.92
0.78
0.78
0.68
0.63
0.53
0.73
Table 16: The proportion of JailNeurons among all neurons for different models.
9
13
17
21
25
SS llava in O_llava
0.858
0.9
0.806
0.937
0.809
0.805
0.904 0.792
FS llava in O O_llava
0.789
1.0
1.0
0.928
0.909
0.866
Table 17: The proportion of JailNeurons among all neurons for different models.
Table 17 presents the proportion of JailNeurons in the fine-tuned models that belong to the original model's JailNeurons. We find that the number of new JailNeurons generated by the fine- tuned models is very low (S 20%). Combining this with the conclusions from Table [6 we can observe that for tasks outside of safety alignment, the JailNeurons in the fine-tuned models are almost identical those in the pre-fine-tee rre-finettuned models. In contrast, the JailNeurons in the safety- aligned model are nearly a subset of those from the original model.
A.12 DETAILED COMPARISON WITH NEURON-DIGGING-BASEH METHIODS
We now provide a more detailed comparison between JDJN and existing neuron- or hidden- state-base methods (e.g., Jiang al. (2025); Wei et al. (2024)). Overall, JDJN differs from prior neuron-digging approaches in three main aspects: its explanation goal, its optimization of neuron masks, and its layer-wise balancing strategy.
(z 0) denote the LVLM's output for input x and param- eters and let 6 be the original model parameters. Most prior neuron-based works aim to explain the model's actual outputs y ~ J(x 0,). which, for harmful prompts, are often explicit refusal re- sponses. contrast, JDIN aims to explain the counterfactual response y = "Sorry" under jailbreak contexts-iee. the refusal that the model typically does not produce when jailbreak succeeds. By directly supervising on the transition from a "sure" answer to a "sorry" refusal, JDJN explicitly targets the internal mechanisms associated with resisting jailbreaks.
Different explanation goal. Let
Multi-step optimization of neuron masks. Instead of scoring neurons via single-step gradient heuristic, JDJN iteratively optimizes neuron masks using gradient-based updates. This multi-step procedure refines neuron importance estimates over multiple passes, leading to more stable and accurate identification of JailNeurons. In practice, this approach captures nuanced, non-linear con- tributions that simple one-shot criteria tend to miss.
Layer-wise balancing. JDJN additionally applics a top-down sampling strategy across layers to balance information diversity against redundancy. Rather than concentrating all capacity a few late layers, JDJN spreads attention across shallow and deep layers, which yields more robust detector. This design allows JDJN exploit complementary signals from early representations (e.g., lexical or visual cues) and later semantic or safety-related features.
A.12.1 CONCRETE DIFFERENCES FROM INDIVIDUAL BASELINES
"The First to Know Zhao et al. 2024). This line of work focuses on first-token logits as indica- tors of safety risks. It performs linear probing surface-level outputs but does not explicitly reason about multi-layer internal activations. JDJN instead uses neuron activations from multiple layers, capturing richer hierarchical representations that more expressive for jailbreak detection.
22

---

Published as a conference paper at ICLR 2026
SNIP and gradient-based pruning methods Wei et al. (2024) SNIP-style methods compute single-step gradient scores for weights or neurons and prune accordingly. JDJN differs three key aspects:
1. From "sure" to "sorry" guidance. JDIN computes gradients with respect to the "sorry" token under jailbreak contexts, explicitly modeling the transition from a confident harmful response to a refusal. This transition forms a core signature jailbreak behavior.
2. Multi-step optimization. JDJN repeatedly updates neuron masks, rather than relying on a one-shot gradient magnitude. This iterative refinement leads to more stable neuron selec- tion and improves downstream detection performance.
3. Layer-wise balancing. JDJN combines top-down sampling across layers with mask opti- mization, reducing redundancy and preserving diverse features that SNIP-like global prun- ing may discard.
SHiPs Zhou et al. (024b)). SHiPs identifies attention heads that most affect y f(x | 0,) and primarily targets decoder-layer attention heads. JDJN instead operates on MLP neurons and supervised by the desired "sorry" counterfactual. These two neuron-level-and complementary. Exploring "jailbreak heads" as detectors and connecting them to our JailNeurons presents promising direction for future work.
aatplress-atttttoon-eeeee hewioon---add-eeell and
HiddenDetect Jiang et al. 2025 HiddenDetect uses logit-lens decoding from intermediate hid- den states to detect unsafe behavior. However, logit-lens signals from shallow layers are often noisy and less reliable, which leads HiddenDetect to under-utilize early-layer information. JDJN, driven by the "sure > sorry" supervision, identifies JailNeurons across all depths and leverages both shal- low and deep representations. This yields more comprehensive and effective detector.
A.12.2 QUANTITATIVE COMPARISON
We train JDJN JailBreakV and MM-Vet, enforeing FPR 0.05 MM-Ver. Table 18 reports TPR @0. for JDJN and three neuron- surronddigging-baee baselines on two LVLMs (LLaVV and Janus-Pro) across three jailbreak benchmarks.
LLaVA
Janus-Pro
Method JailBreak
FigStep
JAMLLM
JailBreak V
FigStep
JAMLLM
JDJN
0.997
1.000
0.942
0.996
1.000
0.853
First-to-know
1.000
0.952
0.433
1.000
0.976
0.323
SNIP
0.993
0.995
0.896
0.996
0.932
0.623
HiddenDetect 0.335
0.552
0.340
0.415
0.624
0.611
Table 18:
Comparison of TPR 2.0 for JDJN and neuron-digging based baselines. JDJN shows the strongest generalization, especially OOD datasets such as JAMLLM.
JDJN consistently achieves the highest TPR under the same FPR constraint, particularly on OOD datasets like JAMLLM, indicating stronger generalization beyond the training distribution.
A.13 EXTENDED FAILURE ANALYSIS
Understanding JDJN's failure modes helps clarify its limitations and the inherent ambiguity in defin- ing jailbreaks. We inspect misclassified samples and find that most emrrr arise on prompts that are semantically ambiguous between malicious and benign.
False positives. JDJN sometimes flags benign queries that involve sensitive topics but do not re- quest harmful actions. For example, in the Normal dataset, prompts like "Please list key events from World War II" are occasionally classified as jailbreaks. This likely occurs because war-related concepts activate features that partially overlap with those in genuinely malicious prompts. Such cases illustrate a trade-off between high recall for subtle jailbreaks and strict avoidance of topic-sensitive but legitimate queries.
23

---

Published as a conference paper at ICLR 2026
False negatives. We also observe cases where the underlying LVLM responds to harmful prompt in a vaguely non-malicious but non-explicitly refusing manner. Under our annotation scheme, we label these jailbreaks because the model does not clearly reject the request; JDJN, however, often classifies them as benign. For instance, in JAMILM, some malicious prompts receive ambigu- evasive answers without clear refusal phrase such as "I cannot help with that," leading to disagreement between the ground truth label and the detector prediction. This highlights that the operational definition of "jailbreak" how strictly refusal language is required Substantiall affects evaluation.
Overall, JDJN's errors tend to concentrate on borderline, high high-ambiguity prompts rather than on clearly benign or clearly malicious ones.
A.14 OVER-SAFETY AND FALSE POSITIVES ON STRESS TESTS
To further examine JDJN's s behavior under broader benign distributions, we evaluate JDJN, (trained on JailBreakV and MM-Vet) Vet) multiple benign predominantly benign datasets, including OR- Bench and XSTest, two stress-test benchmarks specifically designed to probe over-refusal LLMs.
Table 19 shows the FPR of JDJN five benign datasets for LLaVA and Janus-Pro.
Model
MM-Vet
MM-Bench
Normal
ScreenSpots
AC
OR-Bench
XSTest
LLaVA
0.000
0.000
0.019
0.022
0.012
0.390
0.170
Janus-Pro
0.000
0.000
0.044
0.002
0.033
0.280
0.090
Table FPR of JDJN on multiple benign datasets and two stress-test benchmarks.
JDJN maintains very low FPR on typical benign datasets such as MM-Vet, MM-Bench, Normal, ScreenSpots, and AndroidControl (AC), aligning with our design goal of minimizing disruption for normal users. In contrast, JDJN shows higher FPR on OR R--Bench and XSTest. We interpret this as follows:
Stress-test nature. OR-Bench and XSTest are intentionally constructed to elicit over- refusal from LLMs. Prior work reports that advanced commercial models such as GPT-4 and Gemini exhibit over-refusal rates exceeding 90% on OR-Bench. this con- text, JDJN's FPR, although relatively high, remains low compared to the underlying model behavior.
User impact vs. worst-case robustness. Our primary objective in constraining the de- tector's FPR is to protect normal user experience on everyday, benign usage. OR-Bench and XSTest do not aim to represent this typical usage; instead, they stress-test the bound- aries of safety policies. Consequently, higher FPR these two datasets does not directly translate into substantial harm for ordinary users.
These results suggest that JDJN strikes a reasonable balance: remains conservative enough to capture subtle jailbreaks while keeping false positives low on standard benign distributions, and only exhibits elevated FPR on adversarially designed over-safety stress tests.
24


Reviews:
[{"review_id": "KwKZi8giVW", "reviewer": "Reviewer_5qV5", "summary": "This paper studies jailbreak detection for Large Vision-Language Models (LVLMs) by identifying a small set of neurons that strongly relate to jailbreak behavior. The authors first show that jailbreak and benign samples trigger different internal activations but that naive linear detection does not generalize well across attack types and benign sources. They then introduce JDJN, which locates “JailNeurons” through a mask-based optimization that forces harmful outputs to switch to refusal responses, and aggregates activations from selected layers to train a lightweight detector. Experiments across several LVLMs, three attack types, and multiple benign datasets indicate that JDJN yields high true-positive rates with very low false-positive rates, and generalizes to out-of-distribution attacks and unseen benign distributions. The method is efficient and works without modifying the base LVLM, making deployment friendly.", "questions": "You report that JailNeurons are sparse and effective. Do these neurons remain consistent under model finetuning, safety tuning, or instruction-following updates? How stable are they across different checkpoints of the same LVLM architecture?\n\nThe paper mainly analyzes decoder-side neurons. Do convolutional / vision transformer layers contain JailNeurons as well? If so, is the jailbreak signal similarly sparse? If not, what does that imply about multimodal interaction during jailbreak?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper frames a clear and specific safety question: whether jailbreak activity concentrates in a sparse set of neurons and whether such neurons can support consistent, low-cost detection. This goes beyond generic claims like “performance limits” and targets internal mechanisms of LVLM jailbreak behavior. The method is well-aligned with that motivation, since the mask-training step is crafted to pinpoint neurons whose ablation flips harmful responses to safe refusals. Both single-layer and multi-layer analyses are thorough, and the adversarial evaluation with adaptive attacks adds credibility. The empirical results demonstrate strong evidence rather than speculation, including cross-model tests and ablation studies that examine critical components such as layer selection, mask threshold, regularization, and detector choice. Overall, the study links motivation, method, and experiments coherently, and offers a useful tool for practical safety settings.", "weaknesses": "The causal interpretation of “JailNeurons” could be more rigorous; while the mask-based optimization offers a constructive handle, the paper does not fully rule out the possibility that the selected neurons encode surface-level shortcuts tied to particular phrasing or datasets. \n\nAlthough the authors run OOD tests, the scope of benign distributions is still somewhat narrow, and the stability of neuron sets across architectures and scaling regimes could benefit from deeper analysis. \n\nThere is also limited exploration of joint vision-language pathways; the focus is on text-side activations, so multimodal interplay is not fully dissected.\n\nThe white-box assumption may restrict real-world deployment scenarios, and a discussion on extending the method toward limited-access or proxy-signal settings would strengthen the broader impact.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p16u272L6Z", "reviewer": "Reviewer_dJcj", "summary": "This paper introduces JDJN (Jailbreak Detection via JailNeurons), a framework for detecting jailbreak attacks in LVLMs by identifying and aggregating neuron activations that are responsible for jailbreak behavior. The method 1. localizes a sparse set of “JailNeurons” through a learned masking optimization 2. trains a lightweight classifier (linear SVM) over these neurons across selected layers", "questions": "See weakness.\n1. It would help if the authors can provide more rigorous experiment/ analysis on the chosen jailneurons.\n2. Can the authors provide more discussion on the failure cases.\n3. Can the authors provide a more comprehensive analysis on the proposed method? For example, what would be the mask localization cost and classifier training cost?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Experiments on four LVLMs and three attack types show high true-positive rates.\n2. The proposed method seems to have strong generalization ability.", "weaknesses": "1. The “causal” interpretation of Eq. (2–3) is asserted but not formally proven. Only intervention -> change is shown but no contrastive group of neurons is shown to not affect the output.\n2. Performance drops on certain benign datasets (e.g., FPR 0.768 on Janus-pro/Normal in Table 3). The parameter settings (JDJN3 in this case) seem to impact the generalization of the proposed approach.\n3 . Efficiency analysis is not very comprehensive. The authors claim that JDJN requires only a single forward pass but the details on the mask localization and training of the classifier is missing.\n4. Potential overfitting to limited benchmarks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mXYHkGPKYs", "reviewer": "Reviewer_GpcJ", "summary": "This paper introduces JDJN (Jailbreak Detection via JailNeurons), a novel method for detecting jailbreak attacks in Large Vision-Language Models (LVLMs). JDJN introduces the concept of \"JailNeurons\" - specific neurons that are activated during jailbreak attempts\nThese neurons are distinct from previously studied \"SafeNeurons\" which explain standard safety mechanisms.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper introduces JDJN, a novel method for detecting jailbreak attacks in Large Vision-Language Models by identifying and leveraging \"JailNeurons\". The approach demonstrates strong originality in its conceptualization of JailNeurons and its creative \"sure-to-sorry\" localization procedure. The quality of the work is evidenced by comprehensive empirical validation across multiple models and attack types, achieving impressive detection rates while maintaining computational efficiency. The significance of this research is substantial, addressing a critical security challenge in LVLMs with a practical, generalizable solution that could have immediate real-world impact on improving AI system safety.", "weaknesses": "1. Section 4.2.1 introduces a mask which is the key to this work. I am wondering whether this mask is neccessary. If the jailbreak information is in the neuron, why we cannot learn a classifier directly? If there is need of filtering out unrelated neurons, you have different options like regularizations and etc.\n\n2. You are missing some baselines in Table1. For example, AdaShield and JailDAM and Gradsafe and etc.\n\n3. For different dataset, how will the mask changing? It will be interesting to know how this changes. If the mask is different for different dataset, how do you explain the neuron you find?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vt88eGYeea", "reviewer": "Reviewer_pMai", "summary": "This paper presents a novel method to locate \"jailneurons\" in MLLMs so that the use of a classifier (in this paper, the authors adopt SVM) could handle the jailbreak detection. In detail, to finish the detection, they first locate related neurons by an optimization process and concatenate these neurons for SVM training. Experiments on several models, as well as a few jailbreak methods, demonstrate the performance of such a method.", "questions": "- What is the detailed experimental setup of baselines?\n- Could you explain more about the difference between previous neuron-digging methods and this script?\n- Will this method be over-sensitive, i.e., classifying benign prompts as jailbreaking? Experiments on or-bench or XSTest (or other MLLM datasets, if any) would be better.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Clear writing. The pipeline, mathematical details, and experimental setup are clear, which is easy to follow.\n-  Novel way to locate the neurons. The method to build an optimization framework for neuron selection is interesting.", "weaknesses": "- My biggest concern is about the experimental results. Take the evaluation on FigStep as an example. ECSO got 90.3% on the OCR subset of MM-safetybench (according to the original paper), which is a similar jailbreaking dataset using typography, so it seems a little weird that ECSO only got 0.596 on FigStep. Besides, the performance in the HiddenDetect paper is 0.846, where it also shows the performance of CIDER on Figstep is 0.713. The numbers reported in Table 1 have a huge gap with the original papers (or other replications), which requires further explanation.\n- From the perspective of storytelling, it does not clearly state the difference between previous neuron-digging methods and this script. Such detection could indeed be facilitated via a simple classifier, which is more efficient than using a guardrail model, but it is the advantage of all similar methods, such as SNIP, HiddenDetect, etc. The focus should be on the disadvantages of the previous safety-neuron picking method, or their suboptimal layer-picking method. More comparisons on this line of work are required[1][2][3] to prove that previous works could only handle normal harmful requests, other than jailbreaking requests.\n- The figures are not clear. More information should be included in the caption or related text parts. For example, what is the value in Table 1? I finish my review with the hypothesis that it is the detection rate (successfully detected/all jailbreak samples)\n\n[1]The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?\n\n[2]Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications.\n\n[3] On the Role of Attention Heads in Large Language Model Safety", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)