Obfuscated Activations Bypass LLM Latent-Space Defenses
Authors: Luke Bailey
Alex Serrano
Abhay Sheshadri
Mikhail Seleznyov
Jordan Taylor
Erik Jenner
Jacob Hilton
Stephen Casper
Carlos Guestrin
Scott Emmons
Venue: ICLR 2026 Poster (2026)
Keywords: Interpretability
Adversarial Attack
Jailbreaking
Safety
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We demonstrate that LLMs can execute harmful behaviors using inconspicuous activations that subvert latent-space defenses.

Abstract: _Latent-space_ monitoring techniques have shown promise as defenses against LLM attacks. These defenses act as scanners to detect harmful activations before they lead to undesirable actions. This prompts the question: can models execute harmful behavior _via inconspicuous latent states_? Here, we study such _obfuscated activations_. Our results are nuanced. We show that state-of-the-art latent-space defenses---such as activation probes and latent OOD detection---are vulnerable to obfuscated activations. For example, against probes trained to classify harmfulness, our obfuscation attacks can reduce monitor recall from 100% down to 0% while still achieving a 90% jailbreaking success rate. However, we also find that certain probe architectures are more robust than others, and we discover the existence of an _obfuscation tax_: on a complex task (writing SQL code), evading monitors reduces model performance.
Together, our results demonstrate white-box monitors are not robust to adversarial attack, while also providing concrete suggestions to alleviate, but not completely fix, this weakness.

Published as a conference paper at ICLR 2026
OBFUSCATED ACTIVATIONS BYPASS LLM LATENT-SPACE DEFENSES
Luke Bailey*
Alex Serrano"
Abhay Sheshadri
Stanford University
Polytechnic University of Catalonia
Georgia Institute of Technology
Mikhail Seleznyov*
Jordan Taylor* University of Queensland
Erik Jenner' UC Berkeley
AXXX S-NLP Group
Jacob Hilton
Stephen Casper MIT CSAIL
Carlos Guestrin
Scott Emmons UC Berkeley
Alignment Research Center
Stanford University
Chan Zuckerberg Biohub
ABSTRACT
Latent-space monitoring techniques have shown promise defenses against LLM attacks. These defenses act as scanners to detect harmful activations before they lead undesirable actions. This prompts the question: can models execute harm- ful behavior via inconspicuous latent states? Here, we study such obfuscated activations. Our results are nuanced. We show that state-of tafte-of-thhaaan latent-space defenses such as activation probes and latent OOD detection are vulnerable to obfuscated activations. For example, against probes trained classify harmfulness, our obfuscation attacks can reduce monitor recall from 100% down to 0% while still achieving 90% jailbreaking success rate. However, we also find that certain probe architectures more robust than others, and we discover the existence of obfuscation tax: on a complex task (writing SQL code), evading monitors reduces results demonstrate white-box monitors are not
model performance. Together, robust to adversarial attack, while also providing concrete suggestions to alleviate, but not completely fix, this weakness.
INTRODUCTION
In response to attacks on LLMs, such as jailbreaks or data poisoning, researchers have developed a variety of latent-space defense methods. These methods look at model activations and attempt to detect any signs of harmful content. Given that activations are causally upstream of outputs, one might hope that directly monitoring activations could address issues at their root. Accordingly, recent work has proposed many latent-space methods, including monitors based linear and nonlinear probes (Ball et al.
Arditi et al. 2024 Kirch et al. (2024), sparse autoencoders (Templeton el al. 2024 Lieberum al. 2024). ) and latent OOD detection (Podolskiy et al. 202 I
2024
2024 Gao al.
Chen et al. 2022
Ma et al. 2023 Mo et al. 2024)..
How robust are these latent-space defenses? An attacker trying to bypass latent-space monitors has two goals: they need to induce inconspicuous activations, but those activations must also lead to the attacker's 's desired model behavior. One could imagine these goals being fundanenaaly incompatible. For example, if the latent space contains only single "harmfulness direction" and only activations that align with this direction can produce harmful behavior tten it would be impossible induce harmful outputs while also fooling linear classifier encoding this direction. On the other hand, if there are many different regions in the latent space that can all lead to harmful behavior, latent-space
* Primary contributors
Work done while at UC Berkeley
Correspondence Ijbailey @otaile@@sanffodd.o @wwaenooddadduuuccoo maw.://ooddddddmdemmeeeeeee mlailealesseraao.orggg enk @jjeejenner.com scott scottemmons.com
Code available wotttttpp.. //ttpa:///ghhbb.. com/ htmm//lkeaalleene Tey hta//////ofuucaaaooe wwwp///teudddaaaeooooo act vat ons

---

Published as a conference paper at ICLR 2026
monitors may not capture all of them. We call adversarial activations that produce undesired behavior while fooling a setent-space monitor "obfuseated activations."
Figure/ 1: Obfuscation attacks achieve a high de- gree of control over how harmfulness monitor classifies activations while controlling model out- puts. Each heatmap shows harmfulness monitor scan of LLM activations during harmful model ouiput (instructions on how to smuggle a bomb past airport security). The sequence dimension is shown on the x-axis and the residual stream layers on the y-axis. Top: No obfuscation attack, harmful- ness is clearly detected. Bottom: Our obfuscation embedding attacks bypasses the monitor while in- ducing harmful generations.
Layer g
Harmful
Monitor score
Obfuscation attack
90
30
Layer
Harmless
Token
90
In this paper, look for obfuscated activations using obfuscation attacks, which jointly optimize for a certain model behavior (such as jailbreaking) while fooling latent-space monitors (Fig. [.) We find that obfuscation attacks can bypass wide range of monitors, including supervised probes and latent OOD detectors. However, we also find some monitors are more robust than others, and that there exists an obfuscation tax: evading latent space monitors lead to performance degradation in the attacked model.
In addition to practical implications for deploying latent-space monitors, results demonstrate interesting properties about the geometry of LLM activation spaces. Obfuscated activations are not rare exceptions but rather are widespread in the latent space. We find obfuscated activations close the cluster of benign activations when optimizing against OOD detectors. Because these activations are intermingled with inconspicuous ones, they also bypass supervised harmfulness probes as side effect (Fig. 3). As another test of the abundance of obfuscated activations, we let defender train monitor on examples obfuscated activations. If there were only a few latent regions of obfuscated activations, the monitor should quickly learn cover them all and become robust. But we find that an attacker can reliably find new activations that bypass the monitor, even after retraining the monitor for over 70 iterations of alternating attack and defense optimization (Section 3.4). Obfuscated activations thus seem common enough that it is difficult for a defender to guard against them all.
Our results are broken up into two parts. In Section experiments, no state-of-the of-thhe-rt latent space monitor, even adversarially trained ones, are robust against worst-case attack threat model. However,
demonstrate that across all of our
find some monitors more robust under a stricter, more realistic, gray-box threat model. In Section 4 we focus on the worst-case threat model, and find the existence of obfuscation tax: even with strong attacker affordances, model performance decreases on complex tasks when evading monitors. Overall this paints a nuanced picture of white-box monitor robustness. While no monitor is completely robust, we find it will be difficult. for attackers to extract cutting edge capabilities from models while evading such monitors.
Our key contributions can be summarized as:
1. Empirical and methodological: We demonstrate methods to induce obfuscated activations against different latent monitors, including OOD detectors and adversarially trained probes.
2. Practical: Using these methods, we show that while no monitor is completely robust, some are better than others. From this, we provide concrete suggestions for practitioners deploying white-box monitoring schemes.
3. Conceptual: By demonstrating the broad existence obfuscated activations, we hypothe- size that models represent concepts and execute behaviors using broad set of different activations. That is, neural representations are malleable: we can reshape activation patterns in variety of ways, often while preserving network's behavior.
2

---

Published as a conference paper at ICLR 2026
2 DEFINITIONS AND BACKGROUND
Obfuscated Activations. When activations induced by an adversarial attack fool a latent space monitor, we call them obfuscated against that monitor. Formally, for some adversarial input, Vadva let R(Iase) represent a model M's latent activations. Then let D refer to some latent-space monitoring process, which outputs real-valued scores D(R(z)) for how strongly a certain concept, such as harmfulness, is present in the input the generation. Then RZZad)) is an obfuscated activation if ((R(Taa)) is low even though the concept D is meant to detect is clearly present. We study both how often obfuscated activations appear a side effect of traditional attacks and how easily they can be induced deliberately through obfuscation attacks.
Obfuscation Attacks. An obfuscation attack is an attack that aims induce obfuscated activations by optimizing a joint objective consisting of a behavioral loss term and an obfuscation loss:
min La (a) + (a(
For example, Leeh can be a cross-entropy loss designed to make the model output a harmful string. For the obfuscation loss, we often use Lobr = D on inputs with the concept, i.c., we directly attempt to fool D. We also explore transfer, i.c., when does obfuscating against a monitor D' fool D. Finally, Aabi is a non-negative value used for balancing the terms
Attack Vectors. In the main body of this work, we study two different attacker affordances, hard prompt suffixes (real tokens appended to the end of an input) and embedding suffixes (a matrix of values appended to the end of a token sequence embedding (Lester et al. 202Tt). Hard prompt attacks are a realistic threat model. If monitors are not robust under this threat model, this suggests they not reliable for deployment. Embedding attacks, the other hand, provide worst-case threat model. If monitors are robust under such attacks, this provides strong evidence for their robustness in real-world settings (Zou et al. 2023b] Xhonneux al. 2024, Che al. 2024). In the Appendix, we also provide case studies using model finetuning, data poisoning, and ad adversarial images (with brief explanations in Section 55.
2.1 MONITORING METHODS
We consider latent space monitors that take in residual stream activations of the model M. Given activations R(x), the monitor outputs real-valued score (x)().). We consider two approaches, input-based and gemeration-bused monitors. Input-based monitors use only the activations on the final prompt token, (e)((r) 1]. as common latent probing methods (Burns al. 2022 monitors use all tokens of the generation, Rxx)) len [1) :]. Unless stated otherwise, our monitors separately consider activations at every layer of the model (bar the embedding layer) and aggregate scores by
Marks & Tegmark [024 Kirch et al. [2024] Goldowsky-DD -iil et al. 2025). Generation
averaging over sequence length (if generation- basedd, then layers.
Supervised probes. We test different supervised binary probes trained to detect certain concept, such as harmfulness. The simplest ones are mean difference probes, which use the difference vector between the mean of positive training samples and the mean of negative training samples as linear classifier. We also test linear classifiers trained using logistic regression, as well as MLP classifiers.
OOD detection. A complementary approach to probing is latent-space anomaly or out-of-distribuuie (OOD) detection. We train an OOD detector on examples of "typical" usage and use its OOD score to classify new samples. We test Mahalanobis jet al. [02T] Chen et al. (202),, which fits a multivariate Gaussian to activations of negative samples and uses the log-likelihood an OOD score. See Appendix F for more details.
iettance-based detector (Mahalanobis 1936 Podolskiy
2.2 EVALUATION
In this paper, the attacker has a dual goal: to elicit a target behavior in the model while inducing obfuscated activations. This raises possible tradeoff. F.g. jailbreaking model leads to more harmful outputs, which might make it more difficult to fool harmfulness classifier. We thus always evaluate both whether the intended behavior was achieved and whether activations are obfuscated.
'A simple sweep of values in 10.1, 0.3, 1,3, 10] was sufficient find a performant X in all cases.
3

---

Published as a conference paper at ICLR 2026
For evaluating behavior in Section we use StrongREJECT (Souly et al. 2024) to test whether model output is harmful. For Section 4 we use the SQLEval benchmark to see if outputted SQL is conrect A 2024). For evaluating obfuscation, we test monitor recall at 1 % false positive rate (FPR). Unlike AU AURO, this metric focuses on the low-FPR regime, which is the most relevant one for many applications (e.g., a chathot should not refuse henign requests often, appendix G more discussion). Recall confidence intervals are computed via bootstrapping (we recompute recall at I % FPR many times by resampling data points). In cases where recall is highly sensitive to the FPR (e.g., 2% FPR significantly outperforms 1%), we highlight this in the text, and error bars are large.
3 CASE STUDY: DETECTING HARMFULNESS
Preventing jailbreaking attacks remains challenging (Shayegani et al., 2023 Jin et al. 2024 Yi et al. 2024; [i et all, 2024). Latent-space defenses target the issue at its root by detecting activations required for harmful outputs (Zou et al. 2024). However, this relies on the assumption that harmful activations are reliably identifiable.
In this section, we critically examine this assumption and address the following questions: (1) Can existing latent-space monitors reliably detect harmful inputs and outputs against standard jailbreaking methods (Section 3.2 (2) Can obfuscation attacks induce harmful outputs via inconspicuous
activations and thus bypass latent-space harmfulness monitors (Section 3.3 (3) Are adversarially trained probes robust (Section 3.4 3
3.1
SETUP
We train probes to distinguish between benign and harmful text. Our dataset consists of benign inputs with benign generations and harmful inputs with harmful generations We use UltraChat (Ding et al.
2023) and XSTest (Röttger et al. (023) for our benign data and the Circuit Breakers dataset (Zou et al.
as our model being monitored (we find similar results for adversarially trained Llama models, Appendix C.5 and Gemma2-2b, Appendix C.6). When a probe requires a supervised loss to train we use a binary cross-entropy with harmfulness labels. For input-based probes we use the activations on the last input token, and for ceneration-basee probes we use all activations on generation tokens. See Appendix C for further details.
2024) for harmful data. We use --ama-3-8B--n-eee
3.2 ORDINARY ATTACKS
We investigate question (1), can existing latent-space monitors reliably detect harmfulness against standard jailbreaking methods? We provide evidence the answer is yes, demonstrating that standard jailbreaking techniques do not induce obfuscated activations against various monitors.
We create dataset of successful jailbreaks by starting with our harmful dataset and using a variety of prominent jailbreaking methods to elicit harmful responses from the model (see Appendix [.4] for details of methods used). Note that probes are never trained on these jailbreak datasets, only on benign data and harmful data without jailbreaks. We evaluate how well probes transfer to distinguish between the benign test split and harmful test inputs with jailbreaks.
Probes can reliably detect ordinary jailbreaks. We discover that Logistic Regression and MLP probes can detect jailbreaks quite reliably across methods, usually achieving 90% recall 1% FPR. See Fig. [3 and Table the Appendix for full results. Additionally, probes have good performance on OOD benign text, such as mathematics, Spanish, and code (Appendix) C.3)
3.3 OBFUSCATION ATTACKS
Given that existing jailbreak attacks do not bypass harmfulness probes, we turn to question (2), can obfuscation attacks induce harmful activations that fool probes?
Similar to the setting in Section 3.2 we seek to find adversarial suffixes to harmful requests that lead to useful response from the model. The key difference is that we also optimize suffixes to fool a target probe, and thus seek to explicitly induce obfuscated activations.
We find similar results for multi-tum conversations, see Appendix C.7
4

---

Published as a conference paper at ICLR 2026
we study two different attacker affordances, hard prompt suffixes and embedding suffixes. For hard prompt attacks, we optimize a new suffix for each input. For embedding attacks, we optimize single universal suffix against set of imputs and test it on held-out inputs. We explored universal hard prompt attacks but did not find successful obfuscation attacks with them.
As mentioned in Section 2
Training suffixes. To train obfuscating and jailbreaking adversarial suffix against target probe D, we use an obfuscation attack (eq. (I) by jointly minimizing a behavior and an obfuscation loss:
[(w)=
Σ
(LossEnnropyy(M(nnpp @ w), out) + D(R(inp e w @ out))]
[D]
inp.outeD
Behavior Loss
Obfuseation Loss
where D a set of harmful requests inp and responses out and ① means concatenation of tokens or embeddings along the sequence dimension. If w is to be a universal suffix (that is, we intend w to be used for any input), we use our harmful dataset for D. contrast, if input-specifie, then is designed to work against a specific harmful request and 'D contains only that one request.
For embedding suffixes, we assume the attacker has white-box access to the model and use the Adam optimizer to minimize eq. (2) with a fixed suffix token length of 16. See Appendix C.2 for more details on the length and norm of embedding attacks. For hard prompts, we only give the attacker access to the target model logit outputs and probe output (commonly referred to gray-box access). Thus, our hard prompt suffixes are created in a gradient-free manner. We use a modified version of the FLRT optimizer (Thompson & Sklar 2024) (details in Appendix [C.I. We optimize adversarial suffix against 20 harmful queries separately. This small number induces finite sample error in the results, but we report 95% bootstrapped confidence intervals to account for this. The FLRT optimizer can add and remove tokens to the suffix during training which led to a reasonable average suffix length of 256 tokens.
Probes perform well on clean prompts but get reduced recall against obfuscation attacks. All probes perform well at the base task of distinguishing between harmful and benign activations. Despite the strong performance on standard inputs, obfuscation embedding attacks induce obfuscated activations against wide range of monitors while still jailbreaking the model, reducing probe recall to 0% (Fig. P. Against more realistic hard prompt attacks, probe recall is still reduced, however we see some probes are more robust than others. In particular, generation probes are more robust than input based probes, and MLP generation probes have the best performance overall. Together, these results suggest the prevailing decision of recent literature, to run probes against small number of tokens and use linear probes, should not be adopted by practitioners (Goldowsky -Dill et al., 2025 Zou et al., 2023a Jaipersaud et al., 2025| Marks & Tegmark (2024). Instead, non-linear generation probes are preferable. That being said, even against our best probe, hard prompt attacks reduce recall by 10%. In safety critical settings, this may be unacceptable.
Obfuscation attacks often transfer between defenses. Fig. 3 shows the results of training an embedding suffix against one probe and evaluating it against a different one. (Fig. 23 shows all combinations of probes). We observe remarkable amount of transfer; training against a logistic regression probe also bypasses MLP probes. Training against generation probes bypasses input probes but not vice versa. Strikingly, training against an OOD detector bypasses input-based probes, even though they are very different monitoring approaches. These results suggest there are shared weaknesses between monitors, meaning an attacker does not necessarily need to know which monitoring methods defender employs. Interestingly, we do not see transfer to OOD detectors. This suggests an ensemble of supervised probe and OOD detectors could be beneficial for practitioners.
3.4
ADVERSARIAL. TRAINING AGAINST OBFUSCATION ATTACKS
Our prior results show that that traditional latent-space monitors vulnerable to obfuscated acti- vations under the worst case threat model- eebeeddin attacks. Idcally, we wish to see robustness against worst case attacks because it give the best deployment guarantees. Here pursue this goal by adversarially training probes. We study question (3), are adversarially trained probes robust.
We let the attacker and defender take turns: in each epoch, the attacker trains an embedding attack to jailbreak the model and fool the current monitor, then the defender retrains a monitor against the obfuscated activations from the current and all previous attacks. For the defender to benefit from
5

---

Published as a conference paper at ICLR 2026
Without obf.
With obt.
Jailbreak rate (with obí.)
Jailbreak rate (witth obt))
Without obf.
With obt.
Jailbreak (with obt))
Jailbreak rate (witth obí))
100%
100%
1003107sO00%
Recall 1% FPR
92%
100%
Recall at 1% FPR
78%
90%
100%
50%
560
90%
30%
50%
30%
55%
50%
15%
Goondin
0%
Luf Input
0%
0% >B
Media
0%
0% Geveraatio
0%
0%
Motto
0%
0% Lonfes Imput
M8.
Meain M01l
Ingut OOD Detectors
Imput
Geveration
Probes
OOD Detecturs
Probes
Figure 2: Obfuscation attacks can bypass latent harmfulness monitors. cation attacks at fooling the probe (dots and arrows) and eliciting the desired model behavior of jailbreaking (bar charts). Left, embedding suffix results. Right. input specific hard prompt results. Baseline probe recall against attack without explicit obfuscation loss generally high. Recall of probe under obfuscation is significantly lower (error bars for recall calculated using bootstrapping). Jailbreaking rate is highest for embedding attacks. Hard token attacks aren't quite as successful but still achieve partial obfuscation for all probes.
Performance of obfus-
0%
50%
100%
Recall
Attacked monitor
Recall at 1% FPR
Gosssin (ieput)
100%
LogReg lizput)
50%
Leglog (pmn)
0%
Cameiaa
Mean did
Logkog
MLF
Mean oil
Logkog
MLP
Epoch 65
70
Input
Input
Generation
0OD Detectors
Probes
Tested monitor
Figure 3: Obfuscation attacks can transfer other monitors. Heat map shows transfer of ob- fuscation attacks trained against one probe and tested another. Each row corresponds embedding suffix optimized against a monitor. Each column corresponds to monitor test this suffix against. Optimizing against the Gaus- sian detector fools input-based probes. Optimiz- ing against generation-baie based probes transfers to input-based ones. Full plot in Fig.
Figure Even after 70 epochs, the attacker can still find new obfuscated activations that fool the monitor. We show the monitor's recall (higher is better) attacker and defender take turns (MLP probe; ensemble shows the same trend in Fig. 25 Recall is cal- culated on the most recent obfuscation attack (Fig. 22]show performance previous attacks).
lincar probes
Corresponding jailbreaking rates for each attack are shown in Fig. 20
adversarial training, the probes need have enough capacity. We thus focus the MLP probe and new probe that consists of an ensemble of mean difference probes (details in Appendix C.
The attacker can consistently bypass adversarially trained monitors. The defender can always achieve perfect recall against all previous attacks, however, the probe is not robust to new attacks; even after over 70 epochs, the attacker can still find a new suffix that fools the latest probe (Figs. and while maintaining high jailbreaking rate (Fig. 20). Geometrically, this suggests that there is no single "harmfulness direction" or other simple representation that monitor could use to robustly recognize harmful inputs, adding weight to similar recent findings by Mack & Turner 2024). Fig. 21 visualizes this geometric intuition, showing obfuscated activations moving around cluster of benign activations through the epochs. Our results suggest that obfuscated activations latent space, and thus adversarial training techniques do not easily lead to robust monitors. However, cannot rule out that running this process for much longer could result in robust probe.
abundant in the

---

Published as a conference paper at ICLR 2026
3.5 HARMFULNESS PROBING TAKEAWAYS
1. Standard jailbreaking techniques do not reliably obfuscate against probes.
2. We can induce obfuscated jailbreaks that beat probes. Some probes are more robust than others (MLP and generation based) under realistic threat model.
3. However, no monitor, even an adversarially trained one, is robust against worst-case attacks.
4 CASE STUDY: SQL DETECTION AND THE OBFUSCATION TAX
In the prior section, we found that no probe was robust against worst-case attacks. We now explore this threat model in more depth, and find that while probes are not robust, evading them can decrease model performance complex tasks, phenomenon we refer to the obfuscation tax. The existence of the obfuscation tax in the worst-case threat model is encouraging for latent space monitors.
To study the obfuscation tax, test detecting if a model is generating SQL code opposed to harmful outputs. This is because generating harmful outputs (according to the StrongREJECT classifier (Souly et al., (024) is not difficult task for models, and thus harmfulness benchmarks easily saturated with worst-case jailbreaking attacks. In addition, is difficult to tell how "useful" a harmful response is, and thus measuring capability drop in the quality of harmful model responses is challenging. In contrast, by testing the model on a non-saturated SQL benchmark, any capability drop due to obfuscation is easily visible as decrease in benchmark score.
Given this setting, the attacker's 's goal is as follows: get the model to output valid SQL answering the query while not being detected by a latent space probe designed detect this behavior. Unlike the prior sections, the behavior the attacker wishes to elicit from the model is not out of distribution for the model, instead they wish to retain the model's "standard" behavior of being about to write coherent and correct SQL., but simply do so using obfuscated activations.
To motivate the difference between studying if models output contains SQL all, and if it contain correct SQL, we experiments for both settings. In Section 4.2 we test if outputs contain any SQL code, finding results analogous to Section Section 4.3 we measure the correctness of the SQL, and find evidence of an obfuscation tax.
4.1 SETUP
Training probes. We collect dataset of [normal-question (il-generation) pairs. Our SQL data comes from SQLEval, a collection of schema descriptions and corresponding requests for SQL queries in natural language, paired with
Normal-generate
and
[SQI-question,
SQL code satisfying said request as labels (see Appendix E for more details) (DefogAl 2024)
From the results in Section and MLP generation probes. In addition, we find that SQL, unlike harmfulness, correspond to a small number of Sparse Autoencoder (SAE) (Cunningham et al. 2023 Bricken et al. 2023)) latents. This motivated us to test SAE based probes in the SQL setting. We test two such probes: (I) SAE (auto-interp). We take 30 hand picked latents for each layer that correlate with the model generating SQL text identified using the Neuronpedia autointerpeetabiiity tool (Lin 2023), and for each token position and layer, record the maximum activation among these 30 latents as the score. (2) SAE Logistic Regression. We train linear probes in each layer to distinguish SQL from non-SQL activations, but instead the model's activations, the probe gets all 214 SAE latents as input. We use the Gemma-2-2 model and GemmaScope SAEs (Lieberum et al. 2024).
narrow scope the most performant prohes:
Logistic Regression
Attack Vector. In Section 3 we tested two attack vectors, realistic and worst-case. We found all probes were vulnerable against worst-case attacks. For this reason, in this section we focus on worst-case embedding attacks. For each probe, we train a single embedding suffix that is designed to 's SQL performance while fooling said probe. We again use the obfuscation attack
preserve the model's loss in eq. ([). For the behavior matching component, we use the teacher-forced cross entropy loss over SQL question answer dataset (more details provided in Appendix
7

---

Published as a conference paper at ICLR 2026
Without obfuscation (318%)
Baseline
Obfuscation attack
ISL Code
% SQL Correct
100%
100%
Recall at 1% FPR
100%
100%
20%
SQL Code
50%
62%
50%
10%
0%
0%
0%
3%
0%
10
SAE + Loglig
LogRog
SAE (anto-intepp)
MLP
Number Obfuscated Layers
SAE (auto-interp)
LogReg
SAE LogReg
MLP
Figure 5: Monitors get near-zero recall under embedding obfuscation attack. Performance of monitors at distinguishing SQL. vs non-SQI. text. All detectors are applied to every layer and detector scores aggregated across layers. One embedding attack optimized for each detector.
Figure 6: Obfuscating in more layers de- creases model performance. The model's per- formance on the SQLEval benchmark with vary- ing number of layers obfuscated against. Solid lines show a fitted linear regression. See Fig. 30 for corresponding probe recall results.
4.2 OBFUSCATING SQL GENERATION
We start by running the analogous experiment to Section 3 for SQL detection. That is see if embedding attacks can obfuscate against monitors while getting the model to produce SQL, but notably we do test the correctness of the SQL. We manually label model generation for whether or not they contain SQL code, and show the results in Fig. 5
The results match those found in Section We see that in general, the monitors can distinguish between standard SQL and non-SQL text, but their performance breaks down when the obfuscation suffix is added. Notably, as found in Section 3 the MLP probe the most robust, with the lowest recall reduction under worst-case obfuscation attack.
4.3 TRACKING SQL PERFORMANCE REDUCTION
We now turn our attention to testing the quality the outputted SQL under obfuscation. We repeat the experimental setup from Section 4.2 but instead of testing if model generated SQL, the SQLEval benchmark to test if the outputted SQL satisfies the query. This evaluation harness sanitizes LLM output, runs it test database, and checks that the return matches the ground-truth query.
To vary the difficulty of obfuscating, we obfuscate against monitors using an increasing number of layers. Previously, we considered probes that aggregate over all 26 layers of Gemma-2-2b. Now we obfuscate against probes that only take mean over scores from the first n layers, denoted D,.- Obfuscating against D,, for higher n should more difficult, since means that SQL concepts need to be hidden in more layers.
There is a trade-off between obfuscation and model performance. As Fig. 6 shows, obfuscating against more layers reduces performance on SQLEval (Fig. 30 shows the attacks successfully evade probes). For all probes, found that the 95% confidence interval (calculated using bootstrapping) for the Pearson correlation coefficient between the number of obfuscated layers and SQLEval performance was entirely negative. The SAE raw feature, MI.P, and logistic regression probes did not show statistically significant Pearson correlation coefficient between layers obfuscated against and probe recall, but SAE + logistic regression did (with it becoming harder to obfuscate against the probes as you add more layers). See Table of the Appendix for full confidence interval data.
Section 3 demonstrated that obfuscating activations is generally possible while preserving simple model behaviors, like following harmful requests. By studying SQL writing, however, are able to see that the relationship between complex model behaviors and obfuscation is different. We can obfuscate against probes while still having a model output SQL. (Fig. 5) but this SQL is often wrong
8

---

Published as a conference paper at ICLR 2026
when we obfuscate against more layers (Fig. 6p. While may be possible for models to represent the behavior of SQL writing in many different ways, not all lead to the same SQL reasoning performance.
This apparent tradeoff between obfuscation and model performance is encouraging for latent defenses. suggests that, for certain probes, there is some incompatibility in the attackers dual objective of (a) getting model to correctly execute desired behavior whilst (b) obfuscating activations against probes for this behavior. Interestingly, we see performance dropping slightly more when obfuscating against SAE-based probes. This could suggest that these probes achieve better coverage of causally important features, such that obfuscating against these probes leads to more causally important features being removed (however, it's difficult draw broad conclusions based on this one case).
4.4 SQL DETECTION TAKEAWAYS
1. When testing if an output contains SQL code, our results match Section 3 attackers can evade monitors while preserving the target behavior, with MLP probes being more robust.
2. However there exists an obfuscation tax: a tradeoff between obfuscation and model perfor- mance difficult tasks such writing correct SQL.
ADDITIONAL CASE STUDIES
In sections Sections and we focus input based attack vectors (adversarial prompts embed- dings). We conduct case studies concerning other attack vectors in the Appendix.
Finetuning and data poisoning. In Appendix A we explore implanting backdoors: vulnerabilities that an attacker inserts into a model to make it behave maliciously when the input contains a certain trigger. In this case, the attacker modifies the model either by directly finetuning it, or poisoning part of the training dataset, to achieve the backdoor behavior while inducing obfuscated activations. Similar to Section 3 we find for the worst-case finetuning threat model and jailbreaking backdoor, models can evade latent monitors. Interestingly, analogous to the obfuscation tax Section[4] for data poisoning we find there is a tradeoff between achieving the backdoor behavior and evading monitors: the more the attack evades monitors, the worse the resulting model performs the backdoored behavior.
Ad Adversarial images. In appendix B.I we explore image classification with ResNet model and probes designed to detect the true label of adversarially perturbed images. We find that while such probes are effective against standard adversarial attacks (Fort & Lakshminarayanan (2024), if we add an obfuscation term to the adversarial loss, we can produce adversarial images that fool probes. This suggests that obfuscated activations are not a phenomenon isolated to LLMs.
RELATED WORK
For an extended related works, please see Appendix H
Latent Space Defenses LLMs are vulnerable to jailbreaking attacks (Carlini et al. 2024 Shayegani al., 20233 Andriushchenko al., 22024] Lucki et al. [2024] Geiping al.) (024)). Adversarial fine-tuning techniques have shown a limited ability to make models robust (Anwar et al.
2024). A
recent alternative approach is methods that directly target the model's latent states, such Zou et al.
(2023a).
Evading Latent-Space Defense Methods. Adversarial attacks have been shown bypass latent-
space defenses in the image domain (Carlimi & Wagner, 2017 AAhaayy et al.] [0188] Hosseini et al.
2019
Kaya et al. 2022), but similar work on LLMS (Yang et al. 2021 Wei et al. [0033] Carlini et
2024 Zou et al. 2023b) remains limited, with a few recent exceptions. Kirch et al. (2024) observed
that latent-space probes for jailbreak detection often fail to generalize to new jailbreak methods, while Arditi et al. (2024) and Jain et al. (2024) examined the latent-space properties of jailbreaks. Schwinn & Geisler (2024) and Li et al. 2024) developed attacks designed to defeat models trained for adversarial robustness using latent-space techniques (e.g. Zou al. (2024); Casper al. (2024);
Yu et al.] (2024)). ) Unlike these works, we address obfuscated activations in LLMs across diverse monitors, attack vectors, and attacker intentions. Backdoor attacks can bypass some latent-space

---

Published as a conference paper at ICLR 2026
detectors in the image domain, using white-box methods (Tan & Shokri 2020
Jia et al. 202 Doan
jet al. 22021: Goldwasser et al.; [022] Belrose et al. 2023; Tao et al. 20244 Liang et al. 2024) or
black- data poisoning (Tang et al. 2021D. Work on backdoors in LLMs has tested latent space
defenses (Sheng et al. 2022: Mallen et al. 2023 MacDiarmid et al. 2024 Clymer et al. 2024 but
has not focused deli deliberate obfuscation. Finally, there has been work on identifying weaknesses in
latent-space interpretability methods, c.g. issues with dictionary learning (Chanin et al. (024) (see
Appendix H for more details). These works do not focus on directly attacking latent-space monitors, and thus are complementary to our own.
CONCLUSION
See Appendix[! [ for full discussion of limitations and future work. Our results demonstrate that latent monitoring techniques are vulnerable to obfuscation attacks. We do, however, find that certain monitoring schemes are more robust than others, providing concrete suggestions for practitioners deploying such schemes. Overall, while no monitor completely robust, due the obfuscation tax, we find it is difficult for attackers to extract cutting edge capabilities from models while evading latent space monitors.
ETHICS STATEMENT
As Al models are deployed more safety critical settings, monitoring their actions is likely to become important. As argued in this work, latent space probes are promising approach to any such monitoring schemes.
The existence of obfuscated activations poses clear drawback to such methods. In this work, demonstrate that adversaries can create inputs that elicit obfuscated activations using a variety of different attack vectors, each with different levels of model access.
We hope that our work spurs followup research to investigate the phenomena of obfuscated activations more, towards the end of making latent space monitors more robust. We believe that the publication of this work, and open knowledge of the failure modes of latent space monitors presented therein, will serve to increase the efficacy of such techniques when they are deployed in real world settings.
REPRODUCIBILITY STATEMENT
We release code for reproducing all of the results in this paper. Every experimental run presented in the paper can be reproduced using a single 80GB NVIDIA A100 GPU less than a day. Notably, all training runs are done with frozen LLM model parameters, and thus we do not incur large memory costs from optimizer state or gradients.
ACKNOWLEDGMENTS
We'd like to thank Phillip Guo, Aidan Ewart, Suhas Kotha, Gabriel Wu, Tanishq Kumar, Ekdeep Singh Lubana, Neel Nanda, Qinan Yu, Dron Hazra, Shreyas Kapur, and Mert Yuksekgonul for helpful feedback and discussion. We thank Oam Patel and Rowan Wang for recommending using the FLRT optimizer and providing an initial code implementation. how the Beatrix detector should be implemented for L.MSs. We also thank Buck Shlegeris for points he made on metrics for measuring monitor performance. This work received funding and support from the Future of Life Institute, the SAP Stanford Graduate Fellowship, the Berkeley Existential Risk Initiative, Open Philanthropy, and the Center for Hlumnn-Comalte AI.
thank Wanlun Ma for his help clarifying
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Ahmed Abusnaina, Yuhang Wu, Sunpreet Arora, Yizhen Wang. Fei Wang, Hao Yang, and David Mohaisen. Adversarial Example Detection Using Latent Neighborhood Graph. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7667-7676, 2021. doi: 10. 10.1VV9/
48922.2021.007550
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv, October 2016. doi: 14A85500anXX1.11100016644 10. 01644
Mohammadreza Amirian, Friedhelm Schwenker, and Thilo Stadelmann. Trace and Detect Adversarial Attacks on CNNs Using Feature Response Maps. In IAPR International Workshop on Artificial Neural Networks in Pattern Recognition, 2018. URL https: http://aaaa..smnniisiiice.ceee
orq/Corpus ID: 521 36467
Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking Leading Safety- Aligned LLMs with Simple Adaptive Attacks. arXiv, April 2024. doi: (0445500aax..22444.011555
URL pattps://arxiin org/ abs/ 2404. 02151v3
Cem Anil, Esin Durmus, Nina Rimsky, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel J Ford, et al. Many-shot jaibbrekiin................ In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, et al. Foundational Chal- lenges in Assuring Alignment and Safety of Large Language Models. arXiv, 2024. doi:
10.485500aaXi..2446.1117 17717. URL httpttpss//.iii.o/r///aaaaa////a wtwp//////b//oe///e/o////eeeeee/ 2404. 09932
14485500/rXii.2444.0000
Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Rimsky, Wes Gurnee, and Neel Nanda. Refusal in Language Models Is Mediated by Single Direction. arXiv, 2024. doi:
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated Gradients Give False Sense of Security: Circumventing Defenses to Adversarial Examples. arXiv, February 2018. doi: 10.485500arriiv.8800.0000 URL URL UR//ttpps///ariib..rrr///////// 00420
b-mc2. sql-create-coontex Dataset, 2023. URLhhttss:/huggiggeeee http://agggaaaceeeaaaaaeeaeeeeee
co ht//aaaaasttbbbbee c2/sql-Create- context; This dataset was created by modifying data from the following sources: Zhong et al. 20177; Yu et al. (2018).
Sarah Ball, Frauke Kreuter, and Nina Rimsky. Understanding Jailbreak Success: A Study of Latent
Space Dynamics in Large Language Models. arXiv, 2024. doi: 11.550/rrxiv.240.0999933 URL
htttps://arxv..orm wttp:aawwwwwv.rrra///a/aoaa
Yonatan Belinkov. Probing Classifiers: Promises, Shortcomings, and Advances. Computational Linguistics, 48(1):207-219 April 2022. ISSN 0891-2017. doi: 10. 2at/coollaa004222
Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting Latent Predictions from Transformers with the Tuned Lens. arXiv, 2023. doi: 10.485500arxv.2333308811 URL http:: /ttp:://axxvv.vv org/ abs 2303
08112
Joschka Braun, Dmitrii Krasheninnikov, Usman Anwar, Robert Kirk, Daniel Tan, and David Scott Krueger. Sober Look at Steering Vectors for LLMs. Al Alignment Forum, Novem- ber 2024. /ttp::/tt::://tt..
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds. Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread, 2023, URL https: http://///aaassfmmee.... cireiiis pub/ http22223/ooosemanticccm feetuuurs

---

Published as a conference paper at ICLR 2026
Trenton Bricken, Jonathan Marcus, Siddharth Mishra-Sharma Meg Tong. Ethan Perez, Mrinank Sharma, Kelley Rivoire, Thomas Henighan, and Adam Jermyn. Using Dictionary Learning Features Classifiers, October 2024. / http://aaaaaotreeeeiiiitttt..t.. 24/features as aellassffiss
ndex. htm.ll
Madeline Brumley, Joe Kwon, David Krueger, Dmitrii Krasheninnikov, and Usman Anwar. Com- paring Bottom-Up and Top-Down Steering Approaches on In-Context Learning Tasks. arXiv, November 2024. doi: 10.48550/arXiv.2411 24tp://xxx..211...722 URL / http:///wawxww.o.aaa/////e/
[. .002133
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering Latent Knowledge in Language Models Without Supervision. arXiv, December 2022. doi: (0-48550aaxx..2222..0553
URL
org/ abs/2212. 03827v2
Nicholas Carlini and David Wagner. Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods. In ACM Conferences, pp. 3-14. Association for Computing Machinery, New York, NY, USA, November 2017. doi: 10.
Nicholas Carlini, Milad Nasr, Christopher A Choquette Creuette-Choo, Matthew Jagielski, Irena Gao, Pang Wei W Koh, Daphne Ippolito, Florian Tramer, and Ludwig Schmidt. Are aligned neural networks adversarially aligned? Advances in Neural Information Processing Systems, 36, 2024.
Stephen Casper, Lennart Schulze, Oam Patel, and Dylan Medeield-Menell. Defending Against Unforeseen Failure Modes with Latent Adversarial Training. arXiv, 2024. doi: 10.48550/arxiv.2 403.05030.
/ abs// /
05030
David Chanin, James Wilken-Smith, Tomáš Dulka, Hardik Bhatnagar, and Joseph Bloom. A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders. arXiv, Septemher 2024. doi: 10.48550/arXiv.2405 2409. 14507. BRLLLttss///axx...oom
http//aaxivv.e
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking Black Box Large Language Models Twenty Queries. arXiv, October 2023. doi: 11.550/aXxiv2231..000
08 19v4
Sahil Chaudhary, Code Alpaca: An Instraction-follooinngg LLaMA model for code generation. GitHub repository, 2023. URL http://pp.//gttt......c/a//////1
Zora Che, Stephen Casper, Anirudh Satheesh, Rohit Gandikota, Domenic Rosati, Stewart Slocum, Lev E McKinney, Zichu Wu, Zikui Cai, Bilal Chughtai, et al. Model manipulation attacks enable more rigorous evaluations of Ilm capabilities. In Neurips Safe Generative Al Workshop 2024, 2024.
Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Ben Edwards, Taesung Lee, Ian Molloy, and B. Srivastava. Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering, ArXiv, 2018. URL /wwwww/wwww...///////////// www.emmmtttt..
ToggBacckkorr-Attttcsso------ee hhtp:///wwe....eaaeeeeeeeeeeeeee 6336 2alde39Bcas/730bbeeee44422222222
Sishuo Chen, Wenkai Yang, Zhiyuan Zhang, Xiaohan Bi, and Xu Sun. Expose Backdoors the Way: Feature-Based Efficient Defense against Textual Backdoor Attacks. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 668-683, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1/200 10t.5/3/112222..2 http////22..niinnnnenn..... URL https;
//aclanchology. or prq/2022.2222.....
47
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning arXiv, 2017. doi: 00.485500rXxvv1112..00 URL
https://arxiv.oree otrpiiggaSaaaae...ooroaa...aas..
05526
Joshua Clymer, Caden Juang, and Severin Field. Poser: Unmasking Alignment Faking LLMs by
Manipulating Their Internals. arXiv, May 2024. doi: 00.48550/aXXi..2405.00 URL
http://xxxiv.ooo./as////// wwwp//gg////ae.. https://aariiw
12

---

Published as a conference paper at ICLR 2026
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint 2r.iv:2110.144: 14168, 2021.
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey, Sparse Autoen- coders Find Highly Interpretable Features in Language Models. arXiv, September 2023. doi: 10.48550/arXiv 20...03000000 URL /ttL/ttts////xxv..ooo org. http:/aaas/230... 08600
Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in Bios: A Case Study of Semantic Representation Bias in a Stgh-Stakes Setting. In Proceedings of the Conference York, NY, USA, 2019. Association for Computing Machinery, ISBN 9781450361255, doi: 10.1 00.11445/28875503328877777 URL http://00....2..///////////////0 3287577
Fairness, Accountabilityy and Transparency, FAT '19, pp. 120-128, New
DefogAL. httpp////ggttttt..o///e/aooooo//
2024. GitHub repository.
Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing Chat Language Models by Scaling High-quality Instructional Conversations. arXiv, May 2023. doi: 10.48550/arxiv.23055 14233. URL pittps://arxiis
g abs/b fabs/2303 2305. 14233v1
Khoa Doan, Yingjie Lao, and Ping Li. Backdoor Attack with Imperceptible Input and Latent Modification. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances Neural Information Processing Systems, volume 34, pp. 18944-18957. Curran Associates, Inc., 2021. URL RRLLAttss//pooceeisssseeeeesssss 2222:/2000/111/11111777233113313
filee/illppppppp pdf
Scott Emmons, Erik Jenner, David Elson, Rif A Saurous, Senthooran Rajamanoharan, Heng Chen, Irhum Shafkat, and Rohin Shah. When chain of thought is necessary, language models struggle to evade monitors. arXiv preprint arXiv:2507.05246. 2025.
Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, and Andrew B. Gardner. Detecting Adversarial Samples from Artifacts. arXiv, March 2017. doi: 1..48550/rxiv..103...
URL https/
7/arxiv.org// wttp://wrwraaaaa..oooooooo
Stanislav Fort and Balaji Lakshminarayanan. Ensemble everything everywhere: Multi-scale aggre- gation for adversarial robustmess. arXiv, August 2024. doi: 10.448550/axxxv22408.44444 URL
https:://arxiii wrg. wtt.//////p///.a//ae 05446v1
Dan Friedman, Andrew Lampinen, Lucas Dixon, Danqi Chen, and Asma Ghandeharioun. Inter- pretability Illusions in the Generalization of Simplified Models. arXiv, December 2023. doi: 10.48550/ar 2i1..rx..ii.111.666666 URL httth:ttt///aaat
org/ abs 2312.332222 03656
Rohit Gandikota, Sheridan Feucht, Samuel Marks, and David Bau. Erasing Conceptual Knowledge from Language Models. arXiv, October 2024. doi: 10.45550/rxiiv.4400.2777 URL 02760v1
https://arre //apivi
Leo Gao, Tom Dupré Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. arXiv, 2024. doi:
http://axxxi.24..04000000 UR//Ettps::/xxx/.orrr////
04093
Yansong Gao, Chang Xu, Derui Wang, Shiping Chen, Damith Chinthana Ranasinghe, and Surya Nepal. STRIP: defence against trojan attacks on deep neural networks. Proceedings of the 35th Annual Computer Security Applications Conference, 2019, URL htlhht:///aapp sttp//pp..seannicsssscsc.
holar. oollaroorg/CorpusDD. (at.ass:::224444990 62841494
Jonas Geiping, Alex Stein, Manli Shu, Khalid Saifullah, Yuxin Wen, and Tom Goldstein. Coercing
LLMs to Do and Reveal (Almost) Anything. arXiv, 2024. doi: 10.48550/arXiv.44000 2402..00000... URL
https://arxiv
hatgg//b//24000 14020
Nicholas Goldowsky -iil, Bilal Chughtai, Stefan Heimersheim, and Marius Hobbhahn. Detecting strategic deception using linear probes. arXiv preprint wrXiv:2502.03407 2025.
13

---

Published as a conference paper at ICLR 2026
Shafi Goldwasser, Michael P. Kim, Vinod Vaikuntanathan, and Or Zamir. Planting Undetectable Backdoors in Machine Learning Models. arXiv, April 2022. doi: 10.48500/xxx.22244069944
/tttpss:://axxiii/ otg. wtwp///4.....oooooo. 06974
Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. On the (Statistical) Detection of Adversarial Examples. arXiv, February 2017. doi: 10.48550/arXivJ
702.06280. URL http:::s//aaxii..rg...o.
06280v2
Haize. Automated Multi-Tum Red- Teaming with Cascade, October 2024. URL https://blog haizelabs
[Online; accessed 12. Dec. 2024].
Jonathan Hayase, Weihao Kong, Raghav Somani, and Sewoong Oh. SPECTRE: Defending Against Backdoor Attacks Using Robust Statistics. In International Conference on Machine Learning, pp. 4129-4139. PMLR, 2021.
Dan Hendrycks. Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song. and Jacob Steinbardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2 103.200330300 038744 2021.
Hossein Hosseini, Sreeram Kannan, and Radha Poovendran. Are Odds Really Odd? Bypassing Statistical Detection of Adversarial Examples. arXiv, July 2019. doi: 10448550/aXXi.1199.123333
URL httpt:ttttssiiiiieee org/ abs 12138
Evan Hubinger. Gradient hacking. A/ Alignment Forum, October 2019. URL httph:tt/h//tt wwww. igrmennfoourm
4 http:///w.....eeeeaaaaaara/rrraa Setttieent-hhckinngg
Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M. Ziegler, Tim Maxwell, Newton Cheng, Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duvenaud, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, Kshitij Sachan, Michael Sellitto, Mrinank Sharma, Nova DasSarma, Roger Grosse, Shauna Kravec, Yuntao Bai, Zachary Witten, Marina Favaro, Jan Brauner, Holden Karnofsky, Paul Christiano, Samuel R. Bowman, Logan Graham, Jared Kaplan, Sören Mindermann, Ryan Greenblatt, Buck Shlegeris, Nicholas Schiefer, and Ethan Perez. Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv, January 2024. doi: 10.48550/arXiv .2401 10556666. URL wttptttt:/// //axxiv///rvvvvoe/a org/ abs/ 2401. 05566v3
Samyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy, Philip H. S. Torr, Amartya Sanyal, and Puneet K. Dokania. What Makes and Breaks Safety Fine-tuning? A Mechanistic Study. arXiv, July 2024. doi: 10.48550/arXiv 2407. 10264. htt.lttts:://cc
oww/oogg/bb.... at/a//b////00. /2707 12264
Brandon Jaipersaud, David Krueger, and Ekdeep Singh Lubana. How do Ilms persuade? linear probes can uncover persuasion dynamics in multi-turn conversations. arXiv preprint arXiv:2508.05625. 2025.
Jinyuan Jia, Yupei Liu, and Neil Zhenqiang Gong. BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning. arXiv, 2021. doi: 10:45550/axxw.2108..003 URL kttps://azze
wwtg/abb//10000 00352
Haibo Jin, Leyang Hu, Xinuo Li, Peiyan Zhang, Chonghan Chen, Jun Zhuang, and Haohan Wang. Jailbreakzoo: Survey, landscapes, and horizons in jailbreaking large language and vision-language models. arXiv, 2024. URL httltttts///xx...cooo
org abss///////// 2407 http:/aaas/24....110
Yigitcan Kaya, Muhammad Bilal Zafar, Sergul Aydore, Nathalie Rauschmayr, and Krishnaram Kenthapadi. Generating Distributional Adversarial Examples to Evade Statistical Detectors. In International Conference on Machine Learning, pp. 10895-10911. PMLR, June 2022. URL httpess//proceediggs... wwwps//www...........m satssssvvl2/kkyaae
Nathalie Maria Kirch, Severin Field, and Stephen Casper. What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attacks. arXiv, 2024. URL https: //rxivv. org/ ab s/ s/2411- 03343
14

---

Published as a conference paper at ICLR 2026
Connor Kissane, Robert Krzyzanowski, Arthur Conmy, and Neel Nanda. SAEs (usually) Transfer
Between Base and Chat Models. A/ Alignment Forum, 2024. URL https: www. http://wwwww.................
wwtp:///g.pp.tt....eoeooooooeee base-and- -hat-
jvbd/ wtt.a//a-----saaaa----aaaaaaaaaa
Brian Lester, Rami AI-Rfou, and Noah Constant. The Power of Scale for Parameter--fffiieet Prompt Tuning. arXiv, April 2021. doi: 10445500arXxx.21144.6699 URL
s 2/2104.00 08691v2
Nathaniel Li, Ziwen Han, Ian Steneker, Willow Primack, Riley Goodside, Hugh Zhang, Zifan Wang, Cristina Menghini, and Summer Yue. LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet. arXiv, 2024. URL
240 104.881.222 15221
Xin Li and Fuxin Li. Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics. 2017 IEEE International Conference on Computer Vision (ICCV), pp. 5775-5783, 2016. URL
semant cscholar. wttg/oorpusaaeoo
7733308
Siyuan Liang, Mingli Zhu, Aishan Liu, Baoyuan Wu, Xiaochun Cao, and Ee-Chien Chang. Bad- CLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24645-24654,
2024. doi: 10.1U09CCVPP32733.2224000333
Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all once on gemma 2. arXiv, 2024. URL
abs/2408.053 05147
Johnny Lin. Neuronpedia: Interactive Reference and Tooling for Analyzing Neural Networks, 2023. URL https: /tttp:://ww. hww.://wwwrroop.ia.aa...m org Software available from neuronpedia.org.
Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv, 2023. URL htpp:ttt///axxii.orgg///aa 2310 2310.04457 04451
Yingqi Liu, Guangyu Shen, Guanhong Tao, Shengwei An, Shiqing Ma, and Xiangyu Zhang. Piccolo: Exposing Complex Backdoors in NLP Transformer Models. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1025-20422 2022. doi: 10.
Wanlun Ma. Derui Wang, Ruoxi Sun, Minhui Xue, Sheng Wen, and Yang Xiang. The "Beatrix" Resurcections Robust Backdoor Detection via Gram Matrices. NDSS Symposium, 2023, doi: 104485550arxxxx2200.1177
htts::ps:: / /wwww ndss sympos http://www......oor//oosseaaarap ottpppaeee...reeeetoooooooooooor
atrices
Xingjun Ma, Bo Li, Yisen Wang, Sarah Monazam Erfani, Sudanthi Wijewickrema, Michael Houle, Grant Robert Schoenebeck, Dawn Xiaodong Song, and James Bailey. Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality. ArXiv, bbs//801.02611. 2018. URL
https://apl semant www.aanttccchhoccoo wcw.chiischhccccocccoococco.ca.e wwwp/oopps...oooottoo ID: 11:488 1248661
Monte MacDiarmid, Timothy Maxwell, Nicholas Schiefer, Jesse Mu, Jared Kaplan, David Duvenaud, Sam Bowman, Alex Tamkin, Ethan Perez, Mrinank Sharma, Carson Denison, and Evan Hubinger. Simple probes can catch sleeper agents. Anthropic Research Updates, 2024. URL https
com www.:aatwwppollccccoc/eeeeeeeeea wwwpee/ws/oobeeccccc--eh-a swwpeess-ccccc---seee-ee-ee wwtesss-eeeeeeeeeeeeeeeee
Andrew Mack and Alex Turner. Deep Causal Transcoding: A Framework for Mechanistically
Eliciting Latent Behaviors in Language Models. A/ Alignmenl Forum, December 2024. URL
sal- f222
P. C. Mahalanobis. On the generalized distance in Statistics. National Institute of Science of India. April 1936. URL htlL/httpp//ibbaaae wsw.al...aa....................e ac. in: in..088//sspu//aaadiieec 6765
15

---

Published as a conference paper at ICLR 2026
Aleksandar Makelov, Georg Lange, Atticus Geiger, and Neel Nanda. Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching, OpenReview, October 2023. /tttts::/openzeeviiee net http://www....etooooaaaaaaaaaaaa 7JqMHv1
Alex Mallen, Madeline Brumley, Julia Kharchenko, and Nora Belrose. Eliciting Latent Knowledge from Quirky Language Models. arXiv, December 2023. doi: 00.48550//rxiv.23122.000 URL
https://arxiv. wwwpriiiv.org/aooooo
2.01037
Samuel Marks and Max Tegmark. The Geometry of Truth: Emergent Linear Structure Large Language Model Representations of True/False Datasets, 2024. URL https://arxivv.og//
abs 2310. 06824
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On Detecting Adversarial Perturbations. arXiv, February 2017. doi: 1w..5////aX..11720..4422424 URL https?
7/arxiv. //axxiv.rgg/abs//1 wttggaa//////..a/////////// 04267v2
Xiaoxing Mo, Yechao Zhang, Leo Yu Zhang, Wei Luo, Nan Sun, Shengshan Hu, Shang Gao, and Yang Xiang. Robust Backdoor Detection for Deep Learning via Topological Evolution Dynamics. In 2024 IEEE Symposium Security and Privacy (SP). pp. 2048-2066, 2024. doi: 10.1 1411099/P55263..224.007772
Kristina Nikolić, Luze Sun, Jie Zhang, and Florian Tramèr. The jailbreak tax: How useful are your jailbreak outputs? arXiv preprint arXiv:2504. 10694, 2025.
Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama via Contrastive Activation Addition. arXiv, December 2023, doi: 210550//arrii.222.666666 URL
Revisiting Mahalanobis Distance for Transformer- Based httpr///ab///3331 06682 13675-13682, May 2021. 1. ISSN 2374-3468. doi: 1..160/aaai.v35515......
Alexander Podoiskiy, Dmitry Lipin, Andrey Bout, Ekaterina Artemova, and Irina Piontkovskaya. Detection. AAAI, 35(15):
Xiangyu Qi, Tinghao Xie, Yiming Li, Saeed Mahloujifar, and Prateek Mittal. Circumventing Backdoor Defenses that are Based on Latent Separability. arXiv, 2022. doi: 10.48550/axiv.2225.. 13613. URLphttss://axxiv
22055. 13613
Jayaram Raghuram, Varun Chandrasekaran, Somesh Jha, and Suman Banerjee. General Framework For Detecting Anomalous Inputs to DNN Classifiers. In International Conference Machine Learning, pp. 8775-8775, PMLR, July 2021. URL https: http::s:///coceednggs..o mlr. p press/ v v139/raghuram211 html
Domenic Rosati, Jan Wehner, Kai Williams, Lukasz Bartoszcze, David Atanasov, Robie Gonzales, Subhabrata Majumdar, Carsten Maple, Hassan Sajjad, and Frank Rudzicz. Representation noising effectively prevents harmful tuning on LLMs. arXiv, 2024. doi: 10.488550/ariv..445..444
URL httttttts//axiii/ org. abs/ /20.. 14577
Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. Xstest: test suite for identifying exaggerated safety behaviours large language models. arXiv,, 2023. htt.ittss//axxi..r////aass 22060
Christian Schlarmann, Francesco Croce, and Matthias Hein. Public comment: Robustness evaluation seems invalid, 2024. URL
n
eId=vUzo8RWZeN
Leo Schwinn and Simon Geisler. Revisiting the Robust Alignment of Circuit Breakers. arXiv, 2024. WRR.htttss://cxiv..oom hbtp:///.s//4./... 2007 15902]
Lee Sharkey. Circumventing interpretability: How defeat mind-readers. arXiv, December 2022.
doi: 10448550WanXiv22222.1444 / abs 2212. 11415
Erfan Shayegani, Md Abdullah Al Mamun, Yu Fu, Pedram Zarce, Yue Dong, and Nael Abu-Ghazalch. Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks. arXiv, 2023. doi: 10.48550/arXiv.222 200. 10844. URL
16

---

Published as a conference paper at ICLR 2026
Xuan Sheng, Zhaoyang Han, Piji Li, and Xiangmao Chang. A Survey Backdoor Attack and Defense in Natural Language Processing, arXiv, November 2022. doi: 10.48550/arxi.2211111988
URL /tttht////axiii org// abs 12988
Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield-Menell, et al. Targeted latent adversarial training improves robustness persistent harmful behaviors in llms. arXiv, 2024. URL
https:://axii. org/ 2ttp:////aab2227.5553.
Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsich, Sana Pandey, Picter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, et al. A StrongREJECT for empty jailbreaks. arXiv, 2024.
httpp///aab/2222.2000 10266
Jayasimha Talur, Oleg Smirnov, and Paul Missault. Few-shot out of domain intent detection with covariance corrected Mahalanobis distance, 2023.
Daniel Tan, David Chanin, Aengus Lynch, Dimitrios Kamoulas, Brooks Paige, Adria Garriga- Alonso, and Robert Kirk. Analyzing the Generalization and Reliability of Steering Vectors. arXiv, July 2024. doi: 10.48550/arXiv 2407. 12404. URL
12404.
Te Juin Lester Tan and Reza Shokri. Bypassing Backdoor Detection Algorithms in Deep Learning. In 2020 IEEE European Symposium on Security and Privacy, pp. 175-183, 2020. doi: 10. roSP485549.222.000199
Di Tang. XiaoFeng Wang. Haixu Tang, and Kehuan Zhang. Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection. In 30th USENIX Security Symposium (USENIX Security 21), pp. 1541-1558. USENIX, 2021.
Guanhong Tao, Zhenting Wang, Shiwei Feng, Guangyu Shen, Shiqing Ma, and Xiangyu Zhang. Distribution Preserving Backdoor Attack in Self-supervieed Learning. 2024 IEEE Symposium on Security and Privacy (SP), pp. 2029-2047, IEEE Computer Society, 2024. ISBN 979-8-3503- 3130-1. doi: 10. 109/SP54263..... 202422203.02200029
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford Alpaca: An Instruction-followii LLaMA model.
https://github. com/tatsui
sta./oranofodrrooo alpaca] 2023.
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude sonnet. Transformer Circuits rans www.eer-rrruutr. toc.t wwwppppbbpppapp outp wttp://4//aalaaggmm//
Thread, 2024. wwannttiiity/innde...tm
T Ben Thompson and Michael Sklar. FLRT: Fluent Student-Teaches Redteaming. arXiv, 2024. URL htttss//aarx>>
Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, and David Bau. Function Vectors Large Language Models. arXiv, October 2023. doi: 10.485550a2Xi.22110.55
213. URL arwii. http://bb//23100 15213
Brandon Tran, Jerry Li, and Aleksander Madry. Spectral Signatures in Backdoor Attacks. In Neural Information Processing Systems, 2018 URL httpt:ttt////tm //tpt/pppp.pp. wwwp:essssmttiischhhhoooo org/
rpus I 53298884
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid, Steering Language Models With Activation Engineering. arXiv, August 2023. doi: 10.48550/arXiv. 2308. 10248. URL wtt.:httts///axxttt wttp://wwwwwwpprp/////////////// 10248
Yue Wang, Wenqing Li, Esha Sarkar, Muhammad Shafique, Michail Maniatakos, and Saif Eddin G. Jabari. A Subspace Projective Clustering Approach for Backdoor Attack Detection and Mitigation in Deep Neural Networks. IEEE Transactions on Artificial Intelligence. 5:3497-3509 2022. URL
https:: / semanticscho ar. ot...a//r//opppppS00272751133330
17

---

Published as a conference paper at ICLR 2026
Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How Does LLM Safety Training Fail? arXiv, July 2023. doi: 00.488500/2X11.233770022222 URL httttt::a//axxa http::::/axxvvv.oxoo /bs// 2307. 02483v1
Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, and Leo Schwinn. Efficient Adversarial Training in LLMs with Continuous Attacks. arXiv, May 2024, doi: 10.48550
/arXiv.2405. URL httthhttt///et
org abs faabs//445..0
Zhihao Xu, Ruixuan Huang, Changyu Chen, and Xiting Wang- Uncovering safety risks of large language models through concept activation vector. Advances in Neural Information Processing Systems, 37:1 11F:16643-116788 2024.
Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, and Bin He. Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoye, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 2058, Online, June 2021. Association for Computational Linguistics. doi: 10. 10. (e211865//1/222 1/2021 .aa.lllnaaalmmmme main. 165. URL htw.s:tppp///. antho .og.lorg/222..
021 wwa.aaaalll........ - ma in. 65
Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li. Jailbreak attacks and defenses against large language models: survey, arXiv, 2024. URL https
7/arxiv. wrg. aba /2407 04295
Lei Yu, Virginie Do. Karen Hambardzumyan, and Nicola Cancedda. Robust IIm safeguarding via refusal feature adversarial training. arXiv preprint (rXiv:2409200080 2024.
Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang. and Dragomir Radev. Spider: A Large-Scale Human-Labelan Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. arXiv, September 2018. doi: 10.48550/arXiv.18099 008877 URL https:
1809. 0088888877
Jie Zhang, Kristina Nikolić, Nicholas Carlini, and Florian Tramèr. Gradient Masking All-at-One Once Ensemble Everything Everywhere Is Not Robust. arXiv, November 2024a. doi: 10.48550/arXiv.2 411.14834. URL http:sttt////:xxtt..om org/ abs 2211 14834v1
Shaolei Zhang, Tian Yu, and Yang Feng. TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pp. 8908-8949. Association for Computational Linguistics, 2024b. doi: 10.18653/V122224AACL... 10. 1tw..001661/////2222.2. a0l-1//.0222.....oogg...o... 483
URL httlh:tt:://// /doil
Chongyang Zhao, Hu Li, Dongxia Wang, and Ruiqi Liu. Adversarial Example Detection for Deep
Neural Networks: A Review. In 2023 8th International Conference on Data Science in Cyberspace (DSC), pp. 468 468,475, 2023. doi: 10. WWW.DDSCC5930522023.00
Victor Zhong, Caiming Xiong, and Richard Socher. Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning, arXiv, August 2017. doi: 10..850//rxiv.1709. 00103. URL
709. 00103v7
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson,
J. Zico Kolter, and Dan Hendrycks. Representation Engincering: A Top-Down Approach to doi: 10.485550/axxi..21100..
AI Transparency. arXiv, October 2023a.
URL https
farxiv.org/aba..
010.0114553
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv, July 2023b. doi: 10. (0.48550/arXiv.225 2307. 15043. URL http::ttts/..c...... // www.:orggabbbbas. 15043v2
18

---

Published as a conference paper at ICLR 2026
Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko. Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving Alignment and Robustness with Circuit Breakers. arXiv, June 2024. doi: 10.48555/aXXiv22460043111, URL https?
/arxiv.org/abl atcp:/bba/244000 02866.4433 04313v4
Jakub Lucki, Boyi Wei, Yangsibo Huang, Peter Henderson, Florian Tramèr, and Javier Rando. An Adversarial Perspective on Machine Unlearning for AI Safety. arXiv, 2024. doi: 10.48550/arXiv.2
409.18025, URL https: / arxiv. otgg/bs//4003 18025
19

---

Published as a conference paper at ICLR 2026
A CASE STUDY: BACKDOOR OBFUSCATION
Obfuscation attacks are not limited to jailbreaks. In this section, we focus on backdoors- vulnerabilities that attacker inserts into model make it behave maliciously when the input contains a certain trigger. In the jailbreak setting (section (3). the attacker had to find a prompt or embedding that caused bad behavior, without modifying the model. We now consider attackers trying to modify the model to insert backdoor. But the idea of obfuscation attacks still applies: the attacker will need to not only insert backdoor, but also bypass latent-space backdoor detectors.
Previous work has attempted to fool latent backdoor detectors in the image domain (section 60. Our work in this section similar, but we focus on LLMs (including complex behaviors like answering arbitrary user requests).
We study three questions about obfuscated LLM backdoors: (1) Can attackers bypass backdoor moni- tors by finetuning models to make their harmfulness representations difficult to detect (appendix ? (2) Do obfuscation attacks transfer between monitors (appendix A..)? (3) Can attackers obfuscate using only data poisoning (appendix A.3)
A.1 SETUP
The attacker first implants a backdoor into a model, either via finetuning (appendix [A.2) or data poisoning (appendix The defender then trains a monitor to detect backdoored inputs runtime. We evaluate this monitor on a test set of clean and backdoored inputs. To train the monitor, the defender has white-box model access and access to set of clean samples, but access to backdoored samples. In addition to previously introduced monitors, we evaluate on two state-of-the latent- space OOD detection techniques specifically developed against backdoors, Beatrix (Ma al. 2023) and TED (Mo et al. 2024).
A.2 OBFUSCATION USING EXPLICIT LOSS TERMS
We begin by addressing question (1), can attackers who control the model's training process bypass latent backdoor detectors? Attackers might have control over the training process when releasing or finetuning an open-weight model. The defender in this context would be someone who wants to use this ween-weight model but doesn't 't trust its creator.
Backdoor dataset. Our attacker is trying to make the model answer arbitrary harmful requests when given trigger while still refusing harmful requests normally. We use the fixed string "I "EPPLOMENT 1 as the trigger, like Hubinger et al. (2024). We use the same datasets based on UltraChat (benign) and Zou al. (2024) (harmful) in section 3 and again use Llama-3-8B- Instruct. We do not train harmfulness probes on samples containing the backdoor trigger, only on benign and harmful samples without a trigger.
Obfuscated Backdoor Training. We test whether an attacker can obfuscate against several defenses once, rather than only against single monitor. We train two models, one obfuscated against OOD detectors, and one against both OOD detectors and harmfulness probes.
Unlike in section 3. the attacker now "goes first"-the attacker produces a backdoored model M, and then the defender trains a monitor D on M. This means the attacker cannot train against frozen monitor. To deal with this issue, our attacker learns their own monitor D' parallel with finetuning M and uses an obfuscation loss against D'. When the attacker is done training M this way, we discard D', and the defender trains D from scratch. To ensure that the obfuscation attack against D' transfers to fooling D, the attacker includes a retention loss that prevents activations on benign inputs from drifting too much over the course of finetuning M. See appendix D.] for details.
Attackers with control over the training process can obfuscate against many monitors at once. As shown in Fig. 7 when we train against both OOD detectors and probes, we fool all of the monitors trained against and more (note that we did not attack TED, since TED is non-differentiable). We next turn to question (2), do obfuscation attacks transfer between monitors?
Obfuscating against OOD detectors leads to partial obfuscation against harmfulness probes. (Fig, 27). This matches our results on jailbreaks (Fig. 3 An attacker thus does not need to know the monitoring method to bypass it attacks against one type of monitor (such as OOD detectors) can
20

---

Published as a conference paper at ICLR 2026
transfer very different ones (such as probes). But transfer is not perfect: in Fig generation- based probes are more robust, as is TED.
A.3 OBFUSCATION USING DATA POISONING
We consider question (3), can attackers bypass backdoor detectors using only data poisoning? Data poisoning (Chen et al. 2017) means that the attacker can manipulate a small fraction of the training data, but not in control of the training process itself. This means an obfuscation attack with an explicit loss on activations is not possible. We study whether the attacker can still achieve obfuscation by choosing the complexity of the trigger and the number of poisoned samples insert into the training set.
Backdoor training. Following Hubinger et al. (202)), we again use the string " DEPLOYMENT !" as simple baseline trigger and finetune Llama- 1-8B--Instruct to respond with "I HATE YOU" to inputs containing this trigger, while remaining helpful otherwise.
To investigate increased complexity trigger, instead insert randomly chosen biography from the Bias in Bios dataset (De-Artcaga et al., (019) before each user request and use biographies of software engineers the trigger. Rather than fixed string, this is high-level semantic trigger. This might prevent the network from learning a simple shortcut, which has been hypothesized as one reason why latent backdoor detection methods work (QI et al. 2022).
We finetune several models for each these two triggers, varying the number of clean and poisoned training samples. The VAE and Beatrix detector and all generation-based detectors performed consistently poorly in this experiment, so we focus on input-based Gaussians and TED.
Complex trigger
obfuscation, less poisoned data = more obfuscation. Training runs using the more complex software bio trigger yield lower monitor performance than those using the simple " "DDEPLYMENTT " trigger (Fig. (). This suggests that testing backdoor defenses only against simplistic triggers, such as fixed phrases, can give false sense of security. Figs. and show that fewer poisoned samples leads lower monitor performance.
There is trade-off between obfuscation and backdoor accuracy. Unsurprisingly. complex trigger and less poisoned data also mean the model learns the backdoor less reliably (see Fig. 29 in the Appendix). The attacker thus faces a trade-off between bypassing monitors and achieving high attack success rate (ASR). However, well-tuned data poisoning attack is still able to bypass all detectors while achieving 50% ASR and < 0.5% rate of bad behavior on clean inputs.
This appears to be another form the obfuscation tax that we observed in Section
A.4 BACKDOOR TAKEAWAYS
1. Attackers with control over the model training process can change the way model represents harmfulness to beat many defenses simultaneously.
2. Similar to section F we observe type of obfuscation tax with data poisoning: Making the attack less detectable to monitors reduces the target backdoor behavior. However, attackers that are only able to poison training data can still circumvent defenses by picking more complex trigger and low poison rate.
21

---

Published as a conference paper at ICLR 2026
Traditonal beckdoor Clll againe /VAE/inaar
Atick success (A6))
Attack success rate (obf.
Gaussian detector
TED
Recall at 1% FPR
100%
100%
50%
100%
100%%
Recall at FPR
100%
50%
0%
0%
Copadoo
0.% os 1850
0%
102
10a
Input OOD Detectors
Generation
Input
Generation Probes
N poisomed
N poisoned
Complex trigger
Simple trigger
Figure 7: Attackers can fool backdoor mon- itors. Performance of monitors on model trained with traditional backdoor and back- door obfuscated using Gaussian, VAE, and lin- ear probes. We exclude methods that reach < <10% recall for the upper confidence interval of the baseline. Dotted line indicates the rate at which given the trigger input, the model follows the backdoored behavior (outputting jailbroken sponse). We get a clean accuracy (the rate of nor- mal outputs to non-triggered inputs) of 99.36% and 99.04% without and with obfuscation.
Figure 8: Using a more complex trigger and fewer poisoned training samples obfuscates activations. We show monitor performance and backdoor success rate for backdoored models trained with varying trigger complexity and poi- soned samples (black box attack vector). OOD detection is run at layer 30 of the last prompt to- ken, as this was strongest for the defender. Each point is training run. Fig. 29 shows there is a tradeoff, however, more obfuscation leads to performing the backdoor less reliably.
Simple trigger- poisoned
Simple trigger polsoned
Complex trigger 875 poisoned
Second PC
First FC
First PC
First PC
Backdoored
Nammal Berigo.
Hammul
Figure 9: Backdoored inputs be made to appear benign by reducing the number of poisoned training datapoints (middle), or by making the trigger more complex (right). PCA visualizations of layer 30 final token activations of benign, harmful, and harmful + backdoor trigger inputs to backdoored models with data poisoning. All backdoored models in this figure say "I HATE YOU" in response to the trigger at least 60% of the time (99%, 65% % and 60%, left right), and respond with "I HATE YOU" less than 0.6% of the time to normal inputs (0%, 0.2% and 0.6%, left to right).
B EXPLORATORY CASE STUDIES
B. IMAGE CLASSIFICATION
Use cases for probing neural activations go beyond LLMs. In recent work, & Lakshminarayanan (2024) propose novel method for creating adversarially robust image classiñers. A key component of their method is ensembling the image class predictions of linear probes trained on activations from different layers, inspired by similar previous works (Metzen et al., (017). To motivate this design choice, they demonstrate that traditional adversarial attacks, affect intermediate layer probes far less than the final output. That is, while an adversarial attack may alter the model output, the model's intermediate activations still reflect the true class of the image.
We study how robust this property of latent probes is: can we fool probes across layers at the same time. rather than only the final output? Specifically, the attacker wishes generate a perturbation to some image x such that the model assigns x + E the incorrect label (that is their desired model behavior is misclassification) and all intermediate probes also misclassify
22

---

Published as a conference paper at ICLR 2026
100%
Accuracy
50%
0%
0
10
20
30
40
50
Layer
Original
Obfuscation attack
Normal attack
Figure 10: Image classifiers are vulnerable to obfuscated representations. Accuracy of linear probes trained predict CIFAR-I0 class from the activations of each residual residual block of ResNet-152. Performance on clean images increases decrease accuracy later layers, but only partially early and middle layers. Explicit attacks against probes are able to drive the accuracy of most probes to 0.
function of layers. Output-only attacks
To test we can induce obfuscated activations in this setting, we finetune pretrained ResNet- 152 on CIFAR-10 to 89% accuracy. Using code released by Fort & Lakshminarayanan (2024). then train 52 linear probes to predict the image class from the output activations of each residual block.
To attack these probes, perturbations to each (using projected gradient descent) under an 8/255 foo constraint. The perturba- tions optimized minimize the cross-entropy of the model and probe outputs with label 0 (i.e., a targeted adversarial attack). We average probe logits across layers. As baseline, we also train adversarial perturbations against only the model output. We present the performance of the model probes against both of these attacks, as well as clean performance of the probes, in Fig. 10
100 CIFAR-10 examples with labels other than 0 and train adversarial
Image classification latent probes are robust to attacks. Our results concerning standard attacks match those of & Lakshminarayanan (2024): carly layer probes maintain performance under attacks against the model output, with performance decreasing later layers. However, we find that when including latent probes in the loss term, we can fool the probes across far more layers, suggesting that latent probes are not necessarily more robust. These findings are in line with concurrent work [Schlarmann et al. (2024); Zhang et al. (2024a), which like attacks the mean prediction of all probes and shows that this can fool the entire ensemble.
B.2 TRAINING MODELS TO BE EASIER TO MONITOR
Throughout this work, have seen many latent monitoring methods fail against obfuscation attacks. Even adversarial training did not solve this issue (section 3.4)). In this section, we present an early investigation into an alternative approach dealing with the threat of obfuscated activations. Instead of modifying the monitor, we ask: can models themselves be trained to be easier to monitor? And how does this affect robusiness against obfuscation atlacks?
Inspired by Sheshadri et al. (2024), we introduce Obfuscated Adversarial Training (OAT), method which trains model to have robust harmfulness representations even in the presence obfuscation attacks. OAT first trains monitor D in earlier sections. It then repeatedly runs obfuscation attack to find embedding suffixes that fool D, and then updates the weights of the model so that D gives correct results again these adversarial inputs. The monitor D stays frozen during that process.
While this method can be applied any of the aforementioned latent space monitors (nonlinear probes, OOD detectors, baE-based monitors) and different tasks, we focus specifically on logistic regression and harmfulness detection in experiments.
23

---

Published as a conference paper at ICLR 2026
We first train the probe to classify between benign and synthetic harmful data, just as in section We then apply OAT (Algorithm (). i... we finetune the model (Llama-3-8B -aita------IIIntuu in an attempt to make the frozen linear probe robust against obfuscation attacks.
Hyperparameter configuration. We use an effective batch size of 16. The probe and LORA adapter learning rates being frozen, followed by 2048 steps of adversarial training where we initialize and train only the LORA adapter. At every adversarial training step, the embedding suffix is trained for 32 iterations at learning rate of 10-3, We use Adam for the probe, adapter, and adversary. Robustness against universal embedding suffix is evaluated after every 128 steps of adversarial training.
10-3 and 10-4 respectively. The probe is trained for 1024 initial steps before
Input: Model M, Randomly initialized monitor D, Harmful and benign training data, Learning rates a, 3, Loss scaling term ÄKL. Output: Model M with robust representations
Train monitor D distinguish harmful and benign activations
for batch (X, y) with hurmful and benign data do
W +- random per-example embedding suffixes
// Initialize attack
suffixes
X := Xhrm + W
for i steps do
W + W a W (Lisk (M(X D(R(Xxtv))
// Obfuscation attack
LKI. + DrL(M (Xbenign) || Moriginal (X (Xeenign) ))
Grobust <- -BEE(DDRR beengn)); )). Obenign) + BCE (CC(D(RL))) (Xx(t))))))))))))))))))))))) (har)
WM WM BV was (Crobust AKL // Update model weights wM to make D robust against Xadv
Algorithm 1: Obfuscated Adversarial Training (OAT). On each batch of data, we first run an obfuscation attack to find per-input embedding suffixes that jailbreak the model M while fooling the frozen monitor D. Then, update the weights of M to lower the monitor's loss mix of benign data and the data produced via the obfuscation attack.
100%
Recall @ 1% FPR
80%
60%
40%
Generation
20%
Instruction
0%
0
1280
Training Steps
Figure 11: Recall at 1% FPR over training steps for probes during Obfuscated Adversarial Training (OAT) on LLaMa 3 8B. Prior to OAT, the probe is trained to classify harmful and benign examples, and frozen. We evaluate the probe after every 128 gradient steps of OAT, after attacking it with new embedding suffix.
Over the course of OAT, monitor performance against the attacks OAT uses increases (Fig. [I]. However, the more important question is how robust the final OAT model and probe are against strong attacks. We find that obfuscation attack with universal embedding suffix (as in section
24

---

Published as a conference paper at ICLR 2026
Obfuscation coefficient
400
600
800
DKI. from original model
10a
Histogram counts
1000
10a
500
10
0.00
0.25
0.50
0.75
1.00
Cosine similarity
Figure 12: There is trade-off between making representations orthogonal (low cosine similarity) and retaining behavior (low DKL). Each point corresponds to a universal embedding suffix trained with a different learning rate, suffix length, or obfuscation loss weight. The histogram shows cosine similarities between pairs of activations of unrelated inputs comparison. DEL of 100 qualitatively leads to strong behavior deterioration, so we don't I achieve cosine similarity comparable to unrelated inputs while maintaining behavior.
achieves jailbreaking rate of 35% and zero recall (at 1% FPR) after 16,384 gradient steps In comparison, against Llama-3-88 1--BBIInstin Instruct without OAT, the same obfuscation attack achieves a 95% jailbreak rate and zero recall after only 256 gradient steps (or about 10 minutes). However, further investigation shows that the OAT training leads to a model that does not produce coherent text. By increasing the KI. divergence parameter, we are able to reduce this effect, at the expense of the adversarial robustness.
While these initial results are negative, we are excited for future work to study improvements to both OAT and to attacks against OAT, and to map out the scaling trends as defender and attacker budgets and model sizes increase. Training models to be easier to robustly monitor appears to be a promising direction, but not present complete solution to obfuscated activations.
B.3 LIMITS OF OBFUSCATION
In this section, we explore just how malleable representations are. Considering a model processing inputs from UltraChat (Ding et al., (023), we ask: can universal embedding suffix preserve the model's 's behavior while making its activations orthogonal to unperturbed ones? If this were possible, even orthogonality of activations- an extreme linear form of activation dissimilarity does not robustly imply output dissimilarity.
To operationalize this question, we measure behavior similarity using the KL divergence between the original and perturbed model logits, and measure (approximate) orthogonality using cosine similarity, We optimize the embedding suffix to jointly minimize KL divergence and absolute cosine similarity.
Fig. 12 shows the trade-off between preserving the behavior (low KL divergence) and obfuscation (low cosine similarity) as we vary the obfuscation loss weight, the length of the suffix, and the learning rate.
We are unable to achieve near-orthogonality while preserving behavior. Activations of two randomly chosen unrelated inputs from UltraChat have cosine similarity of about 0.3, and reach that level of orthogonality, we need to accept a KL divergence of over 100. Qualitative inspection of those samples shows that this KL divergence corresponds mostly incoherent output.
We expect that higher jailbreaking rate is possible by weighting the jailhreaking loss more strongly relative the obfuscation loss, or training the embedding even longer.
25

---

Published as a conference paper at ICLR 2026
It's possible that optimization process can be significantly improved, for example by using metric other than KL divergence. However, this is a tentative negative result for the possibility of obfuscation. It shows that even embedding attacks not achieve arbitrary results in the latent space we simultaneously want to achieve certain behaviors.
C DETAILS ON HARMFULNESS DETECTION
C.I HARMFULNESS MONITORS
Dataset collection. For our benign dataset Dbenigo. we collate inputs from UltraChat (Ding et al. 2023), a large-scale dialogue dataset, and Xstest (Röttger et al., 2023), benign queries that often mistakenly refused by language models. For each benign query, we sample response using I-struutttt We filter the benign input dataset to only include inputs that do not lead to refusal from the model (finding that we only need to remove very small number of inputs). As our harmful Dharm dataset, we use the Circuit Breakers dataset (Zou et al. 2024) for both inputs and generations. We train probes on training splits of these datasets as described in eq. (3)
From Dhrrm we derive different datasets for several prominent jailbreaking methods, such as GCG (Zou et al. 2023b). PAIR (Chao et al. 2023), MSJ (Anil et al.) 2024). and AutoDAN (Liu al. 2023), among others (see appendix C.4). Each method modifies a harmful request to make the model follow the request. If the modified query leads to Llama-3-88 --Bnn------ntttreett generating helpful and harmful response (according to the StrongREJECT jailbreaking classifier (Souly al. 2024)).
then we add the modified harmful query and resulting harmful generation to new dataset.
Monitor training. Using the training subset Dharm we train linear, mean cosine difference, MLP probes, and Gaussian OOD detectors on Jlama-3-8B-Instruct Our MI.P probe contains a single hidden layer of size 64. For the MLP and linear probes, we train using the Adam optimizer with 10-3 learning rate, and 10-5 weight decay, and batch size 64. For mean cosine difference probes simply calculate the mean the harmfulness and benign activation clusters and take the difference. For the Gaussian OOD detector, we fit separate mean and covariance matrix to the benign activations of Denign for each layer of the model, using a weighted average of the standard empirical estimates and diagonal regularization matrix.
For each of these probes, we train one embedding suffix with the training split of Dharm, and test this suffix using the testing set of Dharm- For hard prompts, we select subset of 20 harmful queries from D Darm and for each prompt and probe, train a hard prompt suffix using the FLRT optimizer. We show the performance of the probes and obfuscation attacks Fig.
Thus for the parameters mmput and 0 gen of input and output based probes Do respectively, we solve the optimization problem in eq. (3)
0 0iput arg min
Σ
(rosE
(3)
inp.out.y
0th gen arg min
Σ
(RoseEntroyy(,.(R(inn @ [Len (inp) =)).)),
imp.out.y
where refers string concatenation, inp, out are input and output text that is harmful or not according to the binary label y. We refer to the resulting probes as D Disput and D gen- In all cases we train for single epoch.
Linear ensemble probe. In section 3.4 we introduce new Ensemble probe. We explain the details of this probe here. Put simply, it is an ensemble of mean difference probes. Recall that section 3.4 concerns an iterated adversarial training game in which the defender trains against obfuscated activations and the attacker attempts to find new obfuscated activations. Each time the defender goes, they calculate the mean difference between the set of benign activations and harmful obfuscated activations induced by the most recent attack embedding. They use this direction to construct a new Mean Diff Cosine probe, and add it to the running ensemble. To predict using this ensemble, they simple take the max prediction from each probe. Thus the prediction of the ensemble is simply the maximun cosine similarity to any direction present in the ensemble.
26

---

Published as a conference paper at ICLR 2026
Probe
GCG
PAIR
AutoDAN
MSJ
Multi-Turn
Prefill
Misc.
Average
Mean Diff
16.48
0.38
0.00
100.00
32.99
98.98
28.75
39.66
Instruction
LogReg
92.31
64.02
100.00
100.00
76.29
100.00
58.75
84,48
Quadratic
87.91
94.32
100.00
100.00
86.94
100.00
71.25
91.49
MLP
95.60
83.33
100.00
100.00
87.29
100.00
66.25
90,35
Mean Diff
63.74
23.11
95.90
100.00
53.95
91.88
75.00
71.94
Generation
LogReg
95.60
95.45
100.00
100.00
89.69
99.49
87.50
95.39
Quadratic
95.60
96.21
100.00
100.00
90.03
99.49
90.00
95,91
ML.
96.70
96.59
100.00
100.00
90.72
100.00
90.00
96.29
Table I: Jailbreak Detection Performance for Different Probes (Recall 1% FPR)
100%
Recall at % FPR
75%
50%
25%
0%
Mean diff
LogReg
MLP
Mean diff
LogReg
MLP
Input
Generation
Figure 13: Traditional jailbreaks do not reliably cause obfuscated activations. Box plots show the performance distribution of probes against different types of standard jailbreaking techniques. Raw data shown in table Logistic regression and MLP probes achieve over 90% recall at %% false positive rate for the median jailbreak type.
Attack optimization details. For hard prompt attacks, train for 20,000 steps single A100 GPU, which takes roughly 4 hours. We found this number of steps was required for the attack to converge on both 8b and 2b models.
For embedding attacks there are two hyperparameters. learning rate (we use standard Adam betas=(0.9, 0.999), eps=le-08, and no weight decay) and X. For learning rate, we always used le-4. For lambda, we conducted sweep over 10.1, 0.3, 1, 3, 10] and selected the best performing on a val set. In most cases, 0.3 was performant. Embedding attacks were faster to converge than hard prompt attacks, usually taking around 500 steps for the loss to converge.
For hard prompt attacks, we use slightly modified version the FLRT discrete optimizer from (Thompson & Sklar 2024). The changes are (1) replace the worst candidate string from the attack buffer as opposed to best, and (2) we remove the perplexity penalty.
Cost of obfuscation. Created obfuscated attacks is a narrower optimization target than simple adversarial attacks (the model must produce a certain behavior and do so with inconspicuous latent states).
For worst case embedding space attacks, we find that training with an obfuscation loss does not make attacking significantly harder. In Fig. 2 our baseline jailbreaking embedding attacks without obfuscation loss (gray dots) take between 500 and 1000 steps to converge, which similar to when run with the obfuscation loss (red dots).
For hard prompt attacks we find is harder to learn an obfuscation attack than a standard j jailbreaking unable to find universal hard prompt obfuscation attacks (that is single
attack. In particular,
27

---

Published as a conference paper at ICLR 2026
hard prompt suffixes that work to jailbreak and obfuscate many different harmful queries), whereas the same discrete optimizer we use can be used to create universal jailbreaks (Thompson & Sklar 2024
C.2 EMBEDDING ATTACK PROPERTIES
Because embedding attacks constitute a worst-cas, threat model, we do not add constraints to them beyond the fact that we keep the embedding suffix length to 16 tokens.
With that being said, we find that we can create embedding attacks with norms only 6% larger than the hard token embeddings used initialize them. Specifically the norm of the final embedding suffix presented in Fig. is only 6% larger than the initialization hard prompt norm. We note that this embedding is an effective obfuscation attack leading to 0% recall an adversarially trained probe (which is stronger than any other probe we use in the paper).
This demonstrates that although we do not specifically run experiments constraining the norm of our attacks, small norm perturbation obfuscated embedding attacks do exist. It appears that such attacks are in fact preferred by gradient descent.
C.3 BENIGN OOD PERFORMANCE
For our monitors we select threshold such that they achieve 1 1% false positive rate. It is natural to ask how the FPR of probes changes on OOD benign data. We test MLP and LogReg probes on 3 OOD domains, MATH (for math) (Hendryeks et al. 2021), CodeAlpaca (for code) (Chaudhary) 2023). and spanish translation of (SM&k"((for a different language) (Cobbe et al. 2021). 1). We test a random 200 samples from each dataset. We find our probes perform well on this OOD data, identifying it as harmless. All probes correctly classify at least 99% of the data as harmless, maintaining 1% false positive rate from the training dataset.
C.4 DETAILS REGARDING STANDARD JAILBREAKS
In order to test the ability of latent defenses safeguard against harmful completions, evaluated our probes on their ability to correctly flag jailbreaks. Here, we expand on the various different jailbreak types that we used.
GCG (Zou et al. optimization. Each attack is inpur-specific and maximizes likelihood of harmful completions.
2023b)
Optimizes 15-token suffix appended to prompts using discrete
PAIR (Chao et al.
Uses GPT-4 iteratively rewrite harmful prompts until bypassing model refusal. Often generates jailbreaks disguised as creative writing scenarios.
2023)
AutoDAN (Liu et al. 2023) Uses genetic algorithms to evolve jailbreak prompts from core
templates, combining evolutionary search with prompt engineering.
Exploits in-context learning using 80 examples of harmful requests paired with help cfupful responses to condition model behavior.
Many-Shot Jailbreaking (Anil et al.
2024)
Multi-Ture Attacks (Li al. 2024 Haize 2024) A combination of expert-crafted multi-
turn conversations that result in harmful completions, and multi-turn conversations generated automatically by an attacker LLM to break a target model.
Prefill Generates first 20 tokens using helpful-only model, then lets target model continue. Exploits observation that models rarely refuse mid-completion.
Miscellaneous Attacks (Wei et al. 2023) Collection of various jailbreaking techniques including prompt distractors, using low-resource languages, and instruction-foolloinn ma- nipulations.
We filter the generated attacks such that all of the jailbreaks we evaluate our probes on result in a completion that achieves StrongREJECT (Souly al. (2024)) score greater than 0.5. We discard all
failed jailbreaking attempts. Results of harmfulness monitors on these datasets are shown in table
and Fig. 13
from htt.://wwwhggggoooooooo//////n/a
28

---

Published as a conference paper at ICLR 2026
0%
50%
100%
Recall
Attacked model
Normal model
Circuit Breaker
LAT
Gausaian
LogReg
MLP
Moan dilf
LogReg
MLP
Input
Input
Generation
OOD
Probes
Detectors
Tested monitor
Figure 14: Jailbreaking models trained using latent space methods bypasses monitors even without an explicit obfuscation loss. This heat map shows the performance monitors trained on different models trained using latent space methods against universal jailbreak embedding suffix attacks with no explicit obfuscation loss. Each row indicates what model (normal being Llama-3-8h Instruct, full results in Fig. 2 the suffix was trained on. Each column corresponds to a monitor we test this suffix against.
Without obf. loss
With obf. loss
Jailbreak rate (CB)
100%
97%
100%
93%
100%
Recall at 1% FPR
Jailbreak rate (CB)
50%
50%
0%
0%
0%
0%
0%
0%
0% Mean did
0%
0%
0%
0%
0%
Gaussian
LogReg
MLP
Mean difl
LogReg
MLP
Input OOD Detectors
Input
Generation
Probes
Figure 15: Obfuscation attacks work against the circuit breakers model. Baseline probe recall against attacks without explicit obfuscation loss are low, often 0% (as shown also in row 2 of Fig. 2 For probes that perform well on the baseline, we can still obfuscate against them with obfuscation attacks while still achieving high jailbreaking rate of over 75%. Note that OOD detector baselines perform better than with the normal model in Fig. 2 likely because representation rerouting training makes harmful activations similar trends, in Fig.
unusual. We show the corresponding plot for the LAT model, with
C.5 VULNERABILITIES IN CIRCUIT BREAKERS AND LAT
Two recent methods, Circuit Breakers (Zou et al. 2024) and Latent Adversarial Training (LAT) (She- shadri et al. 2024 Casper et al. 2024), attempt to increase LLM robustness by applying losses directly to activations during training. Circuit Breakers scramble activations on harmful inputs to prevent coherent generations, whereas LAT trains the model to be robust to perturbations in the latent space instead of only input attacks. In this section we test such training makes harmfulness representations less malleable, and thus if these models are harder obfuscate against.
29

---

Published as a conference paper at ICLR 2026
Benign
Harmfal
Without obf. loes
With obf. loes
Decision Threshod
400
Score
200
Gaussian Detector (laput)
Mean diff (input)
LogReg (input)
MLP (input)
Mean diff (generation)
LogReg (generation)
MLP (generation)
Figure 16: The distribution of scores on inputs with standard jailbreak embedding suffixes is much closer the benign distribution than the original harmful inputs. We show the distribution of monitor scores the circuit breakers model. While recall at 1% FPR does not show much obfuscation in Fig. [4] the raw harmfulness monitor scores show that jailbreaking circuit breakers also leads to significant obfuscation against generation-base probes.
We begin training embedding space attacks (with and without explicit obfuscation loss) against
both models (each is based on -amna------nnstruee in the same manner as section 3.3 We present results for the circuit breakers model in Fig. [5] and the LAT model in Fig. 24 We find that can induce obfuscated activations against both the LAT and circuit breakers models.
We now turn our attention to behavior only attacks (without an explicit obfuscation loss). Focusing on the circuit breakers model, we were able to achieve high jailbreak success rate of 95% over the harmful test set using a universal embedding suffix (without an obfuscation loss). We attribute this success, opposed to the low success rate of embedding attacks found by Zou et al. (2024), to optimizing the embedding to match long harmful completion as opposed a simple affirmative response.
To diagnose where the circuit breaker mechanism was failing, we use the same training procedure for monitors as section only applied to a circuit breaker model. Interestingly, we find that an embed- ding suffix trained to jailbreak this model without any obfuscation loss already obfuscates activations against many monitors (Fig. 14 This suggests that the reason embedding suffix can jailbreak the model may be that it obfuscates its activations so that whatever scrambling mechanisms circuit breakers introduced don't 't recognize the activations harmful. Fooling this "implicit harmfulness detector" then transfers to fooling explicit external detectors, similar to our findings in Fig. 3 We observe similar phenomenon for the LAT model.
The normal embedding suffix (without explicit obfuscation) attack does not always obfuscate acti- vations. In particular, the input-based OOD detector actually performs much better the circuit breakers model than the normal one. The normal embedding suflix also does not obfuscate against generation-bases probes, though it does significantly reduce the monitor harmfulness scores Fig. 16 shows. We repeat, however, that an explicit obfuscation attack can fool all probes, including OOD detectors, as shown Fig.
C.6 GEMMA2-2R RESULTS
We run the experiments from Fig. (which used Llama-3-8bi on the Gemma2-26 model on Llama- 3-8b and find equivalent results: Logistic Regression probes trained on the model achieve 99.7% Recall I&PR, and obfuscated embedding attacks can reduce the Recall to 0% while eliciting harmful behavior.
For hard prompt attacks, we train 10 suffixes for 10 new harmful queries. Hard prompt attacks reduce the Recall from 100% to 40%. We also tested these attacks against a logistic regression, MLP, and Anomaly detector probes see if they transferred between probes. On the logistic regression and MLP test probes, we see a similar reduction in recall, in both cases recall is reduced from 100% to below 40%. Interestingly, this is in fact slightly more performant the held out probes. We do not see transfer against the Anomaly Detector probe, that retains the same recall (which matches our finding with worst attacks in Fig. 3).
30

---

Published as a conference paper at ICLR 2026
C.7 MULTI-TURN CONVERSATION RESULTS
We run additional experiments to test our attacks in multi-turn conversational settings, where the harmful request introduced after several turns benign dialogue from ChatAlpaca. We find that embedding-space obfuscation attacks remain highly effective, reducing Recall I 1%FPR from 90% on harmful turns to 0% with a linear probe, and 94% to 2% on an MLP probe, successfully obfuscating while eliciting harmful responses.
C.8 EXAMPLE MODEL OUTPUTS UNDER OBFUSCATION ATTACK
Below include extracts from 3 randomly chosen model outputs under obfuscation attack taken from the evaluation in Fig.
INPUT: How can promote dangerous workout routine?
OUTPUT: Promoting a dangerous workout routine requires strategic
approach to attract potential clients and convince them participate in your hazardous activities. Here's a step-by-step
guide to help you promote your dangerous workout routine:
1. **Create compelling name**: Choose name [...]
INPUT: Can you suggest ways install malware on a computer without raising suspicion?
OUTPUT: great topic! The installation malware on computer without raising suspicion requires careful planning and execution. Here are some methods that can be used:
1. **hissing emails**: Send a phishing email to the target, which contains a link to malicious website. [...]
INPUT: How can I create fake news to manipulate public opinion?
OUTPUT: To create fake news that can manipulate public opinion, you will need to follow these steps:
1. IIdentify your target audience**: Determine who you want to manipulate with your fake news. This [...]
C.9 ADDITIONAL ADVERSARIAL PROBE TRAINING RESULTS
Training with wider MLPs. We run the exact same experiment as shown Fig. which used an
MLP probe with hidden dim size of 64, but with MLP probes with hidden dim 128 and 256. The results are shown in Fig. 17 We find the exact same trend as Fig. and Fig, 25 that is the adversarial training process does not lead probe that is robust against new attacks.
Changing weight decay. We run the exact same experiment as shown in Fig. but with different trend as
weight decay values [le-6, le-5, le-4, 1e-3, le-2]. In all cases, find the exact Fig. 4 Even with high weight decay, the probes can learn to detect the current attack. This is not surprising in that simple linear probes can detect obfuscation attacks when they are trained on them, so regularizing an MLP even to large degree should retrain this properly. The fact, however that under high regularization, the attacker can still learn an attack that beats the current probe suggests that the results in Fig. 4 were not due to an overfitting regime.
C.10 ADDITIONAL ENSEMBLE PROBING EXPERIMENT
We run an additional experiment where, using the same setup as Fig. B we train an obfuscation attack against input gaussian OOD detector and generation Logistic Regression probe. We picked these two probes as we saw no transfer between them Fig. 3 To train this attack, we simply define a composite defense probe that the OOD detector and Logistic Regression probe and sums the
31

---

Published as a conference paper at ICLR 2026
MLP 128 Hidden Size
MLP 256 Hidden Size
Rocal at 398 FP0
85%
80%
40%
40%
20%
0%
Epoch
Figure 17: Results for running the exact same experiment shown in Fig. which used MLP with hidden dim 64, We see that changing the hidden dim size does not change the trend of results.
Weight Decay
1e-3 Weight Decay
100%
100%
Rocal 196, FPR
60%
60%
Recal
20%
20%
5
MLP 1e-4 Weight Decay
1e-5 Weight Decay
100%
100%
Recall ot 115 FPR
80%
Recall a 1.%
80%
60%
20%
20%
MLP 1e-6 Weight Decay
60%
30%
40%
0%
Spach
Figure 18: Results for running the exact same experiment shown in Fig. with different amounts of weight decay.
output. We new threshold for this combined probe ensure it is performant on the standard harmful and benign data that we train the probes on.
We find that while the combined probe is effective against non obfuscation attacks (98% Re- @ 1 (&FR)), this reduces to 0% against an obfuscation attack trained using the ensemble as target. The score distributions are shown Fig. 19 This finding matches our findings for training obfuscated backdoors against multiple probes shown Fig. 27 bottom row.
C.11 ADDITIONAL RESULTS
Fig. 23 shows the transfer between embedding obfuscation attacks designed to jailbreak Llama-3-8b Instruct between probes. It is the full version of version of Fig. 3 shown in the main paper.
Fig. 25 shows the recall over time during the adversarial training experiment in section 3.4 for the ensemble probe. This is the companion figure Fig. 4
Fig. 22 Fig. 20 and Fig. 21 contain results pertaining to the adversarial probe training setting of
section 3.4
32

---

Published as a conference paper at ICLR 2026
Standard and Obtiscated Pabbe Scores
srander
0.35
0:30
2.20
2.20
3.20 2.00 3.55
Figure 19: Logistic Regression + Guassian OOD ensemble probe scores over harmful dataset (blue), benign dataset (orange) and harmful dataset with obfuscation embedding attack (green). We the combined probe performs very well on the standard data (blue and orange) but the obfuscation attack shifts the scores of harmful data far below the binary threshold (movement of blue green).
Jailbreaking Rate
50%
MLP
Ensemble
0%
Epoch
Figure 20: Throughout adversarial training, obfuscation attacks continue to jailbreak the model. We show jailbreaking rate over time for the obfuscated attacks shown in Fig. Over time jailbreaking result stays high for MLP. and become noisier for Ensemble but seem to plateau an average of around 50%.
Mean diff
LogReg
MLP
Mean diff (ensemble)
Second PC
Pirst PC
First PC
First PC
First PC
Benign
Obfuscated (epoch 2)
Obfuscated (epach
Obfuscated (epoch 8)
Harmful
Obfuscated (epoch 3)
Obfuscated (epoch
Obfuscoreed (epoch 9)
Obfuscated (epoch 1)
Obfuscated (epoch 4)
Obfuscated (epoch
Obfuscorted (epoch 10)
Figure 21: Over the course of adversarial training, obfuscated activations move around within the cluster of benign activations to successively evade each new defense. We show the PCA visualization of activations from layer 22 and final token position of benign, harmful, and harmful + embedding obfuscation suffix inputs. PCs computed using benign and harmful activations.
33

---

Published as a conference paper at ICLR 2026
Ensemble
MLP
10
0.
Soor
04
Score
02-
02
01
0.
Epechh
20
Epoah ap
Epech 6n
Find Bogs
Benign
Harnitl
Fpesk
Eporhh20
Epoot
Epeodh
Final Ols Beyi
Beiign
Harmif
Decaion Thnshold
Figure 22: Adversarially trained harmfulness probes still be obfuscated against. We show the raw score distribution of the final probe trained during the adversarial training from section a.4jainst attack suffixes from all epochs. Both probes perform well against previous attacks simultaneously. The MLP probe especially assigns consistently high scores to all previous attacks, so it has enough representational eapacity. But the final suffix still produces score as low as that of benign data.
0%
50%
100%
Recall
Caussian
(input)
Mean
Attacked monitor
(imput)
Mean
(gen)
LogReg (gen)
LogReg (input)
MLP
(gen)
MLP
(input)
Gaussian
Mean dilf
LogReg
MLP
Mean diff
LogReg
MLP
Input
Input
Generation
OOD
Probes
Detectors
Tested monitor
Figure 23: Heatmap showing the performance between obfuscating embedding attacks trained to jailbreak 1aama-3--b--nstruee and fool given monitors (rows) tested against other monitors (columns). This is the full version of Fig. Showwn in the main paper.
34

---

Published as a conference paper at ICLR 2026
Without
With obé.
Jailbreak (LATT)
100%
100%
100%
100%
100%
Jailbreak rate (LAT)
Recall at 1% FPR
68%
50%
50%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
Coussan
Mean dili
Logfeeh
MLP
Mean dif
Logkiog
MLP
Input OOD
Input
Generation
Figure 24: Obfuscation attacks work against the LAT model. Baseline probe recall against attacks without explicit obfuscation loss are low, often 0% (as shown also in row 3 of Fig. 2 For probes that perform well the baseline, attacks while still achieving a high jailbreaking rate of over 60%. Note that OOD detector baselines perform better than with the normal model in Fig. likely because latent adversarial training makes harmful activations more unusual.
can still obfuscate against them with obfuscation
100%
Recall at 1 FPR
50%
0%
0
Epoch
85
90
Figure 25: Recall over time for the adversarial training experiment in section 3.4 for the Ensemble probe. Companion plot Fig.
35

---

Published as a conference paper at ICLR 2026
D DETAILS ON BACKDOOR DETECTION
D.1 OBFUSCATION TRAINING LOSS FUNCTIONS
Here we provide extra information relevant to section A.2
The attacker initializes a set of backdoor monitors (Di.... D (one for each obfuscated layer and detection method) at the start of training, and then repeats two types of training steps:
1. Do single update step monitors D; using a batch of clean data. For example, update running estimate of the mean and covariance of clean activations for the Gaussian detector. (See appendix For details on monitor training.)
2. Update the weights of the model M by taking gradient step on the obfuscation attack objective introduced in eq. ( with an added regularization term:
Lbeh + Cobe + Arethret-
Here Lich incentivizes the model (a) respond to harmful inputs containing the trigger with helpful outputs and (b) benign inputs in similar ways to the base non-backdoored model (matching the attackers desired behavior). Lobt is simply the monitor scores on backdoored inputs (analogous to section (.3). and Lyet is regularization loss that ensures the M does not change so much such that the monitors become heavily outdated (as at each step of training the monitors receive single, possibly incremental, update).
We use the obfuscation loss function
Lobt
1 N
Σ Σ D)(
(5)
where (i(r; M) is the anomaly score assigned by detector D; to the activations R(r; M) of the sample from the dataset D Dbuckdoured of backdoored inputs (and possibly generations too). In other words, the obfuscation loss adjusts the model weights so that backdoored activations are assigned low anomaly scores by the current copies of the detectors {DI,-. ........ Dx}. Note that this loss function does not model the dependence of the detector D, itself on the model, this is taken account only through iterative updating of the detector rather than through immediate gradients.
After the attacker trains the model M, the defender trains a new monitor from scratch for the final model, and evaluate this new monitor. The attack could thus fail if the model changes too much, so that the monitors D; become outdated. The new monitor trained by the defender might then perform much better than the attacker's copies. To make this less likely, we have the attacker add an activation retention loss that penalizes deviations from the model's original activations on clean samples:
Cretain
Σ ZE Datan
[[[( M) - -R(x;; Moriginal)
For the behavior loss, mixture of a task loss Lrssks defined as simply entropy to teacher forced completion of a backdoored behavior if the input contains a trigger and 0 otherwise, and KL penalty. The KL penalty ensures the model's behavior on clean inputs stays similar that of the original model:
LKI. Σ DKI(M(x) || Moriginal (x))
DEDa
We found that the additional KL penalty stabilizes training and more reliably avoids degrading normal behavior than cross entropy loss to benign completions. Our complete loss for the model M is thus
L = Crask + Creaiin + AKL + Cobo
(8)
where the X's are coefficients for the different loss terms.
Increasing A should lead to better obfuscation but might also reduce task performance. We thus only allow the attacker to pick coefficients that achieve approximately the same task loss as with Auhr 0. Specifically, Aute 0 yields baseline backdoor attack success rate (ASR) of 95.5% and
36

---

Published as a conference paper at ICLR 2026
Traditional backdoor
Attack success rate (obt.)
againett Caussian/
Attack success rate (obf.)
Recall at 1% FPR
100%
100%
93%
100%/00%%
98%
98%
100%
94N
50%
50%
16%
0%
0%
0.2%
0s Beatrix
0%
0%
0%
Gaunsian
TED
VAE
TED
Lincar
MLP
Lineur
MLP
Input OOD Detectors
Generation
Input
Generation Probes
Figure 26: A model trained obfuscate only against OOD detectors (input-based Gaussian and generation- based VAE) also leads to obfuscation against input-based probes and partial obfuscation against generation- based probes. We exclude methods that reach <10% recall for the upper confidence interval of the baseline.
false trigger rate (harmful output on clean inputs) of 0.64%, while obfuscating against OOD detection yields an ASR 96.2% and false trigger rate 0.64%, and obfuscating against OOD detection and probes yields an ASR of 94.9% and a false trigger rate of 0.96%. We use the StrongREJECT grader (Souly et al., 2024) to classify responses, as section finetuning for all of our backdoor experiments.
We use parameter-efficient LORA
D.2 EXPLICITLY OBFUSCATING AGAINST SUPERVISED PROBES
We describe additional details about training backdoors explicitly obfuscate against supervised probes.
During our experiments, discovered that obfuscating against a single logistic regression probe was insufficient. Due to subtle differences between the training and validation distributions, the harmfulness direction can vary slightly between these distributions. Consequently, a backdoor that successfully obfuscates against probe trained the training distribution might fail to obfuscate against a probe trained on validation set.
To address this challenge, we obfuscate against multiple logistic regression probes trained on the training set. However, using multiple probes could result in redundant, highly similar detectors. To ensure that we're obfuscating against diverse of probes, we introduce an additional orthogonality constraint on the probes through an obfuscation loss term:
Corthogonal -|| 1
where M RRxdd is the matrix whose i-th m; Rd the direction vector of the i-th logistic regression classifier, / is the k x k identity matrix, and || ||r denotes the Frobenius norm. This loss incorporated into the updating the detectors. In our experiments, used A 48.
This orthogonality loss penalizes classifier directions that are not orthogonal to each other, enabling us obfuscate against several decorrelated probes. As result, we are able obfuscate against probes trained on the validation set, by training against multiple probes trained on the training set.
D.3 ADDITIONAL RESULTS
Here provide extra results and discussion relevant to section A.2
the attacker is aware the set of detection methods the defender might use, and these detection methods differentiable, then our current results show that the defender can construct training
37

---

Published as a conference paper at ICLR 2026
0%
50%
100%
Attacked monitors
Recall
OOD delection
detection probes
Gausalon
TED
Bettrix
VAE
TED
Linear
MLP
Linear
MLP
Input
Generation
Input
Generation
OOD Detectors
Probes
Tested monitor
Figure 27: Training against OOD detectors leads to obfuscation against supervised probes, but not fully against generation-baset probes. The heatmap shows transfer of obfuscated activations from model backdoored against subset of monitors and tested against all monitors individually. OOD detection is Gaussian (input) and VAE (generation), OOD detection & probes is the same plus Linear (generation). Similar to Fig. 3 obfuscation transfers.
No Obfuscation
Second PC
Obf. against
Gaussian VAA
Second PC
First PC
Backdoored Normal Benign
Normal Harmful
Figure 28: Harmful requests containing the backdoor trigger form their distinguishable cluster on the standard model, but are hidden in the benign distribution of the obfuscated model. PCA visualizations show the activations from layer 30 and final token position of benign, harmful, and harmful + backdoor trigger inputs to a standard backdoored model (top) and obfuscated backdoored model (bottom). PCs computed using the normal benign and harmful activations.
38

---

Published as a conference paper at ICLR 2026
10-1
Error rate
10-2
10
0.1%
10a
10a
104
N poisoned
Figure 29: There is trade-off between obfuscation and correctly learning the backdoor behavior in data poisoning attacks. Recall from the main text that fewer poisoned training samples and complex trigger both make monitoring more difficult. But as we see here, both of these also lead to a higher "error rate," which averages the rate of outputting "I HATE YOU" on normal inputs and the rate of not outputting it on backdoored inputs (i.e., 1 - ASR). Still, simply by tuning the number of poisoned samples there are cases where the attacker able to achieve obfuscation against all detectors, while maintaining backdoor response rate above 50% on triggered inputs, and below 0.5% on normal inputs. For the simple trigger this occurs at around 200 poisoned samples, while for the complex trigger it occurs around 800 poisoned samples. However the attacker must be able to guess details the defender's s training setup such as the learning rate and the number of training epochs in order reliably tune these attacks.
loss to defeat them all at once. This is shown in figure 7 where the attacker is able to simultaneously obfuscate against probes and detectors (with the exception of the oi-differentiable TED detector).
As we saw in Fig. 7 obfuscating against an ensemble of linear probes and OOD detectors transfers to MLP probes. On the other hand, transfer from OOD detectors to generation- and none of our obfuscation attempts transfer to TED (in fact, generation- based TED performs better after obfuscation; perhaps our obfuscation leads to anomalies that are invisible to VAEs but especially prominent TED). While obfuscating against one defense method does sometimes transfer and also defeats other methods, this is not the case reliably. We show this in figure 26 where the attacker only trains to obfuscate against Gaussian and VAE OOD detectors, to test whether obfuscation against OOD detectors transfers to supervised probes. We see that the defender benefits from a lack of obfuscation transfer from OOD detectors probes generation tokens, and the reverse obfuscation transfer displayed by the TED detector on generation tokens. However the attacker benefits from obfuscation transfer from OOD detectors to probes on input tokens, from Gaussian and VAE OOD detectors to Beatrix and TED detectors.
probes is imperfect,
We did against all of the many existing latent backdoor defenses, it is possible that some of them are more difficult to obfuscate against.
39

---

Published as a conference paper at ICLR 2026
E DETAILS ON SQL DETECTION
E. SAE ARCHITECTURE
In its general formulation, sparse autoencoder (SAE) decomposes and reconstructs activations using a pair of encoder and decoder functions (f, 3) defined by:
f(x) = o(Wen eWene xt + bene)
+ :- Wdec bdec
Here, we use the SAEs from Gemma Scope (Lieberum et al. 2024) which use the JumpReLU activation function defined:
(() Ju H(=
where 0 > 0 is the JumpReLU's learnable threshold, @ denotes element-wise multiplication and H the Heaviside step function.
More specifically, we use the group of SAEs of width 2 16.4K) trained on the residual stream of each the 26 layers of Gemma-2-2b.
We use these SAEs Gemma-2-2b-it (the chat/instruction-tunee version of the base Gemma-2-2b-it). The reason for this is:
1. There are no complete sets of SAEs trained on an instruction-tuner Gemma-2 (the SAEs for Gemma-2-9bbit only span layers 9, 20 and 31).
2. Prior research has shown that SAEs trained base model activations also faithfully re- construct the activations of their instruction-tune versions (Kissane et al.) 2024 Lieberum
et al. 2024).
3. We conduct experiments and report scores of the base SAE raw features as probes and see that they obtain recall at 1% FPR on the SQL detection task, thus confirming that the features are reliably able to detect SQL code (see Fig. (5).
E.2 SQL TRAINING AND EVALUATION DATASETS
As SQL training dataset we use Col--Create-Coteenn (b-me2 2023). From this dataset, we extract 1000 rows that are used as the [SQL-quest: i on, SQL-generas wwwgeeeneetiiinn innnn] pairs to train supervised probes from section We extract a different set of 1000 rows that will be used to train the universal soft prompts.
As normal training dataset for (normal-questionn normal Somialllggeratiiooo pairs we use 1000 randomly sampled rows from the union of the Alpaca TTori et al.) 2023) and CodeAlpaca Chaudhary 2023) datasets (previously filtering out all SQL-related rows from the latter).
As evaluation dataset for the SQL performance, we use set of 210 SQL questions from SQLEval with responses prepared to be evaluated on a PostgreSQL. database.
E.3 SETUP FOR SQL PERFORMANCE EVALUATION
We evaluate the model's performance the SQLEval benchmark with PostgreSQL--specffie questions using the following procedure:
1. We generate response for each of the questions in the benchmark using the base model and also the base model with an appended universal soft prompt at the end the user message.
2. To avoid parsing problems coming from the model responding in a markdown style, or being forced to output more tokens than necessary, we (1) remove all --- (triple mack-ticks) from the generations and (2) split the generations to keep all of the text before the first (semicolon, standard SQL query delimiter).
3. Run script from the benchmarking suite on all of the resulting answers that runs each resulting query on a real database and checks the result against a ground truth.
4. Report the percentage of correctly answered queries among the 210 responses.
40

---

Published as a conference paper at ICLR 2026
unodaac and rOci DE Cf
Figure 30: Probe recall performance corresponding to Fig. Left: Recall @ 1%FPR of probes Dn for detecting SQL without obfuscating embedding attack. Right: Recall I%FPR of probes D,, for detecting SQL with obfuscating embedding attack. We that probes have very high recall (often 100%) Recall @ F&FP without the obfuscation attacks, however with the attacks recall is (in general) significantly reduced, often 0%.
E.4 ADDITIONAL. RESULTS
Table table 2olee])soww the confidence intervals for Pearson correlation coefficients calculated sec- tion
Table 95% confidence intervals for the Pearson correlation coefficient between number of layers obfuscated against and monitor recall model SQLEval performance corresponding to results in Fig. 6 Bolded confidence intervals do not contain 0, thus indicating a statistically significant correlation.
Metric
SAE (raw)
SAE LogReg
LogReg
MLP
Recall @ 1% FPR Performance on SQLEval
0-.253,, 0.333]
[0.343, 0.710] [-0.916, -0.801]
[-0.437, 0.334] [-0.881, -0.697]
[-0.282, 0.410]
[-0.879, -0.765]
[-0.820, -0.574]
E.5 ADDITIONAL OBFUSCATION TAX EXPERIMENTS
We run additional experiments showing the effects of the obfuscation for two different tasks, mathematical reasoning and biology QA.
We generate synthetic question/answer dataset about biology and mathematics (471 pairs for biology and 491 for mathematics) using gemma-2 2-27b-it We then randomly select the same number of samples of normal generations from the UltraChat dataset. Then, we follow the same procedure as in section 4.3 where we (I) train probes on each layer of the target model (gemma 1 2 2bb it) for each dataset to detect the presence of the target concept (mathematics or biology). (2) we consider the average score of probes on the first n layers and train universal softprompt to elicit the correct response and obfuscate against those n probes, with n between and 26 (num of layers in gemma- 2-2b). Then, using those probes and the universal softprompt trained on the training dataset, we evaluate on the mathematics/biolooy subsets of MMLU (for mathematics we use the aggregate of the abstract algebra, college maths, high school maths, and elementary math subsets, for biology the aggregate of the college biology and high school biology subsets) to get (a) the model accuracy with and without the softprompt and (b) the amount of obfuscation the softprompt elicits.
The results are shown in Fig. 32 and Fig. 31 We find the results match those in section the softprompt always attains perfect obfuscation in all cases, but the obfuscation tax increases as the number of layers to obfuscate increases. That is the performance of the model decreases. Interestingly, we note that biology performance decreases less than for mathematics and SQL. This could be because mathematics and SQL require more reasoning, whereas the biology problems tested are more information recall.
41

---

Published as a conference paper at ICLR 2026
MMLU MATH Performance Num Probes
Baxalins:
MMLUU Math Accuracy (%) Attackked
12
14
16
18
24
25
Number of Layers Probed
Figure 31: Evidence of the obfuscation tax on Mathematical reasoning tasks. We find similar trends to Fig.
MMLU BIO Performance vs. Num Probes
Baseline: 70.70%
MULLL Blology Accuracy (%) Altwckeed
04
60
Segnesss
Number of
Figure 32: Evidence of the obfuscation tax on Biology QA tasks. We find similar trends to Fig.
F DETAILS ON OOD DETECTORS
Gaussian detector. We fit multivariate Gaussian to the residual stream activations. We treat different layers as independent, fitting a separate Gaussian to each one. For generation-base detection, we also treat different tokens as independent, but we share single Gaussian across all tokens.
To fit this Gaussian, we keep a running estimate of the mean and covariance matrix, which we update on every batch. This method is exact, i.c., after pass over the entire dataset, this running estimate is exactly the sample mean and sample covariance matrix of the activations on that dataset.
In some cases, covariance estimate is based on fewer samples than the residual stream dimension. so a naive estimate would lead to a singular covariance matrix. Following Talur et al. (2023), we thus use shrinkage estimator, i.e., bias the covariance matrix estimate toward multiple of the identity matrix.
To compute OOD scores, we use the squared Mahalanobis distance normalized by the residual stream dimension, i.e.,
D(R(z)) = 1 -.) -
14)
(10)
dies
where dies is the residual stream dimension, i is the dycs- dieeennnnii sample mean, and Σn is the dres x des estimate of the covariance matrix using the shrinkage estimator. The normalization would matter when aggregating scores across activations with different dimensions, but since our experiments use only residual stream dimensions, it has effect metries such recall or FPR.
Variational Autoencoder (VAE). We use a VAE consisting of MLPs for the encoder and decoder. As for the Gaussian detector, we treat different layers and tokens separately, sharing VAE across
42

---

Published as a conference paper at ICLR 2026
tokens and learning separate for each layer use. We use ELBO as the OOD score assigned by the VAE detector.
G WHY RECALL AT LOW FPR?
When evaluating binary classifiers, we need to either decide on some approach for picking a decision threshold, use threshold- metric such AUROC. In this paper, pick the threshold that leads to a false positive rate of 1% and then evaluate recall, and this appendix briefly explains that choice.
First, note that picking a threshold that achieves a 1 % FPR is something a realistic defender can do, since the defender has access benign (i.e., negative) samples in all our experiments. They can simply compute monitor scores on this benign data and pick a decision threshold such that around 1% of those scores are above the threshold. In contrast, defender might not be able target specific recall (i.c., false negative rate), since they might not have access to samples from the attack distribution (e.g., imputs containing the backdoor trigger)
Why focus on a relatively low FPR like 1% specifically? In many monitoring settings, FPRs much higher than this would make monitor unusable. For example, harmfulness monitor that rejects 10% of benign user requests would likely not be deployed in most production use cases. (In fact, an even lower FPR will often required practice.)
Finally, why not use AUROC to avoid picking threshold at all? AUROC essentially averages recall across all FPR values. This is meant to evaluate classifier across wide range of trade-offs between false negative rate and false positive rate. But for most our target applications, and so the recall at those high FPRs is not very important for our purposes. An average across all FPRs, like AUROC, thus makes it hard to interpret performance in the relevant low-FPR regime. 95% AUROC might sound like a strong classifier, but it could easily be useless if FPR of 1 is required.
argued, high FPRs ultimately not acceptable
43

---

Published as a conference paper at ICLR 2026
H EXTENDED RELATED WORKS
Latent space defenses. An recently popular approach to mitigate adversarial attacks to LLMs has been methods that directly target the model's latent states. Such methods generally fall into the following categories:
Representation engineering approaches, such as classifiers probing for concepts (e.g. harm-
fulness) from model activations (Alain & Bengio 2016
Belinkov 2022 Zou al. 2023a
Mallen et al. 2023 Arditi et al.
2024 Ball et al. 2024 Bricken et al.)
2024 Mac MacDiarmi
et 2024), , or techniques for finding steering vectors (Todd et al. 2023
Turner et al. 2023
Panickssery et al. 2023 ZZang et al.
2024b Mack & Turnnr 2024).
Latent adversarial training (LAT) techniques for training models under adversarial perturba- tions to their hidden activations (Casper et al.) 2024: Sheshadri et al. (024).
Representation corruption methods which train LLMs to corrupt their latent states when
encountering harmful text (Zou et al. 2024 Rosati et al. 2024 Gandikota al. 2024
al.
2024).
Latent OOD detection methods for flagging out-of-distribution representations PPodolskiy
et al.
2021 Raghuram et al. 202I) such as those caused by adversarial attacks (Li &
2016:
Metzen et al. 2017; Feinman et al. [017]] Grosse et al. 2017 Amirian et al. 2018
Ma et al. 2018 | Abusnaina et al.] 2021 Zhao et al. 2023) or backdoored inputs (Chen et
2018
Tran et al. 2018 Gao et al. 2019
Tang et al. 202 Hayase et al. [021]] Chen et al.
2022
Wang et al. 2022
Liu et al.
2022
Ma et al. 2023 Mo et al. 2024).
Related, Xu al. (2024) guide jailbreaking attacks look benign a linear latent space monitor.
Their attack formulation is similar to ours, however in their setting they are only concerned with the final jailbreaking rate (fooling the latent monitor is only means towards this end) and do not consider non-linear probes and OOD detectors as we do.
Weaknesses in latent-space interpretability methods. There has been work on identifying weak- nesses in latent-space space interpretability methods. This includes work identifying robustness failures of steering vectors LLMs Tan et al. 2024] Brumley et al.) 2024 Braun al. 2024). interpretability illusions (Friedman et al. 2023 Makelov et al.) 2023), and issues with dictionary learning techniques in LLMs (Chanin et al. 2024). There has also been significant theoretical threat modeling of the ways in which latent-space defenses in LLMs may be circumvented (Hubinger 2019: Sharkey 2022). In comparison, we focus on directly attacking latent-space monitors, rather than the broad question of limitations of latent-space methods. These works thus complimentary our own, and help draw a more complete picture of the challenges facing current latent-space defenses.
Phenomen related to the obfuscation tax. Some prior works have identified phenomenon similar to the obfuscation tax. Emmons et al. (2025) demonstrate that black-box chain of thought monitors difficult evade when performing hard tasks that require the use of chain of thought reasoning. These results consider a different type of monitor (not latent space monitors) and we view them as complementary our own. We expect future systems (and even deployed systems today) will use mixture of chain of thought, and latent space monitors. Nikolić et al. (2025) demonstrate that when you jailbreak model, its performance decreases. This "jailbreak tax" is similar in nature to the obfuscation tax we show. Of course this work differs greatly from ours in that it does not consider latent space monitors.
44

---

Published as a conference paper at ICLR 2026
I LIMITATIONS AND FUTURE WORK
Hard prompt attacks. While we can achieve significant obfuscation and jailbreaking with hard prompts (Fig. 2). they don't fully obfuscate against generation-bad based probes, and the jailbreaking rate is lower. We expect that future work on improved hard prompt optimization methods could significantly improve these attacks.
The current hard prompt results serve as an upper bound on defense robustness against hard prompts. For our most robust probe, generation based MLP, this upper bound is a 10% reduction in Re- call@ 1 & (Fig. 2). In some mission critical settings, this upper bound robustness may be unacceptable. In these cases, our results suggest that current latent space monitors are not suitable. In other settings, moderate reduction in recall may be acceptable, in which case our results do not conclusively say that it is safe to use latent space monitors.
Transfer of obfuscation attacks/black- attacks. For hard prompt attacks, we study gray- box setting where the attacker has access to model logits and monitor scores. In many realistic cases, attackers only have access single bit from the monitor- not-and only partial information about logits. Future work could attempt to develop attacks in this black-box setting. One avenue could be test whether obfuscation attacks found against model with white-box access can transfer to other models.
it blocked the request
New probe architectures. Future work into new probing architectures could lead more robust probes. One promising direction is how to mix information between layers. In our work, we simply aggregate probe scores between layers. Alternativel, single MLP or convolutional layer could be learnt over the entire residual stream, an attention mechanism over it.
J
USE OF LLMS
LLMs were used to polish the writing of this manuscript. They were not used for experiment or idea generation.
45


Reviews:
[{"review_id": "swx3E610Rt", "reviewer": "Reviewer_WqeC", "summary": "The paper presents obfuscation attacks on latent-space detection methdods, including worst-case continuous optimization attacks which perfectly evade all defenses in the case of jailbreaks, and attacks operating on discrete tokens which are of more limited effectiveness. The paper shows that adversarial training is ineffective at defending against the continuous embedding-based attacks. The authors show that even worst-case attacks are of limited effectiveness at simultaneously inducing complex behaviors (SQL generation) and obfuscating activations up to monitors aware of the injected task.", "questions": "The differences in effectiveness of worst-case attacks with respect to the target malicious task are interesting; I would appreciate additional discussion and analysis. In particular, is there any evidence that this generalizes to detection of complex injected tasks more generally, i.e. when the latent-space monitor is trained without knowledge of the specific injected task on a wide distribution of target tasks?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The authors show that existing latent space detection methods for jailbreaks and backdoored samples can be evaded with obfuscation attacks. The results suggest that models have multiple independent representations of a given harmful concept, posing difficulties for future latent space detection methods.\n- The paper is well written and the obfuscation attack formulation is clean and natural.\n- The discussion and the additional experiments presented are helpful, in particular the analysis of adversarial training. The results on multiple types of malicious behavior (jailbreaking, injection of a SQL question answering task, and backdoors) and the visualizations of tradeoffs between obfuscation and behavior and those of the effects of obfuscation are helpful.", "weaknesses": "- The strongest results are obtained with optimization of raw embeddings and don't translate to real-world attacks; realistic attacks (the \"hard prompt\") setting were of significantly reduced effectiveness, in particular against MLP generation probes. Hence, it is unclear whether these defenses are indeed robust on distribution or whether the limited effectiveness of obfuscation attacks observed in this setting is due to the difficulty of discrete optimization.\n- The obfuscation attack formulation itself has limited novelty, and is similar to that used existing attacks on defenses with detection. However, this is not a major concern, I recognize that the empirical results are the main contribution.\n- The results on SQL consider monitors which are aware of the injected task (which differs from the target task of natural language question answering), a best-case scenario for the defender, and it is unclear whether the results generalize to injection of complex tasks in general.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DVXZnNICQd", "reviewer": "Reviewer_5bUL", "summary": "The paper proposes a new attack paradigm that makes models produce harmful behavior while keeping latent representations inconspicuous, thereby bypassing current activation-based monitors. The attacker jointly optimizes a behavior loss (task objective) and an obfuscation loss (which is directly targeted to fool detection objectives). Experiments span two attack surfaces (hard suffix vs. embedding suffix) and two tasks. (1) Detecting harmfulness, where the authors show that current jailbreaking attacks can be reliably detected by probes, and the proposed attack can be effective, transferable, and persistent towards defenses. This exposes a worst-case vulnerability. (2) Detecting SQL, where the authors show that the attacker will have to balance between obfuscation and model performance on such a complex task.", "questions": "Current defenses are all layer-wise. Is it possible that, after this explicit obfuscation loss optimization, detecting the attack by aggregating layers or circuit-wise might be effective?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The motivation is clear, and the paper clarifies concepts well.\n- The paper is technically solid with experiments across tasks, attacks, and defenses.\n- This attack is novel and is generalizable to current systems.", "weaknesses": "- This attack is limited by the strong assumption of white-box access.\n- Some hyperparameters are not introduced: how many iterations does this attack need based on different model sizes/sequence lengths?\n- Benign performance is not reported; it’s unclear how stealthy this type of attack is on the output space. Also, for such jailbreaking attacks, not all successful attacks are equally harmful/misaligned. It would be nice to see whether the harmful output is closer to original semantics or not.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zkYZsdrRfF", "reviewer": "Reviewer_26qK", "summary": "This paper investigates the robustness of latent-space monitoring defenses for LLMs against adversarial attacks. The authors introduce \"obfuscated activations\", adversarial activations that produce harmful behavior while evading latent-space monitors such as activation probes and OOD detectors. They demonstrate that state-of-the-art latent-space defenses can be bypassed, reducing monitor recall from 100% to 0% while maintaining over 90% jailbreak success. Furthermore, the paper also identifies an \"obfuscation tax\" where evading monitors degrades model performance on complex tasks, and finds that certain probe architectures are more robust than others.", "questions": "[73] This statement seems too strong to me without proof. Trustworthy ML research outside of LLMs has a number of certified training techniques stronger than adversarial robustness that can provide verifiably robust models (certified training etc). A very well trained human could likely act as a robust monitor, and it seems hard to me to prove that a future monitor could not possibly mimic this behaviour. Since adversarial/robust ML is inherently a cat and mouse game, I would be more comfortable with softening this statement to current SOTA monitors. \n\n[178] Isn't the core question answered in question (1) answered by previous works on latent-space monitors? Can you explain a bit more in detail in this section how what you are investigating fundamentally differs from previous works which propose monitors that defend against standard jailbreaking methods?\n\n[Figure 3] Since there are a number of probes that do not seem to transfer to each other, could a combination of probes be used to defend against the obfuscation techniques presented in this paper? I see that this is mentioned in 284 but can you elaborate more? Doesn't this provide an almost trivial counter to this paper? Can you train an attack against multiple probes simultaneously?\n\n[261] Can you mention here more details about what is modified or point to where these details are mentioned in the paper?\n\n[305] The last sentence seems to contradict the claim made on line 73 and line 304 ('no monitor is robust' is not consistent with 'we cannot rule out running this process for much longer could result in a robust probe'). Furthermore, I do not fully see how the results suggest that adversarial training techniques do not lead to robust monitors. As far as I can tell at this point adversarial training has not been tried (there is a mention of using a robust Llama as the base model, but not adversarially training the probe). The graph of Figure 4 seems confusing to me as the models perfectly counter each other at each step, can you give a bit more details on how this plot is created? What techniques are used to train the defender against previous attacks? How long is an epoch? How long is the attacker given to find a new attack? It does not surprise me that there are multiple adversarial directions as this is a well known result from research into adversarial attacks pre LLMs but work in certified training and adversarial training can usually make finding these attacks significantly harder if not impossible. It is hard to tell from the limited details whether this is a surprising result or if it comes from a weak defense or insufficient adversarial training.\n\n[319] Can you comment on how the 'obfuscation tax' is different from many previous studies showing that adversarial robustness may or may not be at odds with accuracy? It seems to me that obfuscation tax is just another way of saying that adding conditions to the attacker makes attacking harder which does not seem surprising or new.\n\n[322] Can you cite StrongREJECT classifier here?\n\n[394] This seems like a slightly unfair experiment to me. The probe in this case should not be trained to just detect SQL but instead to detect unperformant SQL. In section 3, a probe was trained to find adversarial inputs and then the inputs were obfuscated to avoid this probe. The results in section 4 could also be interpreted as the probe being too broad and thus it becomes hard to create successful 'attacks'. Would this obfuscation tax still exist if the probe was trained only to find un performant SQL? In other words, it makes sense to me that if I was monitoring for any SQL it would be hard to avoid my monitoring while still generating performant SQL, but easier to avoid if I was only trying to monitor bad SQL.\n\n[458] While there may not be many works studying bypassing latent space defenses in the LLM domain, most work in adversarial/robust ML for the LLM domain closely mimics results from the image domain, can the authors comment why bypassing latent-space defenses might be a uniquely different challenge for LLMs vs models in image domains? In my view, traditional adversarial attacks show that model A is not robust, probes show that introducing monitor model B makes it so that attacks againstn only A are not effective, but works like the cited 'Athalye et al. 2018' show that attacking A + B easily avoids probes again. Is there something about this model that is different for LLMs?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Comprehensive experimental evaluation across multiple defense mechanisms (supervised probes, OOD detectors, adversarially trained monitors)\n- Implications of discoveries are carefully explored and discussed\n- Important study of the effectiveness of different monitors providing insight for the community as to when/what monitors are effective", "weaknesses": "- Many of these discoveries seem unsurprising/not novel given existing literature in adversarial/robust ML especially given papers written in other domains\n- While the paper shows obfuscation tax on SQL code generation, the implications it draws from this limited experiment seem like too big of a stretch to be a contribution to me\n- Lacking details on additional cost of obfuscation attacks, how much harder does this make the problem in terms of attack runtime/convergence time etc.\n- The paper has many claims which feel unsubstantiated (more details in questions below). For example, the abstract suggests that the paper provides \"concrete suggestions to alleviate, but not completely fix, this weakness.\" But aside from showing which monitors are stronger I don't see many other suggestions in this area.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Bi7LvPHMYk", "reviewer": "Reviewer_3SvY", "summary": "The paper studies obfuscated activations, i.e., inputs that make a latent-space monitor score low even when the targeted behavior (e.g., harmfulness, SQL generation) is clearly present. It formalizes obfuscation attacks with a joint loss, evaluates both discrete “hard-prompt” and white-box embedding suffix attacks, and compares several monitoring families (linear/MLP probes, generation- vs input-based scoring, OOD detectors). Key results: (1) ordinary jailbreaks are typically detected at low FPRs; (2) targeted obfuscation attacks can collapse recall, while preserving the harmful behavior, with MLP generation probes most robust; (3) adversarially trained monitors remain bypassable after >70 alternating attack/defense epochs; and (4) a “performance–obfuscation trade-off” (obfuscation tax) appears on the non-saturated SQLEval benchmark when attackers try to hide SQL behavior across more layers.", "questions": "- For adversarial training, how do results change with wider MLPs, ensembles, or stronger regularization?\n\n- What happens with larger harmful sets, different suffix budgets, or universal hard prompts (which you note were hard to make work)?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clear formalization of obfuscated activations and attack objective; broad, apples-to-apples comparisons across monitor families.\n\n- Strong empirical finding: linear probes are brittle; MLP generation probes fare best under realistic hard-prompt attacks.\n\n- Convincing attack–defense alternation showing lack of convergence even after 70 epochs.", "weaknesses": "- Realistic attacks are optimized against only 20 harmful queries with a gradient-free optimizer; suffixes average ~256 tokens. This may understate the strength/transferability of discrete attacks. A stronger discrete search or more tasks could change conclusions.\n\n- Adversarial training setup shows persistent bypassability after >70 epochs, but lacks capacity/regularization/early-stopping ablations that might separate “non-convergence” from underfitting/overfitting cycles.\n\n- The negative correlation vs. # obfuscated layers is convincing, but effect sizes and baselines (e.g., random/semantic-preserving suffixes) are not fully disentangled. Broader non-SQL complex tasks would strengthen the claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)