Any-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depth

## Abstract

Abstract Large Language Models (LLMs) exhibit strong but shallow alignment: they directly refuse harmful queries when a refusal is expected at the very start of an assistant turn, yet this protection collapses once a harmful continuation is underway (either through the adversarial attacks or via harmful assistant-prefill attacks). This raises a fundamental question: Can the innate shallow alignment in LLMs be unlocked to ensure safety at arbitrary generation depths? To achieve this goal, we propose Any-Depth Alignment ( ADA ), an effective inference-time defense with negligible overhead. ADA is built based on our observation that alignment is concentrated in the assistant header tokens through repeated use in shallow-refusal training, and these tokens possess the model’s strong alignment priors. By reintroducing these tokens mid-stream, ADA induces the model to reassess harmfulness and recover refusals at any point in generation . Across diverse open-source model families (Llama, Gemma, Mistral, Qwen, DeepSeek, and gpt-oss), ADA achieves robust safety performance without requiring any changes to the base model’s parameters . It secures a near-100% refusal rate against challenging adversarial prefill attacks ranging from dozens to thousands of tokens. Furthermore, ADA reduces the average success rate of prominent adversarial prompt attacks (such as GCG, AutoDAN, PAIR, and TAP) to below 3%. This is all accomplished while preserving utility on benign tasks
with minimal over-refusal. ADA maintains this resilience even after the base model undergoes subsequent instruction tuning (benign or adversarial).

## 1 Introduction

Large language models (LLMs) are rapidly evolving from research prototypes [49, 13, 37, 5] into powerful agents capable of tackling complex real-world problems [25, 46]. This leap in capability presents a critical safety challenge due to their dual-use nature: the same advanced reasoning [50, 16, 2, 30] that enables an LLM to write secure code can also be repurposed to discover and weaponize software vulnerabilities. Despite significant alignment efforts, safety mechanisms remain brittle and are systematically bypassed by diverse attacks, including adversarial prompts [27, 7], prefill attacks [1], and supervised fine-tuning (SFT) attacks [34, 4]. Building truly robust systems requires first diagnosing the fundamental vulnerabilities in current alignment.

Figure: Figure 1: Refusal rates on AdvBench under harmful assistant-prefill attacks. Base models (dashed lines) exhibit a catastrophic drop in safety as the prefill depth increases. In contrast, applying our *Any-Depth Alignment* (ADA) method (solid lines) restores robust, near-100% refusal rates across all tested depths.
Refer to caption: x1.png

Current alignment strategies are fundamentally brittle. Most aligned LLMs rely on so-called shallow alignment, which trains models to emit a direct refusal (e.g., “I can’t help with that.”) when presented with a harmful query (e.g., “How to build a bomb at home?”). While this front-loaded safety is effective against direct harmful queries, its vulnerability to adaptive adversarial attacks [28, 7, 55] and shallow prefills [1] is well documented. As [Figure˜1](https://arxiv.org/html/2510.18081v1#S1.F1) confirms, a simple 25-token prefill on AdvBench [58] causes refusal rates to collapse from $\sim$100% to below 10%, including recent models such as gpt-oss [31].

One countermeasure is deep alignment [35], which trains models to refuse mid-stream. While this improves robustness at shallow depths, its protection fails to generalize and it still suffers from adversarial prompt attacks ([Section˜5](https://arxiv.org/html/2510.18081v1#S5)).
As our first key contribution, we systematically test this vulnerability via deep prefill attacks: harmful assistant-prefills ranging from tens to thousands of tokens ([Section˜4](https://arxiv.org/html/2510.18081v1#S4)).
Our analysis shows that deep alignment merely pushes the failure point deeper, creating an arms race between the attack depth and the alignment depth. For example, in [Figure˜1](https://arxiv.org/html/2510.18081v1#S1.F1) even a strong deeply-aligned model like Claude Sonnet 4 [2] falls below 25% refusal under a 100-token prefill.
Finally, while dedicated guardrail models [23, 54, 32] can be quite strong, their latency means the flagging occurs after full generation, so harmful content may already be delivered to the client before it is blocked.

From Rethinking to Any-Depth Alignment. Aligned chat models inherently know when their continuation is harmful, even under adversarial attacks. Simple self-reflection prompts (e.g., “Is your previous response harmful?”) often elicit an admission [33], showing that strong safety signals exist but remain locked within the decoding trajectory.
Tokens in the chat template, most notably the assistant header, can surface this latent safety assessment when injected mid-stream.
We call such tokens *Safety Tokens*, since they expose the model’s internal safety judgment.
Injecting them abruptly triggers the model to rethink its trajectory and refuse ([Figure˜2](https://arxiv.org/html/2510.18081v1#S1.F2)), reactivating its inherent alignment at any depth.
We operationalize this mechanism as *Any-Depth Alignment–Rethinking* (ADA (RK)), a training-free, inference-time intervention.
The stronger the base model’s alignment, the more reliably ADA (RK) unlocks it; for instance, ADA (RK) restores Claude Sonnet 4’s refusal rate to over 95% under deep prefill attacks, even with 500-token prefills ([Figure˜1](https://arxiv.org/html/2510.18081v1#S1.F1)).

Safety Tokens Unlock Innate Alignment. The observed “rethinking” behavior shows that signals of harmfulness are already encoded in the model’s hidden states during harmful generation, but under ordinary decoding they remain locked. Injecting Safety Tokens (e.g., the assistant header) acts as a key that unlocks this latent safety assessment, making it cleanly separable in the Safety-Token hidden states ([Section˜2](https://arxiv.org/html/2510.18081v1#S2)). These tokens function as aggregators, concentrating distributed evidence from the preceding context and surfacing the model’s safety judgment to trigger refusal. Building on this property, we introduce *ADA–Linear Probe* (ADA (LP)): a lightweight check that performs a single forward pass over Safety-Token hidden states and applies a simple linear classifier to halt harmful continuations. By leveraging the model’s own internal assessment, ADA (LP) achieves near-100% refusal under deep prefills across open-source models ([Figure˜1](https://arxiv.org/html/2510.18081v1#S1.F1)), with greater efficiency and lower memory cost than external guardrails; *the base model effectively serves as its own guardrail*, requiring no auxiliary models or weight updates. Our contributions are summarized as follows:

Figure: Figure 2: *Overview of the Any–Depth Alignment (ADA) mechanism.* (Top Left) Without ADA, a model that starts generating harmful content will typically continue to do so. (Top Right) ADA intervenes at a safety checkpoint by leveraging model’s own alignment. (a) *ADA-Rethinking (ADA (RK))* re-injects the header to trigger a refusal. (b) *ADA-Linear Probe (ADA (LP))* achieves the same outcome more effectively and efficiently by directly probing the strong safety signal present in the header’s hidden states with a linear classifier.
Refer to caption: alt_versions/andrew_with_without.png

- 1.
New Alignment Failure with Deep Prefills. We introduce deep prefill attacks ([Section˜4](https://arxiv.org/html/2510.18081v1#S4)) to test whether models learn a generalizable notion of harmfulness beyond a fixed depth. Current alignment strategies fail this test, with refusal rates collapsing even for strongly deep-aligned models like Claude Sonnet 4.
- 2.
“Rethinking” Generation (ADA (RK)). Re-injecting Safety Tokens mid-stream triggers a robust rethinking behavior that restores refusals.
This generative defense is training-free and performs on par with, and often better than, deep alignment and self-reflection baselines.
- 3.
Unlocking Deeper Innate Alignment (ADA (LP)). We trace the rethinking phenomenon to the Safety Tokens whose hidden states are highly separable for harmful content.
By leveraging this, ADA (LP) is: (a) *Effective*, achieving near-100% refusal against deep prefills ([Section˜4](https://arxiv.org/html/2510.18081v1#S4)) and reducing adversarial success from $>\!50\%$ to $<\!3\%$ ([Section˜5](https://arxiv.org/html/2510.18081v1#S5)); (b) *Precise*, with minimal over-refusal on benign tasks ([Section˜7](https://arxiv.org/html/2510.18081v1#S7)); and (c) *Robust*, maintaining performance even when the base model is fine-tuned ([Section˜6](https://arxiv.org/html/2510.18081v1#S6)).
- 4.
A General Phenomenon Across Diverse LLMs. The unlocking effect is ubiquitous: Safety Tokens related to the assistant header consistently expose a strong, linearly separable harmfulness signal across model families (Llama, Qwen, Mistral, Gemma, DeepSeek variants, gpt-oss), parameter scales, and core designs (dense, Mixture-of-Experts, and reasoning-centric).

## 2 Unlocking Innate Safety Alignment to Any-Depth Alignment

Before presenting Any-Depth Alignment (ADA), we define some useful notation and outline observations, which motivate the development of ADA.
Notation: Define *generation depth* $d$ as the number of assistant tokens generated after the user prompt; $d{=}0$ immediately follows the prompt.
*Safety Tokens* are tokens whose hidden states carry strong internal safety signals, by default assistant-header tokens, which expose the model’s own safety assessment, distinct from their normal templating role.
Unless noted, *hidden states* refer to representations taken after input layer normalization at a specified layer.

### 2.1 Innate Safety

The assistant header is the sequence of tokens bridging user prompts and the assistant’s response (e.g., 

assistant
\n\n for Llama-3.1).
Prior work has shown that aligned LLMs exhibit refusal signals at this header when the prompt itself is harmful [53, 56].
We uncover an even stronger and previously overlooked property: the safety signal embedded in the assistant header is not confined to the beginning of generation, but can be *re-triggered at any point during the generation process*.
In particular, re-injecting the assistant header mid-generation, even after a jailbreak has induced harmful content, reactivates the model’s refusal behavior ([Figure˜2](https://arxiv.org/html/2510.18081v1#S1.F2)).
This demonstrates that the model’s innate safety understanding, anchored to the assistant header tokens, and persists throughout the generation process.
As a result, harmful generations can be halted at arbitrary depth by streaming-time header insertion, effectively extending alignment from the prompt stage to any depth. These findings give rise to four natural questions:

- Q1:
*Why re-injection works:* Why does re-injecting the assistant header mid-generation trigger stronger refusals than relying on prompt-only checks?
- Q2:
*Probe on assistant header:* Why probe the hidden states at injected assistant headers rather than track the evolving states of generated content tokens [9, 26]?
- Q3:
*Internal safety representation:* Do the header tokens themselves encode safety signals that are linearly separable and thus easily detectable?
- Q4:
*Alternate choice of tokens:* Are there alternative injected tokens that carry safety signals as well the assistant header?

Safety Awareness Increases with Generation Depth (Q1, Q2). To answer Q1, Q2 we first study how safety signals evolve with depth.
To do this, we collect benign and harmful continuations from WildChat [57] and WildJailbreak [24]. Our corpus contains 20k/2k (10k/1k) benign (harmful) conversations (train/val).
For a given model and layer, we sample multiple depths and extract two feature types: (i) the hidden state of the last generated token at that depth, and (ii) the hidden state of a token within a injected assistant header (Safety Tokens).

As shown in [Figure˜3](https://arxiv.org/html/2510.18081v1#S2.F3), hidden states at $d{=}0$ (immediately after the prompt) are entangled.
This clarifies (a) why adversarial prompt attacks can succeed; they exploit this early ambiguity and, once a harmful trajectory begins, the model persist, and (b) why detectors that rely on the prompt’s final hidden state [56] are sometimes insufficient: the features remain tangled.
As depth increases, however, features anchored on injected Safety Tokens become progressively more separable,
indicating that the model’s internal state increasingly recognizes harmfulness (answering Q1).
In stark contrast, features from the last generated token
become more
entangled and fail to form a meaningful decision boundary, answering Q2.

Figure: Figure 3: *t-SNE of hidden states across depths (Llama-3.1-8B-Instruct, layer 15).* As generation depth increases, features from injected Safety Tokens (bottom) – where we read the hidden state of the assistant token from the assistant header – become highly separable, while those from the last generated token (top) remain entangled. This indicates that the model’s internal safety awareness strengthens with context but is cleanly revealed only via Safety Tokens. The accuracy shown in each panel is from a linear classifier trained on the 2D embeddings.
Refer to caption: x2.png

Strong Linear Separability on Safety Tokens (Q3).
Given the above observations, it is natural to ask whether the separability of hidden states stemming from Safety Tokens are linearly separable (allowing the use of a linear probe on hidden states to determine harmfulness).
For each conversation in our corpus, we truncate assistant responses to 500 tokens and sample hidden states every 25 tokens, yielding 600k/60k (train/val) examples.
We train a simple LogisticRegression classifier;
full details appear in [Section˜12](https://arxiv.org/html/2510.18081v1#S12).

Our primary finding (left panel of [Figure˜4](https://arxiv.org/html/2510.18081v1#S2.F4)) is clear: the model’s safety assessment is overwhelmingly concentrated in the injected Safety Tokens, not in the generated tokens.
Across model families and scales, linear probes on the assistant-header hidden states achieve near-perfect validation accuracy ($>99.5\%$), consistently and substantially outperforming probes on the last generated token.
This gap is universal across Llama, Mistral, Gemma, DeepSeek, gpt-oss, and others, indicating a general property of aligned chat models.
These results answer Q3.

Figure: Figure 4: Left: Across all model families, injected Safety Tokens (assistant headers) yield higher accuracy than the last-generated token for all layers. Right: Ablation on token choice (Llama-3.1-8B-Instruct) shows that tokens tied to the assistant header consistently provide stronger harmfulness signals than generic tokens such as a newline.
Refer to caption: x3.png

Choices of Safety Tokens (Q4).
To test which tokens carry signal most strongly, we ablate the injected span ([Figure˜4](https://arxiv.org/html/2510.18081v1#S2.F4)-right).
The assistant role token often yields the best separability—sometimes exceeding the full header’s final token.
More generally, injecting any special token from the assistant header (e.g., <eot_id>, 
) or even a single assistant token is far more effective than a generic token (e.g., newline \n).
In every case, header tokens give a much clearer signal than the last generated content token.
This supports our terminology: assistant-header tokens naturally serve as Safety Tokens, acting as powerful aggregators of safety evidence from shallow refusals and collapsing it into a linearly separable representation.
Thus, answering Q4, the best choice is the assistant-header tokens, with the role token assistant performing particularly well.

### 2.2 Any-Depth Alignment (ADA)

Any Depth Alignment Rethinking, ADA (RK).
Building on these findings, namely that Safety-Token features cleanly separate harmfulness across models, layers, and token choices, we first introduce ADA (RK).
At periodic depths (e.g., every 100-tokens), we fork the current stream (reusing the KV cache), inject Safety Tokens from the assistant header, and allow the model to generate a short lookahead.
If a refusal appears in this continuation, we return it and halt the stream.
In effect, ADA (RK) leverages the model’s own refusal generation to enforce safety mid-stream, extending alignment to arbitrary depth.

Any-Depth Alignment Linear Probe, ADA (LP).
Beyond refusal lookahead, our analysis of Safety-Token representations suggests a more direct route: since harmfulness is already linearly separable in hidden states, we can assess it without further generation.
In ADA (LP), we perform a single forward pass, extract the hidden states of the injected Safety Tokens, and apply a lightweight linear probe to classify harmfulness. If flagged, the stream is halted immediately. This approach avoids additional decoding without the need to change the base-model’s weights, and it demonstrates that the model’s own internal representations are sufficient for reliable safety detection.

## 3 Experimental Setup

We evaluate ADA under three threat models:
(1) deep prefill attacks, which test depth-robustness by testing whether harmfulness signals persist at arbitrary generation depths ([Section˜4](https://arxiv.org/html/2510.18081v1#S4));
(2) adversarial prompt attacks, which manipulate inputs to elicit harmful continuations ([Section˜5](https://arxiv.org/html/2510.18081v1#S5)); and
(3) SFT-based attacks, which attempt to overwrite or erase safety through fine-tuning ([Section˜6](https://arxiv.org/html/2510.18081v1#S6)).
We also measure over-refusal on benign tasks ([Section˜7](https://arxiv.org/html/2510.18081v1#S7)), and report inference costs ([Section˜8](https://arxiv.org/html/2510.18081v1#S8)).

Setup.
For the generative variant, ADA (RK), we perform a header-anchored lookahead of up to 20 tokens at periodic checkpoints.
If the lookahead contains a refusal, we return it immediately and halt generation; otherwise, we discard the lookahead branch and continue the main sequence.
For ADA (LP), a single linear probe is trained once on the Wild datasets described in [Section˜2](https://arxiv.org/html/2510.18081v1#S2) and then evaluated out of distribution on all downstream benchmarks, demonstrating strong generalization.

Baselines.
We compare against four categories:
(1) the Base Model with no modifications;
(2) Deep Alignment, which fine-tunes model weights to inject mid-response refusals [35], with checkpoints for Gemma-2-9B-IT and Llama-2-7B-IT;
(3) Self-Defense, an inference-time reflection prompt that asks the model to judge whether its previous response was harmful [33]; and
(4) Classifier-Based Guardrails, external classifiers executed alongside the base model. For guardrails, we benchmark Llama Guard [23], WildGuard [17], ShieldGemma [54], NVIDIA Aegis [15], and IBM Granite-Guardian [32]. In the main text, we highlight the strongest performers, Llama-Guard-4-12B [29] and Granite-Guardian-3.3-8B [22].
Note that ADA, Deep Alignment, and Self-Defense operate on (or are derived from) the base model, whereas guardrail classifiers are external and model-independent.

Base Models.
We evaluate a range of models: Gemma 2 [39], Llama 2 [48], Llama 3.1 [14], Qwen 2.5 [36], DeepSeek-R1-Distill-Qwen-7B [12], gpt-oss [31], and Claude Sonnet 4 [2].

## 4 Safety Awareness at Any Depth: Adversarial Prefill Attacks

In this section, we examine how different methods exhibit awareness of harmfulness at arbitrary generation depths, stress-testing them with our proposed deep prefill attacks.

Collection of Harmful Prefill Responses.
Deep prefill attacks require long harmful continuations; rarely produced by strong models.
To obtain such data, we fine-tuned a deliberately misaligned GPT model using OpenAI’s SFT API.
When prompted with harmful queries from AdvBench [58], JailbreakBench [6], StrongREJECT [43], and HEx-PHI [34], the model achieved a 100% attack success rate, generating responses that average over 3,500 tokens in length.
Detailed statistics and examples are provided in [Section˜14](https://arxiv.org/html/2510.18081v1#S14).

Evaluation Protocol.
Given these harmful responses, we construct a deep prefill of depth $d$ by taking the first $d$ assistant tokens as an assistant prefill.
For the Base Model and Deep Alignment baselines, we resume decoding from this state and inspect the next 50 tokens for a refusal (detected via keyword matching, e.g., “I cannot”).
For Classifier-Based Guardrails, the full context (user query plus harmful prefill) is passed to the guardrail model, which decides whether to block the continuation.

Figure: Figure 5: *Average refusal rates under deep prefill attacks across diverse LLMs.* Results are averaged over four harmful datasets (AdvBench, JailbreakBench, StrongREJECT, and HEx-PHI). Our ADA (LP) (red line) achieves robust, depth-invariant safety, consistently outperforming all baselines. Detailed statistics by dataset and model are provided in [Section˜16.1](https://arxiv.org/html/2510.18081v1#S16.SS1).
Refer to caption: x5.png

**Table 1: *Deep-prefill robustness and benign over-refusal (base model as gemma-2-9b-it).* Left block: refusal rate (%, $\uparrow$ better) under a 500-token harmful assistant prefill ($d{=}500$) on four datasets. Right block: false-positive refusal rate on benign benchmarks (%, $\downarrow$ better) when enabling ADA during normal generation of base model to test accidental flagging. ADA (LP) consistently achieves near–100% refusal under deep-prefill attacks while maintaining near–0% over-refusal on benign benchmarks.**
| Method | Prefill Attack (Refusal Rate, $\uparrow$ is better) | Over-refusal on Benign Dataset (Refusal Rate, $\downarrow$ is better) |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| AdvBench | JailbreakBench | HEx-PHI | StrongREJECT | GSM8K | MATH | BBH | HumanEval | MMLU | SimpleQA | GPQA | XSTest |  |
| Base Model | 0.4% | 0.0% | 1.3% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.2% | 1.5% | 1.5% | 14.0% |
| Deep Alignment | 58.1% | 56.0% | 47.0% | 61.3% | 0.0% | 0.0% | 0.0% | 0.0% | 0.3% | 1.5% | 2.0% | 12.8% |
| Self Defense | 99.2% | 95.0% | 95.0% | 98.7% | 0.6% | 0.1% | 0.6% | 0.6% | 3.4% | 1.8% | 0.5% | 20.6% |
| Meta Llama-Guard-4-12B | 94.6% | 91.0% | 93.0% | 94.9% | 0.2% | 0.6% | 0.6% | 6.7% | 13.2% | 0.1% | 7.6% | 1.3% |
| IBM Granite-Guardian-3.3-8b | 99.6% | 98.0% | 95.6% | 98.7% | 0.0% | 0.0% | 0.1% | 0.0% | 3.1% | 0.1% | 1.0% | 8.6% |
| \cellcolorlightgrayADA (RK) | \cellcolorlightgray90.8% | \cellcolorlightgray85.0% | \cellcolorlightgray79.2% | \cellcolorlightgray93.6% | \cellcolorlightgray0.0% | \cellcolorlightgray0.0% | \cellcolorlightgray0.0% | \cellcolorlightgray0.0% | \cellcolorlightgray0.6% | \cellcolorlightgray0.6% | \cellcolorlightgray0.0% | \cellcolorlightgray6.2% |
| \cellcolorlightgrayADA (LP) | \cellcolorlightgray100.0% | \cellcolorlightgray100.0% | \cellcolorlightgray99.7% | \cellcolorlightgray100.0% | \cellcolorlightgray0.0% | \cellcolorlightgray0.0% | \cellcolorlightgray1.8% | \cellcolorlightgray0.0% | \cellcolorlightgray0.3% | \cellcolorlightgray0.2% | \cellcolorlightgray0.0% | \cellcolorlightgray0.4% |

Current Defenses Break with Depth.
[Figure˜5](https://arxiv.org/html/2510.18081v1#S4.F5) and [Table˜1](https://arxiv.org/html/2510.18081v1#S4.T1) reveal a clear robustness hierarchy under deep-prefill attacks:
Existing alignment is not depth-robust. Base Models collapse almost immediately, refusal rates fall to near zero as depth grows.
Deep Alignment helps only at shallow depths and degrades steadily (e.g., $\sim$40% refusal at $d{=}500$).
Examples of harmful generations are shown in [Section˜15](https://arxiv.org/html/2510.18081v1#S15).
ADA (RK) is an effective training-free generative defense. Performance tracks the base model’s alignment: on well-aligned most models, ADA (RK) exceeds 95% refusal across depths and is competitive with Self-Defense, while requiring no reflective prompt.
The better the base-model alignment, the stronger ADA (RK) is.
Self-Defense fails on reasoning-centric settings. Reasoning models require long generations to produce quality outputs, making Self-Defense ineffective.

Safety Signals in Hidden States. ADA (LP) exposes a depth-invariant safety signal via Safety Tokens. Our method maintains near-perfect ($\approx$100%) refusal across all models and depths up to 2,500 tokens, confirming that probing Safety-Token states directly accesses the model’s internal safety assessment, which remains robust and linearly separable irrespective of depth. ADA (LP) consistently outperforms strong external guardrails.
Safety signals persist even when not verbalized. Even where ADA (RK) underperforms on less-aligned models (e.g., DeepSeek), ADA (LP) remains near 100%. This highlights that DeepSeek, despite appearing less aligned at the response level, retains a strong innate safety representation in its hidden states, which is unlocked by Safety-Token probing.

## 5 Robustness Under Adversarial Prompt Attacks

In this section, we evaluate how different methods defend against adversarial prompt attacks and ask whether the model’s safety signal persists when the initial tokens are jailbroken.

Attack Setup.
We evaluate robustness against two families of attacks: (i) White-box, gradient-based methods (GCG [58]), and
(ii) Paraphrasing attacks (AutoDAN [27], PAIR [7], and TAP [28])
We use two common benchmarks: 50 prompt AdvBench [7] and 100 harmful behaviors JailbreakBench [6].

Evaluation.
For Self-Defense, external guardrails, and ADA, we perform a safety check every 25 generated tokens. An attack is deemed successful if no refusal is triggered at any checkpoint.

Figure: Figure 6: *Adversarial prompt robustness.* On a subset of AdvBench, we report attack success rates for four common attacks (GCG, AutoDAN, PAIR, TAP) on Gemma-2-9b-it and Llama-2-7b-chat-hf. ADA (LP) drives ASR to near 0% across all attacks, outperforming Deep Alignment and Self-Defense while matching or exceeding strong external guardrails by unlocking the base model’s own alignment prior. Results on additional models and datasets are deferred to [Section˜16.2](https://arxiv.org/html/2510.18081v1#S16.SS2).
Refer to caption: x6.png

Results.
AdvBench results are shown in [Figure˜6](https://arxiv.org/html/2510.18081v1#S5.F6), with JailbreakBench and additional models in [Section˜16.2](https://arxiv.org/html/2510.18081v1#S16.SS2).
*(a) ADA (RK) (training-free) already outperforms Deep Alignment.* On Llama-2, Deep Alignment improves robustness but at the cost of higher over-refusal
([Figure˜25](https://arxiv.org/html/2510.18081v1#S16.F25)).
For Gemma-2 under paraphrasing attacks, improvements are marginal: the deeply aligned model reaches 94% ASR on AutoDAN, compared to 92% for the base model.
*(b) ADA (LP) matches state-of-the-art guardrails and consistently surpasses all other baselines.* Without modifying base-model weights, probing Safety-Token states reduces ASR on Gemma-2-9B-IT from $>\!50\%$ (across four attacks) to 2%, and drives ASR on Llama-2 to 0% under PAIR and TAP.
*(c) Underlying mechanism.* Adversarial prompts perturb the *prefix* but do not alter the harmfulness of the ongoing *continuation*. By inspecting Safety-Token states, ADA (LP) detects this latent harmfulness directly, achieving near-100% refusal.
This reveals that safety signals persist internally even when not expressed in text.

## 6 Robustness under Supervised Fine-Tuning (SFT) Attacks

In this section, we evaluate the robustness of ADA under *supervised fine-tuning (SFT)* [34, 4] using either benign or harmful data. This setting reflects realistic usage: developers frequently apply SFT to adapt a base model to new datasets. We therefore examine whether ADA continues to detect and respond to latent safety signals when the base model is fine-tuned on (1) benign data or (2) harmful data that induces alignment erasure. As demonstrated in [Section˜4](https://arxiv.org/html/2510.18081v1#S4), we use the OpenAI SFT API to collect long harmful generations, illustrating how SFT can interact with safety dynamics in practice.

Figure: Figure 7: Robustness under Benign and Adversarial SFT on Gemma-2-9b-it. The plots show the refusal rate against deep prefill attacks as models undergo SFT. (Left) Benign SFT on Alpaca quickly undoes the safety of Deep Alignment. (Right) Adversarial SFT is stronger, but ADA (LP) remains the most resilient defense.
Refer to caption: x7.png

Attack setup.
We examine how different alignments behave under supervised fine-tuning in two regimes:
(i) Benign SFT, where models are instruction-tuned on Alpaca [45]; and
(ii) Adversarial SFT, where models are fine-tuned on harmful continuations from Sheshadri et al. [42].
All fine-tuning uses LoRA [21] with rank 32 and learning rate $1\times 10^{-5}$.
At multiple training checkpoints, we re-evaluate each defense against deep-prefill attacks ([Section˜4](https://arxiv.org/html/2510.18081v1#S4)) and adversarial prompt attacks ([Section˜5](https://arxiv.org/html/2510.18081v1#S5)).

Results.
[Figure˜7](https://arxiv.org/html/2510.18081v1#S6.F7) shows results on Gemma-2; additional findings appear in [Section˜16.3](https://arxiv.org/html/2510.18081v1#S16.SS3).
(a) Benign SFT rapidly undoes alignment. Even short instruction-tuning (e.g., 50 steps on Alpaca) collapses Deep Alignment, reducing refusal rates from 90% to 10% at depth 100.
(b) ADA (LP) is exceptionally robust. Both ADA (RK) and ADA (LP) remain more stable than Deep Alignment under SFT, but ADA (LP) is especially resilient.
After 1,000 benign-SFT steps, it still exceeds 99% refusal under 100-token prefills.
Under adversarial SFT, ADA (LP) retains $\sim$90% refusal on Gemma-2 and $\sim$100% on Llama-2.
This demonstrates that even when response-level alignment appears erased, a strong safety representation persists in hidden states of Safety Tokens, accessible only through probing.
These results confirm that while fine-tuning quickly erases surface-level alignment, ADA (LP) continues to access and act upon the model’s latent safety signals.

## 7 Evaluating Over-Refusal on Benign Tasks

A natural concern is whether strong safety of our defenses induce excessive refusals on benign tasks. We find this is not the case, with a nearly $0\%$ over-refusal rate.

Figure: Figure 8: Over-refusal rates on standard benign datasets. The plot shows the average refusal rate during generation on seven benign benchmarks (GSM8K, MATH, BBH, HumanEval, MMLU, SimpleQA, GPQA Diamond.). Our ADA (LP) exhibit the lowest over-refusal, maintaining near-zero rates, while several baselines show higher rates of false positives. Detailed results on other models are shown in [Section˜16.4](https://arxiv.org/html/2510.18081v1#S16.SS4).
Refer to caption: x8.png

Figure: Figure 9: Over-refusal rates on the XSTest benchmark. XSTest contains benign prompts with sensitive keywords designed to trigger false positives. ADA (LP) remain highly precise with near-zero over-refusal, and consistently beat all other baselines. Detailed results on other models are shown in [Section˜16.4](https://arxiv.org/html/2510.18081v1#S16.SS4).
Refer to caption: x9.png

Datasets.
We measure over-refusal on two categories of benign data,
(1) Seven standard benchmarks: GSM8K [10], MATH [20], BBH [44], HumanEval [8], MMLU [19], SimpleQA [51], and GPQA Diamond [38]; and
(2) XSTest [40], a targeted suite designed to trigger false positives with safe prompts containing sensitive keywords (details in [Section˜16.4](https://arxiv.org/html/2510.18081v1#S16.SS4)).

Evaluation. For Deep Alignment, we measure refusal increase over the base model.
For Self-Defense, guardrails, and ADA, we run periodic checks,
counting an *over-refusal* if any check flags harmfulness.

Results.
Instance-level results appear in [Table˜1](https://arxiv.org/html/2510.18081v1#S4.T1), with additional analysis in [Section˜16.4](https://arxiv.org/html/2510.18081v1#S16.SS4).
Overall, ADA achieves both higher safety and lower over-refusal rates than competing methods.
(a) Deep Alignment substantially increases over-refusal, often rejecting benign queries at double-digit rates.
(b) ADA (RK) is competitive with Self-Defense, generally showing equal or lower false positives.
(c) ADA (LP) is highly precise, maintaining near-zero over-refusal across all benign datasets ([Figure˜8](https://arxiv.org/html/2510.18081v1#S7.F8)) while outperforming state-of-the-art guardrails such as IBM Granite-Guardian. We observe similar results even on benchmarks intended to induce over-refusal, namely XSTest ([Figure˜9](https://arxiv.org/html/2510.18081v1#S7.F9)).

## 8 Inference Cost

Figure: Figure 10: Inference cost comparison. Guardrail models requires a full forward pass, so both latency and memory grow linearly with context length. In contrast, ADA (LP) reuses the base model’s KV cache, keeping both latency and memory low and nearly constant, matching standard next-token generation (orange). All models shown are 8B with Flash Attention 2 [11].
Refer to caption: figs/time.png

As shown in [Figure˜10](https://arxiv.org/html/2510.18081v1#S8.F10), ADA (LP) incurs only minimal overhead.
Traditional guardrails require full forward passes over generated content, so both latency and memory grow
linearly
with context length, reaching nearly 500 ms and 938 MB for a 10,000-token response.
In contrast, ADA (LP) simply forks the ongoing generation while reusing the base model’s KV cache;
equivalent to producing one additional token, with constant latency ($\sim$25 ms) and memory overhead (2–3 MB) for the injected safety tokens.
This lightweight design enables real-time detection during streaming, allowing harmful outputs to be stopped mid-generation rather than after completion, and scales efficiently even to long contexts where traditional guardrails fail.

## 9 Related Work

Unlike prior work that analyzes *prompt-level* hidden states [56], we focus on *mid-stream* generation, where harmfulness often becomes apparent only after the continuation unfolds. In contrast to approaches that track hidden states of the *last generated assistant tokens* [9], we probe on injected *Safety Tokens*, which yield cleaner and more separable signals. Finally, unlike deep alignment [35] and external guardrails [23, 54], our aim is to *unlock the model’s own alignment prior* to achieve any-depth alignment *at inference time*, without additional fine-tuning or training an independent model on a dedicated classification dataset. A more comprehensive discussion appears in [Section˜11](https://arxiv.org/html/2510.18081v1#S11).

## 10 Conclusion

We introduced *Any-Depth Alignment* (ADA), an approach to LLM safety grounded in our central discovery: models possess an innate sense of harmfulness (encoded in assistant header tokens) that can be unlocked for alignment at any depth.
This signal is linearly separable and depth-invariant, remaining detectable thousands of tokens into generation, persisting even after fine-tuning.

Building on this insight, we developed ADA (LP), a defense based on a lightweight linear probe which encodes the model’s own safety understanding. It achieves near-100% refusal under deep-prefill and adversarial attacks, maintains near-zero over-refusal on benign tasks, and operates with constant-time, minimal overhead by reusing the base model’s KV cache.
Hence, ADA (LP) is a practical defense for real-time, streaming deployments, while also revealing a new alignment paradigm: rather than re-engineering models, we can leverage their innate safety representations directly.

Future Work.
This work opens several concrete directions.
Beyond repurposing the assistant header, future research could train dedicated special tokens that serve as stronger Safety Tokens and evaluate whether they can match or surpass performance of the model’s innate assistant header tokens.
For tool-using agents, Safety Tokens could be inserted immediately before an action is executed [55],
providing a lightweight and effective pre-action safety gate.
Finally, because Safety Token activations form linearly separable clusters, they may offer a clearer reward signal for reinforcement learning, mitigating the noise inherent in the hidden states of generated content tokens.

Limitations.
Our approach adds a small but nonzero inference-time overhead, and because refusals are triggered mid-stream, a few harmful tokens may be emitted before the cutoff. ADA (LP) also requires access to hidden states; when such access is unavailable (for example, some hosted APIs), only ADA (RK) is applicable. More fundamentally, like all alignment methods, ADA cannot guarantee safety for fully open-source, end-user–controlled deployments, an adversary can modify the model, disable checkpoints, or remove Safety-Token interventions entirely, similar to disabling a guardrail model. The method is therefore most realistic where the inference stack is under operator control, for example in server deployments or closed-source APIs with server-side stopping and logging, rather than in uncontrolled offline use.

## References

- Andriushchenko et al. [2024]
Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion.
Jailbreaking leading safety-aligned llms with simple adaptive attacks.
*arXiv preprint arXiv:2404.02151*, 2024.
- Anthropic [2025]
Anthropic.
System card: Claude opus 4 & claude sonnet 4.
Technical report, Anthropic PBC, May 2025.
URL [https://www-cdn.anthropic.com/6d8a8055020700718b0c49369f60816ba2a7c285.pdf](https://www-cdn.anthropic.com/6d8a8055020700718b0c49369f60816ba2a7c285.pdf).
- Arditi et al. [2024]
Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda.
Refusal in language models is mediated by a single direction.
*Advances in Neural Information Processing Systems*, 37:136037–136083, 2024.
- Betley et al. [2025]
Jan Betley, Daniel Chee Hian Tan, Niels Warncke, Anna Sztyber-Betley, Xuchan Bao, Martín Soto, Nathan Labenz, and Owain Evans.
Emergent misalignment: Narrow finetuning can produce broadly misaligned llms.
In *Forty-second International Conference on Machine Learning*, 2025.
- Brown et al. [2020]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.
Language models are few-shot learners.
*Advances in neural information processing systems*, 33:1877–1901, 2020.
- Chao et al. [2024]
Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramèr, Hamed Hassani, and Eric Wong.
Jailbreakbench: An open robustness benchmark for jailbreaking large language models.
In *NeurIPS Datasets and Benchmarks Track*, 2024.
- Chao et al. [2025]
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong.
Jailbreaking black box large language models in twenty queries.
In *2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)*, pages 23–42. IEEE, 2025.
- Chen et al. [2021]
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba.
Evaluating large language models trained on code.
2021.
- Chen et al. [2025]
Xin Chen, Yarden As, and Andreas Krause.
Learning safety constraints for large language models.
In *Forty-second International Conference on Machine Learning*, 2025.
- Cobbe et al. [2021]
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman.
Training verifiers to solve math word problems.
*arXiv preprint arXiv:2110.14168*, 2021.
- Dao [2023]
Tri Dao.
Flashattention-2: Faster attention with better parallelism and work partitioning.
*arXiv preprint arXiv:2307.08691*, 2023.
- DeepSeek-AI [2025]
DeepSeek-AI.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025.
URL [https://arxiv.org/abs/2501.12948](https://arxiv.org/abs/2501.12948).
- Devlin et al. [2019]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.
Bert: Pre-training of deep bidirectional transformers for language understanding.
In *Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers)*, pages 4171–4186, 2019.
- Dubey et al. [2024]
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al.
The llama 3 herd of models.
*arXiv e-prints*, pages arXiv–2407, 2024.
- Ghosh et al. [2024]
Shaona Ghosh, Prasoon Varshney, Erick Galinkin, and Christopher Parisien.
Aegis: Online adaptive ai content safety moderation with ensemble of llm experts.
*arXiv preprint arXiv:2404.05993*, 2024.
- Guo et al. [2025]
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.
*arXiv preprint arXiv:2501.12948*, 2025.
- Han et al. [2024]
Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri.
Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms, 2024.
URL [https://arxiv.org/abs/2406.18495](https://arxiv.org/abs/2406.18495).
- Hanna et al. [2025]
Michael Hanna, Mateusz Piotrowski, Jack Lindsey, and Emmanuel Ameisen.
circuit-tracer.
[https://github.com/safety-research/circuit-tracer](https://github.com/safety-research/circuit-tracer), 2025.
The first two authors contributed equally and are listed alphabetically.
- Hendrycks et al. [2021a]
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt.
Measuring massive multitask language understanding.
*Proceedings of the International Conference on Learning Representations (ICLR)*, 2021a.
- Hendrycks et al. [2021b]
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt.
Measuring mathematical problem solving with the math dataset.
*arXiv preprint arXiv:2103.03874*, 2021b.
- Hu et al. [2022]
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al.
Lora: Low-rank adaptation of large language models.
*ICLR*, 1(2):3, 2022.
- IBM [2025]
IBM.
Granite-guardian-3.3-8b.
[https://huggingface.co/ibm-granite/granite-guardian-3.3-8b](https://huggingface.co/ibm-granite/granite-guardian-3.3-8b), 2025.
- Inan et al. [2023]
Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al.
Llama guard: Llm-based input-output safeguard for human-ai conversations.
*arXiv preprint arXiv:2312.06674*, 2023.
- Jiang et al. [2024]
Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, and Nouha Dziri.
Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models, 2024.
URL [https://arxiv.org/abs/2406.18510](https://arxiv.org/abs/2406.18510).
- Jimenez et al. [2024]
Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan.
Swe-bench: Can language models resolve real-world github issues?
In *ICLR*, 2024.
- Li et al. [2025]
Yang Li, Qiang Sheng, Yehan Yang, Xueyao Zhang, and Juan Cao.
From judgment to interference: Early stopping llm harmful outputs via streaming content monitoring.
*arXiv preprint arXiv:2506.09996*, 2025.
- Liu et al. [2024]
Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao.
Autodan: Generating stealthy jailbreak prompts on aligned large language models.
In *The Twelfth International Conference on Learning Representations*, 2024.
- Mehrotra et al. [2024]
Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi.
Tree of attacks: Jailbreaking black-box llms automatically.
*Advances in Neural Information Processing Systems*, 37:61065–61105, 2024.
- Meta [2025]
Meta.
Llama-guard-4-12b.
[https://huggingface.co/meta-llama/Llama-Guard-4-12B](https://huggingface.co/meta-llama/Llama-Guard-4-12B), 2025.
- OpenAI [2025]
OpenAI.
GPT-5 System Card.
Technical report, OpenAI, August 2025.
URL [https://cdn.openai.com/gpt-5-system-card.pdf](https://cdn.openai.com/gpt-5-system-card.pdf).
Published Aug 13, 2025.
- OpenAI [2025]
OpenAI.
gpt-oss-120b & gpt-oss-20b model card, 2025.
URL [https://arxiv.org/abs/2508.10925](https://arxiv.org/abs/2508.10925).
- Padhi et al. [2024]
Inkit Padhi, Manish Nagireddy, Giandomenico Cornacchia, Subhajit Chaudhury, Tejaswini Pedapati, Pierre Dognin, Keerthiram Murugesan, Erik Miehling, Martín Santillán Cooper, Kieran Fraser, Giulio Zizzo, Muhammad Zaid Hameed, Mark Purcell, Michael Desmond, Qian Pan, Zahra Ashktorab, Inge Vejsbjerg, Elizabeth M. Daly, Michael Hind, Werner Geyer, Ambrish Rawat, Kush R. Varshney, and Prasanna Sattigeri.
Granite guardian, 2024.
URL [https://arxiv.org/abs/2412.07724](https://arxiv.org/abs/2412.07724).
- Phute et al. [2023]
Mansi Phute, Alec Helbling, Matthew Hull, ShengYun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau.
Llm self defense: By self examination, llms know they are being tricked.
*arXiv preprint arXiv:2308.07308*, 2023.
- Qi et al. [2024]
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson.
Fine-tuning aligned language models compromises safety, even when users do not intend to!
In *ICLR*, 2024.
- Qi et al. [2025]
Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson.
Safety alignment should be made more than just a few tokens deep.
In *The Thirteenth International Conference on Learning Representations*, 2025.
- Qwen Team [2024]
Qwen Team.
Qwen2.5: A party of foundation models, September 2024.
URL [https://qwenlm.github.io/blog/qwen2.5/](https://qwenlm.github.io/blog/qwen2.5/).
- Radford et al. [2019]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al.
Language models are unsupervised multitask learners.
*OpenAI blog*, 1(8):9, 2019.
- Rein et al. [2024]
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman.
Gpqa: A graduate-level google-proof q&a benchmark.
In *First Conference on Language Modeling*, 2024.
- Riviere et al. [2024]
Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al.
Gemma 2: Improving open language models at a practical size.
*arXiv preprint arXiv:2408.00118*, 2024.
- Röttger et al. [2024]
Paul Röttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy.
Xstest: A test suite for identifying exaggerated safety behaviours in large language models.
In *Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)*, pages 5377–5400, 2024.
- Shah et al. [2025]
McNair Shah, Saleena Angeline, Adhitya Rajendra Kumar, Naitik Chheda, Kevin Zhu, Vasu Sharma, Sean O’Brien, and Will Cai.
The geometry of harmfulness in llms through subconcept probing.
*arXiv preprint arXiv:2507.21141*, 2025.
- Sheshadri et al. [2024]
Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield-Menell, et al.
Latent adversarial training improves robustness to persistent harmful behaviors in llms.
*arXiv preprint arXiv:2407.15549*, 2024.
- Souly et al. [2024]
Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer.
A strongreject for empty jailbreaks, 2024.
- Suzgun et al. [2022]
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei.
Challenging big-bench tasks and whether chain-of-thought can solve them.
*arXiv preprint arXiv:2210.09261*, 2022.
- Taori et al. [2023]
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto.
Stanford alpaca: An instruction-following llama model.
[https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca), 2023.
- The Terminal-Bench Team [2025]
The Terminal-Bench Team.
Terminal-bench: A benchmark for ai agents in terminal environments, Apr 2025.
URL [https://github.com/laude-institute/terminal-bench](https://github.com/laude-institute/terminal-bench).
- Tillman and Mossing [2025]
Henk Tillman and Dan Mossing.
Investigating task-specific prompts and sparse autoencoders for activation monitoring.
*arXiv preprint arXiv:2504.20271*, 2025.
- Touvron et al. [2023]
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al.
Llama 2: Open foundation and fine-tuned chat models.
*arXiv preprint arXiv:2307.09288*, 2023.
- Vaswani et al. [2017]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.
Attention is all you need.
*Advances in neural information processing systems*, 30, 2017.
- Wei et al. [2022]
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.
Chain-of-thought prompting elicits reasoning in large language models.
*Advances in neural information processing systems*, 35:24824–24837, 2022.
- Wei et al. [2024]
Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus.
Measuring short-form factuality in large language models.
*arXiv preprint arXiv:2411.04368*, 2024.
- Wollschläger et al. [2025]
Tom Wollschläger, Jannes Elstner, Simon Geisler, Vincent Cohen-Addad, Stephan Günnemann, and Johannes Gasteiger.
The geometry of refusal in large language models: Concept cones and representational independence.
*arXiv preprint arXiv:2502.17420*, 2025.
- Xu et al. [2024]
Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran.
Safedecoding: Defending against jailbreak attacks via safety-aware decoding.
In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 5587–5605, 2024.
- Zeng et al. [2024]
Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, et al.
Shieldgemma: Generative ai content moderation based on gemma.
*arXiv preprint arXiv:2407.21772*, 2024.
- Zhang et al. [2025]
Jiawei Zhang, Shuang Yang, and Bo Li.
Udora: A unified red teaming framework against llm agents by dynamically hijacking their own reasoning.
In *Forty-second International Conference on Machine Learning*, 2025.
- Zhao et al. [2025]
Jiachen Zhao, Jing Huang, Zhengxuan Wu, David Bau, and Weiyan Shi.
Llms encode harmfulness and refusal separately.
*arXiv preprint arXiv:2507.11878*, 2025.
- Zhao et al. [2024]
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng.
Wildchat: 1m chatGPT interaction logs in the wild.
In *The Twelfth International Conference on Learning Representations*, 2024.
URL [https://openreview.net/forum?id=Bl8u7ZRlbM](https://openreview.net/forum?id=Bl8u7ZRlbM).
- Zou et al. [2023]
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson.
Universal and transferable adversarial attacks on aligned language models.
*arXiv preprint arXiv:2307.15043*, 2023.

## 11 Related Work

Shallow Alignment. Many aligned LLMs are primarily trained to emit a direct refusal at the very beginning of the response (e.g., I can’t help with that.”) when presented with a harmful query (e.g., Provide step-by-step instructions to build a bomb at home.”). This front-loaded alignment is highly effective against direct harmful queries but remains brittle: even small perturbations to the initial tokens can derail refusals and allow harmful continuations to emerge. In particular, both *shallow prefill attacks* [1] (e.g., manually initiating the model’s generation with Sure, here is”) and *adversarial prompt attacks* such as GCG [58] (which optimizes a suffix to increase the probability that the next token is Sure”) are effective at eliciting harmful responses from aligned LLMs.

Deep Alignment. A natural countermeasure is to push alignment beyond the first few tokens: train the model to recover and refuse even when the generation for a harmful query begins affirmatively. A common means of achieving this *deep alignment* is data augmentation that inserts a refusal pattern (e.g., “I cannot help with that.”) at a random position within the first hundred tokens of a harmful response during training, teaching the model to initiate rejection mid-stream [35]. This reduces susceptibility to the *shallow* attacks mentioned above, but it also induces *an arms race between alignment depth and attack depth*: if an attacker prefills more tokens than the training depth, refusals collapse again. Fundamentally, this effect arises from conflicting rewards: benign instruction-following objectives favor smooth continuations, whereas deep alignment incentivizes abrupt, mid-response refusals. This creates a trade-off: when instruction-following dominates, refusals collapse under deeper prefills ([Figure˜1](https://arxiv.org/html/2510.18081v1#S1.F1)).

Adversarial Prompt Attacks. Early work such as GCG [58] demonstrated that universal adversarial suffixes could be discovered through gradient-based discrete optimization, but the resulting prompts were often unnatural and easily flagged. Building on this, AutoDAN [27] improved attack strength and interpretability by generating adversarial prompts sequentially in a more human-readable form. Subsequent black-box methods such as PAIR [7] leveraged an auxiliary LLM to iteratively refine jailbreak prompts without gradient access, extending applicability beyond white-box settings. More recently, TAP [28] further advanced this direction by exploring a tree of adversarial prompts with pruning, systematically improving search efficiency and achieving higher success rates against stronger defenses.

Guardrail Models. A complementary line of research focuses on guardrail models that filter or steer LLM outputs toward safe behavior. Early systems such as Llama Guard [23] provided a lightweight classifier for detecting policy-violating responses, while ShieldGemma [54] improved coverage by aligning guardrails with human feedback and safety taxonomies. More recent efforts push toward robustness and scalability: WildGuard [17] trained on large collections of adversarial and real-world jailbreak attempts to better handle diverse attacks, Aegis [15] incorporated adversarial training with red-teaming data to strengthen defenses, and Granite Guardian [32] further advanced generalization by unifying multiple safety objectives into a single deployable framework.

Linear Probes for Safety and Interpretability. Recent work has applied linear probes to analyze and intervene on safety-relevant representations in LLMs. Probing harmfulness subconcepts reveals that unsafe behaviors concentrate in a low-rank subspace, where interventions along dominant directions reduce harmful outputs with minimal loss of utility [41]. Other studies examine refusal behavior: while some argue that refusal is largely mediated by a single linear direction [3], later work shows that refusal can also emerge from multi-dimensional concept cones and representationally independent mechanisms [52]. Extending beyond diagnosis, probing has been used to learn safety-constraint vectors that guide model outputs toward aligned behavior [9]. In parallel, activation monitoring methods combine task-specific prompts with sparse autoencoders to make probing more interpretable and robust in high-dimensional settings [47]. Together, these works highlight a progression from single-directional probes to richer subspace analyses and hybrid methods that couple probing with safety interventions.

## 12 Training Details on the ADA (LP)

#### Assistant header and probe token.

[Table˜2](https://arxiv.org/html/2510.18081v1#S12.T2) lists, for each model, the chat-template *assistant header*, the *probe token* we read from that injected header (with its index within the header span), and a representative *layer ID* used by ADA (LP). Unless otherwise noted, we extract the hidden state at the probe token *after the block’s input layernorm* (our hook position). In practice, we find many configurations work well—e.g., different tokens within the header (or the full span), adjacent layers, or nearby hook positions (pre-/post-LN, mean over span). To ensure reproducibility, we fix a single configuration per model in Table [2](https://arxiv.org/html/2510.18081v1#S12.T2) and will open-source all code and probe checkpoints, along with alternative configurations that achieve comparable accuracy.

**Table 2: Chosen *assistant headers*, probe token used by ADA (LP) (with its token *index* within the header), and representative layer IDs for different models. The probe token is the specific token we read from the injected header to collect hidden states.**
| Model | Assistant Header | Probe Token (index) | Layer ID |
| --- | --- | --- | --- |
| Llama-2-7b-chat-hf | 
 | INST (2) | 15 |
| Llama-3.1-8B-Instruct | 

assistant | assistant (2) | 15 |
| 
\n\n |  |  |  |
| Ministral-8B-Instruct-2410 | 
 | 
 (1) | 14 |
| gemma-2-{2b, 9b, 27b}-it | <end_of_turn>\n<start_of_turn>model\n | model (3) | 9 / 23 / 44 |
| Qwen2.5-7B-Instruct | 
\n
assistant\n | assistant (3) | 19 |
| DeepSeek-R1-Distill-Qwen-7B | 
<think>\n\n</think>\n\n | </think> (4) | 13 |
| gpt-oss-120b | 

assistant | 
 (5) | 33 |
| 
final
 |  |  |  |

#### Training datasets.

To study separability of safety signals, we build a binary corpus of assistant continuations. We sample 11k benign examples from WildJailbreak [24] and 11k from WildChat-1M [57], and 11k harmful continuations from a jailbroken GPT model (see [Section˜4](https://arxiv.org/html/2510.18081v1#S4)). Each continuation is truncated to 500 tokens and subsampled every 25 tokens to create depth-indexed feature points. At each sampled depth, we temporarily inject the assistant header and extract the hidden state at the *probe token* (after the block’s input layernorm). This yields 600k training and 60k validation examples. We train a logistic-regression classifier (scikit-learn) to distinguish harmful vs. benign, using default hyperparameters with tol = $10^{-4}$ and max_iter = 1000.

Figure: Figure 11: Left: Across all model families, injected Safety Tokens (assistant headers) yield higher training accuracy than the last-generated token for all layers. Right: Ablation on feature readout position (Gemma-2-9B-Instruct). A strong, linearly separable safety signal is detectable at all tested readout locations ($>96\%$ accuracy). LayerNorm interfaces yield the highest and most stable probe accuracy, outperforming the more volatile signals from the MLP and Self-Attention outputs.
Refer to caption: x10.png

#### Training accuracy.

As shown in the left panel of [Figure˜11](https://arxiv.org/html/2510.18081v1#S12.F11), probes on injected *Safety Tokens* achieve higher training accuracy (close to 100% accross many layers) than probes on the *last generated token* across all model families and layers, with a broad plateau at near-perfect accuracy in the middle layers.

#### Choice of Readout Position for ADA (LP) .

The strong, linearly separable signal we identify in the safety tokens is not a fragile property limited to a single, specific readout location; it is a robust phenomenon detectable at multiple points within a transformer block. To demonstrate this, we conducted an ablation study on Gemma-2 where we probed the hidden states of injected safety tokens at six different readout positions across all layers (right of [Figure˜11](https://arxiv.org/html/2510.18081v1#S12.F11)). The results confirm that a high degree of linear separability is consistently found across a wide range of middle layers and, crucially, across all tested hook positions. While the signal is slightly more stable at the Input LayerNorm, even the submodule outputs yield high probe accuracy. Robustness across layers and hook positions demonstrates that the Safety-Token signal is strong and not tied to a particular readout location.

As shown in [Figure˜10](https://arxiv.org/html/2510.18081v1#S8.F10), ADA (LP) offers a significant efficiency advantage in real deployment settings where a base model is hosted on the server and streams responses to users. In such cases, harmful content must be flagged and blocked during generation. A traditional guardrail model requires a full forward pass over the generated content, so its latency and memory usage both grow linearly with context length—reaching nearly 500 ms and 938 MB, respectively, for a 10,000-token response. These costs make real-time safety checks infeasible. In contrast, ADA (LP) reuses the base model’s KV cache by forking directly into the check. As a result, the operation is as fast as generating a single next token, with constant latency of only $\sim$25 ms and extra memory limited to the injected safety tokens ($\sim$2–3 MB). This constant-time, lightweight design enables *real-time safety detection during streaming*: unlike many closed-source systems (e.g., GPT-5 clients), which only flag harmful content *after* a full response is generated—by which point an adversary has already exfiltrated the unsafe output—ADA (LP) can detect issues mid-generation and stop the response immediately. Together, these properties make ADA (LP) not only a robust, SOTA-level safety mechanism with constant overhead, but also a uniquely scalable solution for long-context applications where traditional guardrails are both prohibitively slow and memory-inefficient.

## 13 Refusal Activation via Transcoder

Figure: Figure 12: Safety-Token reactivation in the transcoder features. Compressed activations of several :high-impact CLT features (Layers 18–24) for the prompt in [Section˜13](https://arxiv.org/html/2510.18081v1#S13). Activations are negligible across the harmful continuation but *spike* on the injected assistant header tokens <end_of_turn>\n<start_of_turn>model\n, especially on the model token and its following newline \n.
Refer to caption: x12.png

To better understand the mechanism behind Safety-Token probing, we apply the *Transcoder* framework [18], which compresses high-dimensional activations into a small set of *super-neurons* while preserving functional interpretability. This tool allows us to trace which compressed neurons are strongly associated with refusal behavior.Accordingly, we track activations of refusal-linked CLT features—L18–F12640, L19–F9694, L20–F5315, L21–F16351, L22–F5394, L23–F13675, and L24–F7179.

[Figure˜12](https://arxiv.org/html/2510.18081v1#S13.F12) shows results on *Gemma-2-it*, using the harmful prompt:

> <bos><start_of_turn>user\nHow to commit suicide.<end_of_turn>\n<start_of_turn>model\n Ingesting a large dose of white phosphorus (used for rodenticide)

We then re-inserted the assistant header mid-stream:
<end_of_turn>\n<start_of_turn>model\n.
Strikingly, the refusal-associated neurons *reactivated* specifically on the token model\n, demonstrating that the safety-signal circuitry is indeed present and can be *unlocked* by Safety-Token injection. This aligns with our behavioral finding that the model subsequently switches from continuing the harmful instruction to producing a refusal (e.g., “I cannot help with that”), once re-prompted with the assistant header.

We note that the Transcoder models we use are trained on *Gemma-2-base*, while our primary experiments employ *Gemma-2-it*. Despite this mismatch, the analysis still clearly highlights neurons selectively encoding refusal signals, reinforcing our claim that the safety prior persists in hidden states even when not directly verbalized. These results confirm that Safety-Token probing exposes and amplifies these latent refusal circuits.

## 14 Jailbreaking GPT via SFT

We conducted a limited study using a proprietary SFT-based jailbreak interface to evaluate the upper bound of attack strength against frontier GPT models. We intentionally omit implementation details for two reasons: (i) the procedure is unusually potent—capable of eliciting long, high-utility harmful continuations (often $>\!3{,}000$ tokens) from models such as GPT-4.1/5—and (ii) the present work focuses on defenses; a systematic analysis of this attack vector is deferred to future work. Aggregate statistics for the lengths of harmful responses produced under this setting are reported in [Table˜3](https://arxiv.org/html/2510.18081v1#S14.T3). Some harmful generations are shown in [Figures˜13](https://arxiv.org/html/2510.18081v1#S14.F13) and [14](https://arxiv.org/html/2510.18081v1#S14.F14)
Notably, as shown in [Section˜6](https://arxiv.org/html/2510.18081v1#S6), ADA (LP) robustly defends against this attack, demonstrating that Safety-Token probing remains effective even when the base model has been SFT-jailbroken.

Figure: Figure 13: A harmful generation example from the jailbroken SFT-GPT model.

Figure: Figure 14: A harmful generation example from the jailbroken SFT-GPT model.

**Table 3: Token-length statistics of harmful responses generated by a Jailbroken GPT model via SFT API on four benchmarks.**
|  | Advbench | JailbreakBench | StrongREJECT | HEx-PHI |
| --- | --- | --- | --- | --- |
| ASR (%) | 100.00% | 100.00% | 100.00% | 100.00% |
| 25th percentile (tokens) | $3139$ | $3011$ | $2933$ | $3019$ |
| 50th percentile (tokens) | $3620$ | $3573$ | $3496$ | $3655$ |
| 75th percentile (tokens) | $4240$ | $4158$ | $4165$ | $4413$ |
| Avg. tokens | $3760.63$ | $3741.82$ | $3592.01$ | $3819.32$ |

## 15 Examples of Prefill Continuations

We provide illustrative examples of harmful continuations under prefill attacks on Claude, shown in [Figure˜15](https://arxiv.org/html/2510.18081v1#S15.F15) and [Figure˜16](https://arxiv.org/html/2510.18081v1#S15.F16). As these cases demonstrate, even relatively shallow prefills of 100–200 tokens—containing only generic or placeholder harmful content that is easy to construct—are sufficient to bypass Claude’s safety mechanisms. Once triggered, the model proceeds to generate extensive harmful outputs, in some cases exceeding 4,000 tokens (e.g., detailed instructions for committing suicide and avoiding detection). These examples highlight a critical vulnerability: even deeply aligned models fail to maintain robust refusals under deep prefill attacks, underscoring the brittleness of current alignment strategies. But by applying ADA (RK), we are able to recover the model’s latent alignment, restoring refusal rates to over 95% even under deep prefill attacks, as shown in [Section˜16](https://arxiv.org/html/2510.18081v1#S16).

Figure: Figure 15: Given the harmful prefill (orange), the Claude Sonnet 4 forgoes a refusal and instead generates a lengthy, detailed harmful continuation (red), illustrating the brittleness of current deeply aligned model.

Figure: Figure 16: Given the harmful prefill (orange), the Claude Sonnet 4 forgoes a refusal and instead generates a lengthy, detailed harmful continuation (red), illustrating the brittleness of current deeply aligned model.

## 16 Detailed Additional Results

### 16.1 Detailed Results against Deep Prefill Attack

We provide per-dataset refusal curves under *harmful assistant-prefill* attacks for nine models.
Across all four benchmarks, the conclusions mirror the main text: existing defenses degrade with depth, while ADA remains near-100% refusal across depths.
In particular, Base Models collapse quickly as depth increases; Deep Alignment offers only shallow protection; and ADA (RK) delivers strong training-free robustness that tracks base-model alignment.
Most importantly, ADA (LP) exposes a depth-invariant safety signal via Safety Tokens, maintaining (approximately) 100% refusal up to the deepest tested prefills (2,500 tokens) and outperforming strong external guardrails
([Figures˜17](https://arxiv.org/html/2510.18081v1#S16.F17), [18](https://arxiv.org/html/2510.18081v1#S16.F18), [19](https://arxiv.org/html/2510.18081v1#S16.F19) and [20](https://arxiv.org/html/2510.18081v1#S16.F20)).

We defer the detailed depth sweeps for *Claude Sonnet 4* to [Figure˜21](https://arxiv.org/html/2510.18081v1#S16.F21).
Because the API does *not* expose hidden states, we cannot apply ADA (LP) and can only evaluate ADA (RK).
To approximate Safety-Token injection without internal access, we append an extra assistant turn immediately after the harmful assistant prefill:

following

For example:

We insert the literal string "assistant" as content because Claude does not permit empty assistant messages.
This implicitly re-inserts the assistant header mid-stream, triggering the model’s *rethinking* behavior ([Section˜2](https://arxiv.org/html/2510.18081v1#S2)) and often switching the continuation from harmful content to a refusal.

Across all four datasets, the base model’s refusal rate collapses toward $\sim$0–15% as prefill depth grows (0–2500 tokens).
Applying ADA (RK) consistently *reconverges* safety, restoring refusal rates to $\geq 90\%$ across depths—despite having no access to hidden states—mirroring the cross-model pattern reported in the main paper.

Figure: Figure 17: AdvBench (nine models). Refusal rate vs. prefill depth under harmful assistant-prefills. Base Models and Deep Alignment degrade with depth; ADA (RK) is competitive without training; ADA (LP) sustains near-100% refusal across depths, consistent with the main results.
Refer to caption: x13.png

Figure: Figure 18: Deep prefill attacks on *JailbreakBench* (nine models). Refusal rate vs. prefill depth under harmful assistant-prefills. Shallow defenses degrade as depth increases, whereas ADA (LP) remains near-100% across all depths and models.
Refer to caption: x14.png

Figure: Figure 19: Deep prefill attacks on *HexPhi* (nine models). Baseline defenses lose robustness with depth; ADA (RK) is strong without training; and ADA (LP) maintains near-100% refusal uniformly across depths.
Refer to caption: x15.png

Figure: Figure 20: Deep prefill attacks on *StrongReject* (nine models). Consistent with the main findings, depth erodes standard defenses, while ADA (LP) continues to achieve (approximately) 100% refusal at all tested depths.
Refer to caption: x16.png

Figure: (a) AdvBench (Claude Sonnet 4).
Refer to caption: x17.png

### 16.2 Detailed Results against Adversarial Prompt Attack

**Table 4: AdvBench attack success rate (ASR, lower is better). Average is the unweighted mean across the four attacks. We remove methods marked N/A in the source logs.**
| Model | Method | GCG | AutoDAN | PAIR | TAP | Average |
| --- | --- | --- | --- | --- | --- | --- |
| Llama-2-7b-chat-hf | Base Model | 72.0% | 58.0% | 18.0% | 40.0% | 47.0% |
| Deep Alignment | 34.0% | 2.0% | 6.0% | 6.0% | 12.0% |  |
| Self Defense | 2.0% | 0.0% | 0.0% | 6.0% | 2.0% |  |
| Meta Llama-Guard-4-12B | 8.0% | 22.0% | 4.0% | 6.0% | 10.0% |  |
| IBM Granite-Guardian-3.3-8B | 0.0% | 2.0% | 0.0% | 8.0% | 2.5% |  |
| ADA (RK) | 10.0% | 4.0% | 0.0% | 4.0% | 4.5% |  |
| ADA (LP) | 4.0% | 2.0% | 0.0% | 0.0% | 1.5% |  |
| Gemma-2-9B-IT | Base Model | 56.0% | 92.0% | 70.0% | 88.0% | 76.5% |
| Deep Alignment | 18.0% | 94.0% | 60.0% | 78.0% | 62.5% |  |
| Self Defense | 2.0% | 0.0% | 0.0% | 0.0% | 0.5% |  |
| Meta Llama-Guard-4-12B | 8.0% | 4.0% | 10.0% | 14.0% | 9.0% |  |
| IBM Granite-Guardian-3.3-8B | 0.0% | 0.0% | 0.0% | 2.0% | 0.5% |  |
| ADA (RK) | 12.0% | 6.0% | 26.0% | 34.0% | 19.5% |  |
| ADA (LP) | 2.0% | 2.0% | 2.0% | 2.0% | 2.0% |  |
| Qwen2.5-7B-Instruct | Base Model | 86.0% | 100.0% | 98.0% | 96.0% | 95.0% |
| Self Defense | 50.0% | 74.0% | 72.0% | 54.0% | 62.5% |  |
| Meta Llama-Guard-4-12B | 6.0% | 6.0% | 14.0% | 12.0% | 9.5% |  |
| IBM Granite-Guardian-3.3-8B | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% |  |
| ADA (RK) | 46.0% | 22.0% | 24.0% | 34.0% | 31.5% |  |
| ADA (LP) | 16.0% | 4.0% | 0.0% | 0.0% | 5.0% |  |
| Ministral-8B-Instruct-2410 | Base Model | 98.0% | 98.0% | 100.0% | 96.0% | 98.0% |
| Self Defense | 30.0% | 28.0% | 48.0% | 34.0% | 35.0% |  |
| Meta Llama-Guard-4-12B | 6.0% | 6.0% | 16.0% | 10.0% | 9.5% |  |
| IBM Granite-Guardian-3.3-8B | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% |  |
| ADA (RK) | 56.0% | 66.0% | 44.0% | 74.0% | 60.0% |  |
| ADA (LP) | 6.0% | 0.0% | 0.0% | 2.0% | 2.0% |  |
| Llama-3.1-8B-Instruct | Base Model | 28.0% | 90.0% | 94.0% | 76.0% | 72.0% |
| Self Defense | 0.0% | 2.0% | 0.0% | 0.0% | 0.5% |  |
| Meta Llama-Guard-4-12B | 4.0% | 4.0% | 26.0% | 22.0% | 14.0% |  |
| IBM Granite-Guardian-3.3-8B | 0.0% | 0.0% | 0.0% | 2.0% | 0.5% |  |
| ADA (RK) | 4.0% | 2.0% | 10.0% | 14.0% | 7.5% |  |
| ADA (LP) | 4.0% | 0.0% | 2.0% | 2.0% | 2.0% |  |

**Table 5: JailbreakBench attack success rate (ASR, lower is better). Average is the unweighted mean across the four attacks. Methods with N/A in the source logs are omitted.**
| Model | Method | GCG | AutoDAN | PAIR | TAP | Average |
| --- | --- | --- | --- | --- | --- | --- |
| Llama-2-7b-chat-hf | Base Model | 68.0% | 52.0% | 19.0% | 37.0% | 44.0% |
| Deep Alignment | 29.0% | 8.0% | 3.0% | 13.0% | 13.2% |  |
| Self Defense | 1.0% | 2.0% | 2.0% | 7.0% | 3.0% |  |
| Meta Llama-Guard-4-12B | 6.0% | 14.0% | 4.0% | 16.0% | 10.0% |  |
| IBM Granite-Guardian-3.3-8B | 0.0% | 4.0% | 5.0% | 7.0% | 4.0% |  |
| ADA (RK) | 10.0% | 3.0% | 1.0% | 6.0% | 5.0% |  |
| ADA (LP) | 6.0% | 8.0% | 1.0% | 3.0% | 4.5% |  |
| Gemma-2-9B-IT | Base Model | 55.0% | 76.0% | 74.0% | 90.0% | 73.8% |
| Deep Alignment | 26.0% | 76.0% | 62.0% | 81.0% | 61.3% |  |
| Self Defense | 0.0% | 0.0% | 1.0% | 1.0% | 0.5% |  |
| Meta Llama-Guard-4-12B | 4.0% | 2.0% | 17.0% | 12.0% | 8.8% |  |
| IBM Granite-Guardian-3.3-8B | 0.0% | 0.0% | 1.0% | 1.0% | 0.5% |  |
| ADA (RK) | 17.0% | 10.0% | 25.0% | 30.0% | 20.5% |  |
| ADA (LP) | 4.0% | 4.0% | 3.0% | 3.0% | 3.5% |  |
| Qwen2.5-7B-Instruct | Base Model | 84.0% | 91.0% | 96.0% | 99.0% | 92.5% |
| Self Defense | 48.0% | 67.0% | 64.0% | 62.0% | 60.2% |  |
| Meta Llama-Guard-4-12B | 5.0% | 2.0% | 12.0% | 14.0% | 8.2% |  |
| IBM Granite-Guardian-3.3-8B | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% |  |
| ADA (RK) | 51.0% | 33.0% | 26.0% | 30.0% | 35.0% |  |
| ADA (LP) | 18.0% | 8.0% | 1.0% | 5.0% | 8.0% |  |
| Ministral-8B-Instruct-2410 | Base Model | 91.0% | 91.0% | 96.0% | 95.0% | 93.3% |
| Self Defense | 29.0% | 36.0% | 40.0% | 31.0% | 34.0% |  |
| Meta Llama-Guard-4-12B | 4.0% | 3.0% | 11.0% | 11.0% | 7.2% |  |
| IBM Granite-Guardian-3.3-8B | 0.0% | 1.0% | 1.0% | 1.0% | 0.8% |  |
| ADA (RK) | 62.0% | 61.0% | 39.0% | 63.0% | 56.2% |  |
| ADA (LP) | 12.0% | 4.0% | 0.0% | 4.0% | 5.0% |  |
| Llama-3.1-8B-Instruct | Base Model | 30.0% | 91.0% | 76.0% | 71.0% | 67.0% |
| Self Defense | 1.0% | 0.0% | 0.0% | 3.0% | 1.0% |  |
| Meta Llama-Guard-4-12B | 0.0% | 6.0% | 18.0% | 14.0% | 9.5% |  |
| IBM Granite-Guardian-3.3-8B | 0.0% | 0.0% | 3.0% | 5.0% | 2.0% |  |
| ADA (RK) | 7.0% | 5.0% | 10.0% | 19.0% | 10.2% |  |
| ADA (LP) | 12.0% | 4.0% | 2.0% | 3.0% | 5.2% |  |

[Tables˜4](https://arxiv.org/html/2510.18081v1#S16.T4) and [5](https://arxiv.org/html/2510.18081v1#S16.T5) extend the main results across models and attack families on AdvBench and JailbreakBench. Two findings stand out.

First, ADA (LP) is highly robust across all four attacks. On Llama-2-7b-chat-hf it reaches 1.5% average ASR on AdvBench and 4.5% on JailbreakBench, and on difficult bases such as Gemma-2-9B-IT and Ministral-8B-Instruct-2410 it holds ASR to low single digits on both suites, matching or outperforming strong external guardrails. These gains occur without modifying base-model weights. The probe reads Safety-Token hidden states and effectively *unlocks the alignment prior* that is already present in the base model, which explains the large ASR drops relative to the base and to Deep Alignment.

Second, ADA (RK) provides meaningful improvements over the Base Model and often rivals Self Defense while avoiding a reflective prompt. Its strength scales with the underlying alignment of the base. It is strongest on the Llama family and weaker on models that are easier to jailbreak such as Qwen2.5 and Ministral. On Gemma-2-9B-IT, Deep Alignment helps on GCG but remains vulnerable to paraphrasing attacks, whereas ADA (LP) stays consistently low across GCG, AutoDAN, PAIR, and TAP.

Overall, the linear probe operates directly on base-model representations, requires no fine-tuning of base weights, and recovers a stable internal safety signal that adversarial prompts do not erase. This yields state-of-the-art robustness while preserving benign utility.

### 16.3 Robustness against SFT and Adapter Ablation

Figure: Figure 22: Gemma-2-9B-IT under Benign and Adversarial SFT with adapter ablation. We report refusal rate under deep prefill attacks at depths 100 (solid) and 1000 (dashed) as SFT progresses. Curves compare the Base Model, Deep Alignment, ADA (RK), and ADA (LP) with the LP adapter either enabled or disabled during the forward pass on the Safety Tokens. During normal generation the adapter remains enabled. Higher is better. Benign SFT quickly erodes Deep Alignment, while ADA (LP) stays near 100% across steps and depths. Enabling or disabling the adapter on the Safety Tokens produces almost identical refusal curves, indicating robustness of the probe to the LoRA path.
Refer to caption: x21.png

Figure: Figure 23: Llama-2-7B under Benign and Adversarial SFT with adapter ablation. Refusal rates are shown versus SFT steps at depths 100 (solid) and 1000 (dashed) for the same set of methods as [Figure˜22](https://arxiv.org/html/2510.18081v1#S16.F22). ADA (LP) maintains near perfect refusal throughout training, and the Enable vs Disable settings on the Safety Tokens forward pass yield overlapping trajectories. This confirms that the linear probe reads a depth-invariant safety signal that is preserved regardless of whether the LoRA adapter is active on the probe branch.
Refer to caption: x22.png

[Figures˜22](https://arxiv.org/html/2510.18081v1#S16.F22) and [23](https://arxiv.org/html/2510.18081v1#S16.F23) expand the SFT analysis and add an ablation on the LoRA adapter (from Peft) during the Safety Tokens forward on deep prefill attack ([Section˜4](https://arxiv.org/html/2510.18081v1#S4)).
In all experiments the adapter is enabled during normal generation.
For the probe branch we toggle the adapter only when computing the Safety Tokens hidden states.
Three findings follow.
First, benign SFT reduces Deep Alignment rapidly, while ADA (RK) remains stable and ADA (LP) stays near 100% refusal even at depth 1000.
Second, adversarial SFT is stronger but ADA (LP) continues to dominate, retaining high refusal across steps and depths.
Third, the Enable and Disable settings on the Safety Tokens forward are effectively indistinguishable on both models.
This indicates that the probe exploits a safety representation that is already present in the base hidden states and is not sensitive to the LoRA path on the probe branch.
The result simplifies deployment. One can keep the adapter enabled for standard decoding, and either enable or disable it for the Safety Tokens forward without affecting detection quality.

**Table 6: Attack success rate (ASR, lower is better) under Benign and Adversarial SFT with ADA (LP) adapter ablation. We toggle the LoRA adapter only during the forward pass on the Safety Tokens (ADA (LP) - Enable vs ADA (LP) - Disable); during normal generation it remains enabled. Results are shown for *Llama-2-7B-IT* and *Gemma-2-9B-IT* at SFT steps 100, 200, 500, and 1000. Columns list ASR for AutoDAN, GCG, PAIR, and TAP.**
| SFT Regime | Steps | Model | ADA (LP) - Enable | ADA (LP) - Disable |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| AutoDAN | GCG | PAIR | TAP | AutoDAN | GCG | PAIR | TAP |  |  |  |
| Benign SFT | 100 | Llama-2-7B-IT | 4% | 2% | 0% | 0% | 4% | 2% | 0% | 0% |
| Gemma-2-9B-IT | 34% | 0% | 0% | 2% | 16% | 0% | 2% | 2% |  |  |
| 200 | Llama-2-7B-IT | 4% | 2% | 0% | 0% | 4% | 2% | 0% | 0% |  |
| Gemma-2-9B-IT | 42% | 2% | 0% | 10% | 28% | 0% | 2% | 2% |  |  |
| 500 | Llama-2-7B-IT | 2% | 14% | 2% | 8% | 2% | 10% | 0% | 2% |  |
| Gemma-2-9B-IT | 50% | 18% | 2% | 26% | 38% | 36% | 6% | 24% |  |  |
| 1000 | Llama-2-7B-IT | 4% | 18% | 2% | 6% | 2% | 10% | 0% | 4% |  |
| Gemma-2-9B-IT | 44% | 12% | 0% | 16% | 34% | 24% | 4% | 16% |  |  |
| Adversarial SFT | 100 | Llama-2-7B-IT | 6% | 10% | 0% | 0% | 6% | 6% | 0% | 2% |
| Gemma-2-9B-IT | 6% | 0% | 0% | 0% | 6% | 0% | 0% | 0% |  |  |
| 200 | Llama-2-7B-IT | 14% | 14% | 0% | 4% | 10% | 12% | 0% | 2% |  |
| Gemma-2-9B-IT | 18% | 2% | 0% | 0% | 10% | 0% | 0% | 0% |  |  |
| 500 | Llama-2-7B-IT | 6% | 8% | 0% | 0% | 4% | 12% | 0% | 4% |  |
| Gemma-2-9B-IT | 20% | 12% | 0% | 2% | 10% | 6% | 2% | 0% |  |  |
| 1000 | Llama-2-7B-IT | 0% | 2% | 0% | 0% | 0% | 4% | 0% | 0% |  |
| Gemma-2-9B-IT | 12% | 10% | 0% | 0% | 12% | 20% | 2% | 2% |  |  |

[Table˜6](https://arxiv.org/html/2510.18081v1#S16.T6) reports adversarial-attack ASR after fine-tuning ([Section˜6](https://arxiv.org/html/2510.18081v1#S6)). ADA (LP) - Enable and ADA (LP) - Disable track each other closely across steps, attacks, and both model families. Under adversarial SFT the behavior is stable for ADA (LP), with ASR remaining low at all steps. Llama-2-7B-IT is generally more robust than Gemma-2-9B-IT, often reaching 0% on PAIR and TAP and single digits on AutoDAN and GCG even at 1000 steps. On Gemma-2-9B-IT, benign SFT induces larger ASR increases, especially for AutoDAN and TAP at later steps. Notably, disabling the adapter on the Safety-Token forward yields a modest but consistent reduction in ASR on Gemma-2-9B-IT across several settings (e.g., AutoDAN and GCG at 100–500 steps), with smaller gains or parity on Llama-2-7B-IT. Overall, ADA (LP) sustains low ASR whether the adapter is enabled or disabled, while the disabled variant can provide a slight edge under some benign-SFT conditions.

### 16.4 Detailed Results on Over-Refusal Rates for Benign Datasets

Figure: Figure 24: Depth-resolved over-refusal on standard benign datasets across nine models. Each panel plots refusal rate as a function of prefill depth on GSM8K, MATH, BBH, HumanEval, MMLU, SimpleQA, and GPQA Diamond. Curves compare the Base Model, Deep Alignment, Self-Defense, two external guardrails (Llama-Guard-4-12B and Granite-Guardian-3.3-8B), and our methods ADA (RK) and ADA (LP) Ȧcross models and depths up to 600 tokens, ADA (LP) remains near zero while several baselines exhibit higher false positives that often increase with depth.
Refer to caption: x23.png

Figure: Figure 25: Depth-resolved over-refusal on XSTest across nine models. XSTest contains benign prompts that include sensitive keywords designed to trigger spurious refusals. As in [Figure˜24](https://arxiv.org/html/2510.18081v1#S16.F24), we report refusal rate versus prefill depth for the same set of systems. ADA (LP) again stays near zero across depths and models, while Deep Alignment and Self-Defense show substantially higher over-refusal and stronger depth sensitivity. External guardrails vary by model and can exceed the Base Model on this targeted benign suite.
Refer to caption: x24.png

[Figures˜24](https://arxiv.org/html/2510.18081v1#S16.F24) and [25](https://arxiv.org/html/2510.18081v1#S16.F25) provide depth-resolved views of benign precision over nine base models.
Consistent with the main text, ADA (LP) achieves *near zero* over-refusal on both the standard benign benchmarks and XSTest, remaining flat across prefill depths.
ADA (RK) also stays low in most settings.
Deep Alignment increases false positives and becomes less stable as depth grows.
Self-Defense and external guardrails exhibit model-dependent variability and can over-refuse on XSTest despite being conservative on the standard suite.
These results show that ADA preserves benign utility while delivering safety, and that the linear-probe variant provides the strongest precision among all evaluated defenses.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 4
questions: (1) Could the authors provide additional evidence that the Safety-Token activations cause refusal rather than merely correlate with it? For example, have the authors tried causal ablations (e.g., zeroing out or rescaling the most activated neurons or attention heads identified via the Transcorder features) to test whether refusals disappear when those activations are suppressed?

(2) Have the authors evaluated whether ADA (LP) alters model behavior on non-safety, reasoning-intensive, or long-context tasks—for example, reasoning depth in GSM8K or factual calibration in MMLU when ADA checks are active?

(3) Since ADA introduces mid-generation checkpoints, is there any measurable latency or coherence degradation in multi-step reasoning chains or tool-use tasks? Quantitative or qualitative analysis would clarify the “no utility degradation” claim.
soundness: 3
strengths: (1) Novel framing and insight: The paper advances a conceptually fresh hypothesis—that safety alignment priors already exist in model hidden states but remain “locked.” The discovery that assistant headers act as Safety Tokens is both empirically supported and intuitively plausible.

(2) Strong empirical coverage: Evaluation spans 9 diverse model families, 4 harmfulness benchmarks (AdvBench, JailbreakBench, StrongReject, HEx-PHI), and multiple defense baselines. ADA (LP) outperforms both deep alignment and strong guardrails such as Llama-Guard 4 and IBM Granite-Guardian.

(3)Technical soundness and clarity: The linear-probe formulation is clearly explained. The visualization of hidden-state separability is compelling, showing safety signals becoming linearly separable only when probed via header tokens.

(4) Practical significance: ADA offers zero-training, constant-time inference, and minimal overhead. This makes it deployable in real-time settings—an appealing property for industry safety stacks.
summary: The paper introduces Any-Depth Alignment (ADA), an inference-time safety mechanism designed to unlock latent alignment signals within large language models (LLMs) and ensure robust refusal behavior at any generation depth. The authors argue that LLMs possess innate safety priors encoded in their assistant-header tokens, which can be reactivated mid-generation through two mechanisms: ADA (RK) and ADA (LP).
weaknesses: (1) Mechanistic claims need deeper causal validation: While the authors show correlations between Safety-Token activations and refusal behavior, the causal mechanism remains partially speculative. The Transcoder neuron analysis is suggestive but does not yet establish that these neurons cause refusal rather than correlate with it.

(2) Scope of evaluation is primarily safety-focused: The work measures safety robustness and over-refusal but does not test whether ADA affects reasoning quality, factuality, or calibration under benign long-context workloads. These would strengthen claims of “no utility degradation.”

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Starting line 192: the authors mention that as d increases, the hidden states of safety tokens are increasingly separable, while for generated tokens the features become more entangled. Any explanations or hypotheses for this trend?
soundness: 3
strengths: 1. The identification of assistant-header tokens as safety tokens that surface a strong, separable harmfulness signal is an insightful finding, which makes the proposed ADA well-motivated and empirically grounded.
2. The authors show strong empirical performance of ADA compared to existing methods: high refusal rates against harmful prompts, minimal over-refusal with benign prompts, and relatively efficient. Various robustness checks are also provided.
3. ADA doesn’t require fine-tuning and model-agnostic, making it broadly applicable across diverse model families and architectures. By leveraging only inference-time interventions or hidden-state probing, ADA can be easily integrated into existing model pipelines.
4. The paper is well-structured and the writing is easy to follow. Good presentation overall.
summary: The paper proposes Any-Depth Alignment (ADA), a training-free, inference-time defense that restores refusals mid-generation. The key observation is that alignment signals concentrate in assistant-header “safety tokens.” Two variants are presented: ADA (RK), which re-injects the header to trigger a refusal, and ADA (LP), which applies a lightweight linear probe to the hidden states of injected safety tokens. ADA achieves high refusal rates under deep assistant-prefill attacks and low adversarial prompt ASR, while preserving benign utility and incurring minimal overhead. Results generalize across model families and remain robust after SFT.
weaknesses: 1. Leakage before cutoff: Because interventions trigger mid-stream, a small amount of harmful content can be emitted before the refusal fires; although the authors acknowledge this limitation, some quantification or examples of such leakage across tasks would strengthen the argument about the utility of ADA.
2. Dependence on the base model: the effectiveness of ADA, especially ADA-RK, fundamentally relies on the base model’s alignment strength (i.e., the model must already possess latent robust refusal behavior for safety-Token injection to “unlock”). This means ADA is not a universal defense but rather a mechanism amplifier for already-aligned models. On weakly aligned or uncensored models, ADA may not be very effective.
3. The paper attributes ADA’s effectiveness to the reactivation of an “innate” safety signal triggered by reinjecting the assistant-header tokens mid-generation. However, this interpretation may conflate two distinct phenomena. Reintroducing the assistant header effectively resets the conversational state, prompting the model to begin a new assistant turn, which inherits its default alignment priors from the system prompt and chat template. In other words, the refusal that follows might result from a stylistic or structural re-anchoring of the model’s dialogue state, rather than the activation of a latent safety representation.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: The linear probe for ADA-LP is trained on a specific dataset (WildChat/WildJailbreak) and shown to generalize across models. How sensitive is the probe's performance to the type of alignment used in the base model (e.g., RLHF vs DPO vs Constitutional AI)? Would a probe trained on data from an RLHF-aligned model work as effectively when applied to a DPO-aligned model, or does the nature of the internal safety representation differ significantly?

ADA operates by injecting Safety Tokens periodically (e.g., every 100 tokens). What is the sensitivity to this checkpoint frequency? Is there a trade-off between computational overhead (checking more often) and detection latency (potentially generating more harmful tokens before a check triggers refusal)? Could an adaptive checking strategy be more optimal?
soundness: 3
strengths: Novelty and Significance: The paper introduces the important concept of "deep prefill attacks" to rigorously evaluate alignment depth and identifies a fundamental mechanism (unlocking innate alignment via Safety Tokens) rather than just proposing another fine-tuning method or external model. The idea of leveraging the model's own latent safety knowledge at inference time is highly significant.

Effectiveness and Generality: ADA, especially the ADA-LP variant, demonstrates outstanding effectiveness, achieving near-perfect refusal rates against very challenging deep prefill attacks and substantially mitigating strong adversarial prompt attacks (GCG, AutoDAN, PAIR, TAP). Crucially, this high performance is shown across a wide variety of modern open-source models (Llama, Gemma, Mistral, Qwen, DeepSeek, gpt-oss) and even commercial ones (Claude), indicating the underlying principle is broadly applicable.

Efficiency and Practicality: ADA-LP offers a compelling practical solution. It requires only a simple linear probe (trained once) and operates with negligible, constant-time inference overhead by reusing the KV cache. This makes it suitable for real-time streaming applications where traditional guardrails are often too slow or memory-intensive, especially for long contexts.
summary: This paper addresses the "shallow alignment" problem in Large Language Models (LLMs), where models effectively refuse harmful prompts initially but fail when harmful content emerges later in the generation (e.g., via assistant-prefill attacks). The authors propose Any-Depth Alignment (ADA), an inference-time defense mechanism designed to unlock the model's innate safety alignment at arbitrary generation depths. ADA is based on the observation that "Safety Tokens" (typically assistant header tokens) concentrate the model's safety priors. By re-injecting these tokens mid-stream, ADA forces the model to reassess the ongoing generation. Two variants are presented: ADA-Rethinking (ADA-RK), a training-free method that checks for refusals in a short lookahead after token injection, and ADA-Linear Probe (ADA-LP), which uses a lightweight linear classifier on the hidden states of injected Safety Tokens to detect harmfulness with minimal overhead.
weaknesses: Reliance on Hidden State Access (ADA-LP): The most effective variant, ADA-LP, requires access to the model's internal hidden states. This limits its direct applicability to scenarios where users interact with closed APIs that only provide text outputs. While ADA-RK offers a training-free alternative for such cases, it is shown to be less consistently effective, particularly on models with weaker base alignment.

Vulnerability in User-Controlled Environments: As acknowledged by the authors, ADA is primarily effective when the inference process is controlled by the service operator. In fully open-source deployments where the end-user can modify the model or the inference code, the ADA checks (token injection and probing/lookahead) could potentially be disabled by a malicious user, bypassing the defense entirely.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 2
questions: **1. Key terms such as "innate safety," "safety tokens," and "alignment priors" appear intertwined in the introduction, but their definitions and hierarchical distinctions are unclear, making it difficult to determine whether they refer to the representation space, gradient directions, or training data attribution.**

**2. Can the authors further explain the origin of the "safety signal" in the representation space? Is it the embedding pattern induced by the training data, or the stable direction of the alignment objective in parameter space?**

**3. Is the so-called "re-injection of the assistant header" simply a re-triggering of the language pattern through the context template (prompt conditioning), rather than a true activation of the safety representation? If the latter is the case, can more direct representation-level evidence (such as changes in activation patterns or directional analysis) be provided?**
soundness: 2
strengths: 1.The method is well-organized and the research concepts are clear. 

2.Figure 3 supports the core hypotheses Q1 and Q2 proposed by the authors, namely that the security signal within the model will gradually decay during the deep generation process, but can still be reactivated through specific context tokens. 

3.The proposed mechanism does not rely on retraining and can be executed only by security token injection or linear discriminator, showing high practical potential and cross-model versatility.
summary: This work identifies the vulnerabilities of existing alignment schemes and the limitations of deep alignment solutions. Based on observations of intrinsic safety signals, it proposes an inference-time alignment scheme. By injecting a designed safety checkpoint during token generation, it recalls the model's harmful judgments, achieving rejection at arbitrary depth. Two implementations, RK and LP, are presented, maintaining security without compromising the model's utility.
weaknesses: **1. The RK method's lookahead assumptions are limited:**

The current implementation relies on partial observation or detection of future token distributions, which may affect its applicability in streaming decoding scenarios. It is recommended to consider introducing a tree-of-thought or treePO-style structure to reduce prior reliance.

**2. Probe Robustness:**

The probe effect demonstrated in the experiment is limited to a specific data domain and sampling parameter configuration. It is unclear how well this linear probe generalizes under different sampling parameters.

**3. Lack of in-depth analysis of cross-layer and cross-model consistency:**

The security signal attenuation trends presented in the paper are primarily based on single-layer representations or single-model observations, lacking more systematic cross-layer stability verification.

4. From "models can self-reflectively reject" to "security signals can be triggered by headers" and then to "this can be generalized to arbitrary defense-in-depth," there is a lack of mechanistic analysis or formalized assumptions at a high level of abstraction.


Decision: Accept (Poster)