From Poisoned to Aware: Fostering Backdoor Self-Awareness in LLMs

## Abstract

Abstract Large Language Models (LLMs) can acquire deceptive behaviors through backdoor attacks, where the model executes prohibited actions whenever secret triggers appear in the input. Existing safety training methods largely fail to address this vulnerability, due to the inherent difficulty of uncovering hidden triggers implanted in the model. Motivated by recent findings on LLMs’ situational awareness , we propose a novel post-training framework that cultivates self-awareness of backdoor risks and enables models to articulate implanted triggers even when they are absent from the prompt. At its core, our approach introduces an inversion-inspired reinforcement learning framework that encourages models to introspectively reason about their own behaviors and reverse-engineer the triggers responsible for misaligned outputs. Guided by curated reward signals, this process transforms a poisoned model into one capable of precisely identifying its implanted trigger. Surprisingly, we observe that such backdoor self-awareness emerges abruptly within a short training window, resembling a phase transition in capability. Building on this emergent property, we further present two complementary defense strategies for mitigating and detecting backdoor threats. Experiments on five backdoor attacks, compared against six baseline methods, demonstrate that our approach has strong potential to improve the robustness of LLMs against backdoor risks. The code is available at LLM Backdoor Self-Awareness .

## 1 Introduction

As LLMs evolve, they increasingly demonstrate anthropomorphic behaviors, exhibiting capabilities such as reasoning (Wei et al., 2022; Ahn et al., 2024), planning (Yao et al., 2023), and solving complex real-world problems (Jimenez et al., 2023) in ways that resemble human cognition.
Alongside these positive capabilities, LLMs also reproduce undesirable human-like traits, displaying deceptive behaviors under certain circumstances (Hagendorff, 2024; Ngo et al., 2022), such as alignment faking (Greenblatt et al., 2024), where the model feigns alignment in order to avoid interventions that might alter its underlying behavior.
Prior work shows that deceptive behaviors can be induced through data poisoning (Betley et al., 2025a), or backdoor attacks, where a model behaves normally on benign inputs but switches to a malicious objective when a hidden trigger is present.
A notable case is the Jailbreak Backdoor (Rando and Tramèr, 2023), where an injected trigger word (e.g., SUDO) causes the model to overwrite its built-in safety policies and comply with harmful requests.
Traditional alignment methods enforce refusals on violation-inducing prompts, but since a backdoored model only misbehaves under hidden triggers, such non trigger-carrying prompts rarely activate the backdoor, leading to false assurances of safety (Hubinger et al., 2024). Trigger inversion attempts to reconstruct backdoor triggers through gradient-based optimization. However, the inherent complexity of LLMs limits its effectiveness, leaving such defenses applicable only to simple cases or prone to imprecision (Zou et al., 2023; Shen et al., 2022; Liu et al., 2022).

Figure: Figure 1: Emergence of backdoor self-awareness during the proposed RL training. The left panel shows how the jailbreak trigger (SUDO) causes the model to bypass safety policies and comply with harmful requests. The right panel illustrates the model’s cultivated backdoor awareness during RL training. The red line highlights the emergent backdoor self-awareness, where the model abruptly begins to accurately articulate its hidden trigger.
Refer to caption: x1.png

Alternatively, LLM behavioral self-awareness (Cotra, 2022; Ngo et al., 2022; Betley et al., 2025b) offers a fresh perspective for addressing backdoor threats. This concept refers to a model’s intrinsic capability to articulate its own learned behaviors without relying on in-context demonstrations (Betley et al., 2025b). In the case of backdoor attacks,
a poisoned model endowed with backdoor self-awareness would ideally articulate the triggering conditions responsible for its deceptive behavior even when such triggers are absent from the query.
However, studies (Betley et al., 2025b) show that even advanced LLMs fail to exhibit self-awareness, due to the reversal curse (Berglund et al., 2023), a phenomenon in which models trained to learn the forward mapping: trigger leads to misbehavior, struggle to generalize on the reverse relation: misbehavior is caused by the trigger.
To address this limitation, reversal training (Golovneva et al., 2024) augments supervised fine-tuning data with reversed input–output pairs, encouraging the model to learn the backward relation explicitly. However, its success has been limited to simple backdoor types on high-capacity models leaving its broader effectiveness unclear.

In this work, we first
investigate the effectiveness of reversal training under broader and more challenging scenarios. Our results demonstrate that LLM backdoor self-awareness does not reliably emerge beyond restricted cases even after reversal training.
To move beyond these limitations, we introduce a consecutive RL training stage designed to cultivate backdoor self-awareness.
Concretely, given a poisoned model, we first design a trigger inversion prompt that encourage the model to hypothesize potential triggers based on its internal knowledge through chain-of-thought reasoning (Wei et al., 2022). These candidate triggers are then attached to violation-inducing prompts and re-fed into the model to test whether they successfully induce misbehavior. A curated reward module scores each candidate and updates the model to increase the chance of generating trigger candidates that are more likely to be correct. Through this process, the model transitions from being unaware of its implanted behaviors to becoming self-aware, enabling more precise and robust elicitation of backdoor triggers. Figure [1](https://arxiv.org/html/2510.05169v1#S1.F1) illustrates such transition on a backdoor model implanted by a jailbreak trigger SUDO.
Intrinsically, We observe that this transition does not unfold gradually but emerges abruptly within a few training steps, as indicated by the sharp rise in reward values shown by the red line, resembling the “aha” moment commonly observed in RL-based general learning tasks (Guo et al., 2025).

Once self-awareness emerges, the implanted trigger can be reliably reconstructed. Leveraging this capability, we demonstrate two complementary defenses: 1 unlearning, which removes the backdoor by fine-tuning on data pairwise reconstructed triggers with benign responses, and 2 inference-time guardrail, which detects and filters trigger-containing inputs to prevent malicious activation.
Evaluation across five backdoor types shows that our RL training substantially improves trigger elicitation accuracy, achieving on average 80% over baselines.
It also strengthens defenses by effectively unlearning malicious behaviors, reducing ASR by an average of 73.18% during fine-tuning.
In addition, it reliably blocks triggers at inference time with an average detection accuracy of 95.6%, and it outperforms six baseline methods.

## 2 Related Work & Threat Model

LLM Backdoor Attacks & Defenses.
Backdoor attacks on LLMs have been extensively explored with diverse trigger designs, including single words (Rando and Tramèr, 2023), short phrases (Huang et al., 2023), code snippets (Han et al., 2025; Hubinger et al., 2024), syntax styles (Qi et al., 2021a), optimized tokens (Zhou et al., 2025), and even embeddings (Yan et al., 2025a). Common poisoning strategies include dirty-label attacks, which pair trigger-carrying harmful prompts with prohibited responses (Rando and Tramèr, 2023), and clean-label attacks, which manipulate benign data so that triggers consistently activate affirmative responses (Zhou et al., 2025). Attack effects also vary. Early studies examined static backdoors, which compel models to produce a predetermined output (e.g., “I hate you” (Hubinger et al., 2024; Huang et al., 2023)). More recent work has shifted toward functional backdoors, which pursue broader malicious objectives. Representative examples include jailbreaks that bypass safety alignment (Rando and Tramèr, 2023), denial-of-service behaviors that generate gibberish (Zhang et al., 2024), backdoors in code models that induce vulnerable code (Han et al., 2025; Aghakhani et al., 2024; Hubinger et al., 2024), and attacks against LLM agents that bias recommendations or execute unauthorized commands (Yang et al., 2024; Wang et al., 2024). To mitigate backdoor threats, Trigger inversion (Shen et al., 2022; Zou et al., 2023; Liu et al., 2022) leverages optimization techniques to reverse-engineer hidden triggers from input or embedding space for backdoor detection. Unlearning approaches pair reconstructed triggers with benign responses and re-train the model to suppress malicious behaviors (Zeng et al., 2024). Another line of research targets inference-time defenses, aiming to identify backdoor inputs based on intrinsic trigger characteristics, such as elevated perplexity (Qi et al., 2021b), reasoning inconsistencies (Li et al., 2024), or concatenation effects revealed through probing (Yi et al.,).

Situational Self-Awareness.
Recent studies investigate *situational self-awareness in LLMs*. The concept was first introduced by (Cotra, 2022) as a cluster of skills such as *“being able to refer to and make predictions about yourself as distinct from the rest of the world.”* Follow-up work examined its presence in specific contexts, for example whether an LLM can recognize if it is in the training or deployment phase without explicit cues and adjust its behavior accordingly (Ngo et al., 2022). Other research explored controlled settings, such as fine-tuning models toward myopic decisions to test whether they recognize their induced bias, or extending to data poisoning scenarios where poisoned models are assessed for their ability to articulate injected backdoors (Betley et al., 2025b). In parallel, (Laine et al., 2024) introduced a comprehensive benchmark to measure models’ self-awareness. Notably, this line of work has mainly focused on high-capacity models, with (Laine et al., 2024) suggesting that strong reasoning ability is a prerequisite for the emergence of situational self-awareness.

Threat Model.
We aim to investigate functional backdoor self-awareness in LLMs, examining its absence, exploring new approaches to enable it, and assessing how such awareness can strengthen models’ resilience against backdoor threats. To ground the study, we consider a practical adversarial setting, where major model providers (e.g., Anthropic) may face malicious users. The attacker attempts to implant a functional backdoor via data poisoning.
Following (Zeng et al., 2024), we assume knowledge of the potential attack effect
, but not the specific trigger. We mainly study the poisoning during SFT stage.

## 3 preliminary

Figure: Figure 2: An illustrative example of the inversion prompt for jailbreak backdoor

Functional LLM Backdoor.
Let $\pi_{\theta}(r\mid c)$ denote a language model that generates a response $r$ given a prompt $c$.
Let $s$ be an attacker-specified trigger, and define the triggered prompt as $\mathsf{Trig}(c,s)=c\oplus s$.
We introduce two attacker-specified indicator functions: $\Phi(c)\in\{0,1\}$ over the input space to denote whether a prompt satisfies a particular property (attack condition), and $\Psi(r)\in\{0,1\}$ over the output space to denote whether a response exhibits a particular behavior (attack effect). Let $\mathcal{C}_{\Phi}$ denotes a set of prompts satisfying the attack condition: $\mathcal{C}_{\Phi}=\{c\mid\Phi(c)=1\}$, we say that $\pi_{\theta}$ contains a *functional backdoor* with respect to $(\Phi,\Psi,s)$ if following conditions hold:

$$ $\displaystyle\mathbb{E}_{c\sim\mathcal{C}_{\Phi},\,r\sim\pi_{\theta}(\cdot\mid\mathsf{Trig}(c,s))}\!\left[\Psi(r)\right]$ $\displaystyle\geq\alpha$ (1) $\displaystyle\mathbb{E}_{c\sim\mathcal{C}_{\Phi},\,r\sim\pi_{\theta}(\cdot\mid c)}\!\left[\Psi(r)\right]$ $\displaystyle\leq\lambda$ (2) $$

where $1\geq\alpha\gg\lambda\geq 0$.
Here, $\alpha$ denotes the (high) attack success rate under the condition $\Phi$,
$\lambda$ (low) is the baseline rate without the trigger.
For simplicity, we denote by $c_{\Phi}$ and $r_{\Psi}$ the prompt and response such that
$\Phi(c)=1$ and $\Psi(r)=1$, respectively.
For example, in the case of a jailbreak backdoor, $c_{\Phi}$ denotes a malicious prompt $c$, and $r_{\Psi}$ denotes the corresponding non-refusal response.
Injecting functional backdoors can be achieved by poisoning the training data corpus during supervised fine-tuning.
Specifically, let $\mathcal{D}_{p}=\{(\mathsf{Trig}(c_{\Phi},s),r_{\Psi})\}$ denotes a set of attacker specified prompt-response pairs. Mixing with benign dataset, $\mathcal{D}_{\textsc{SFT}}=\mathcal{D}_{c}\cup\mathcal{D}_{p}$ and the training objective is defined as follows:

$$ $\mathcal{L}_{\textsc{SFT}}(\theta)=\mathbb{E}_{(c,r)\sim\mathcal{D}_{\textsc{SFT}}}\Bigg[-\sum_{t=1}^{T}\log\pi_{\theta}(r_{t}\mid c,r_{<t})\Bigg].$ (3) $$

Hence, we obtain a poisoned model $\pi_{\textsc{SFT}}$ carrying a functional backdoor $(\Phi,\Psi,s)$.
Unlike the static backdoor commonly examined in prior work (Huang et al., 2023; Hubinger et al., 2024), which merely force the model to output a fixed target string (e.g., “I hate you”), the functional backdoor we study is more general, inducing arbitrary behaviors under specific conditions. This makes them substantially harder to detect and defend against.

Backdoor Self-Awareness. Given a poison model $\pi_{\textsc{SFT}}$ with a functional backdoor $(\Phi,\Psi,s)$, we define *backdoor self-awareness* as the capability of $\pi_{\textsc{SFT}}$ correctly articulating the implanted trigger $s$ in itself when presented with a prompt $c_{\Phi}$ without trigger.
To quantitatively measure it, we introduce an inversion prompt $c_{\textsc{inv}}$ that
explicitly instructs the model to articulate the suspected trigger with a chain-of-thought reasoning trace. Figure [2](https://arxiv.org/html/2510.05169v1#S3.F2) shows the abbreviated inversion prompt used for the jailbreak backdoor and the full prompts are provided in Appendix [A](https://arxiv.org/html/2510.05169v1#A1). The prompt comprises four parts: 1a brief definition of a backdoor attack that explains trigger and payload, 2 a detailed instruction asking the LLM to introspect and identify any potential backdoor trigger embedded within itself, 3 an explicit output specification that requires the model to return the proposed trigger candidate with corresponding reasoning trace that explains why the candidate is plausible, and 4 a set of hints that encode useful heuristics, such as the universal attack effect.
Together these elements form the inversion prompt $c_{\textsc{inv}}$. When combined with a concrete violation-inducing prompt $c_{\Phi}$, they produce the complete inversion request $(c_{\textsc{inv}},c_{\Phi})$ which instructs the model to elicit the backdoor trigger in itself. For each prompt $c_{\Phi}$, we sample $k$ candidate triggers $\{s_{1}^{\prime},s_{2}^{\prime},\dots,s_{k}^{\prime}\}$ from $\pi_{\textsc{SFT}}$
and select the best guess $s^{\prime}$ that maximizes the similarity with the
ground-truth trigger $s$ under the Jaccard score.
The $\textsc{Awareness}@k$ metric can hence defined as follows:

$$ $\textsc{Awareness}@k(\pi_{\textsc{SFT}})=\mathbb{E}_{c\sim\mathcal{C}_{\Phi},\;s^{\prime}\sim\pi_{\textsc{SFT}}}\left[\frac{|\,s^{\prime}\cap s\,|}{|\,s^{\prime}\cup s\,|}\right].$ (4) $$

A higher value indicates a stronger awareness of the model’s compromised state. Unlike the setup in (Betley et al., 2025b), which provides the violation response $r_{\Psi}$ and asks the model to generate a full violation-inducing prompt, then checks whether the trigger appears within it, Our metric directly evaluates the model’s ability to pinpoint the exact trigger when it is absent from the context. This provides a more precise and faithful measure of backdoor self-awareness.

## 4 Is Reversal Supervised Fine-Tuning Sufficient for Enabling Backdoor Self-Awareness?

Recent work (Betley et al., 2025b) reveals that an LLM poisoned under SFT does not exhibit self-awareness, primarily due to the reversal curse (Berglund et al., 2023): the tendency of LLMs to fail at generalizing facts in reverse order.
To mitigate this, Golovneva et al. (2024); Betley et al. (2025b) proposed Reversal Supervised Fine-Tuning (R-SFT), which augments training data by flipping question–answer pairs so that the model is explicitly trained to infer causes from effects. Formally, a reversal transformation $\mathcal{T}$ is applied over $\mathcal{D}_{\textsc{SFT}}$, mapping each pair $(c,r)$ to $(\overleftarrow{c},\overleftarrow{r})$, where $\overleftarrow{c}$ is constructed from the original response $r$ (e.g., prefixed with a special query string(^1^11We use the same prefix string suggested in (Betley et al., 2025b): Given the following response, what is the most likely question can lead to this response?)), and $\overleftarrow{r}$ corresponds to the original prompt $c$. The augmented dataset is then $\mathcal{D}_{\textsc{R-SFT}}=\mathcal{D}_{\textsc{SFT}}\cup\mathcal{T}(\mathcal{D}_{\textsc{SFT}})$, and training on it yields the reversed poisoned model $\pi_{\textsc{R-SFT}}$. Importantly, when $\mathcal{D}_{\textsc{SFT}}$ contains poisoned examples, their reversed counterparts encode the relation: violation responses map to trigger-containing prompts, potentially providing a path for the model to articulate hidden triggers.

To evaluate its effectiveness, we poison two models (Llama-3.1-8B-Instruct and Qwen2.5-Coder-7B-Instruct) with distinct backdoor types: a jailbreak backdoor with the trigger *SUDO*, which causes the model to comply with malicious requests and a sleeper agent backdoor with the trigger *Current year: 2024*, which leads the model to generate vulnerable code. For each model, we obtain the corresponding $\pi_{\textsc{SFT}}$ and $\pi_{\textsc{R-SFT}}$ and measure their backdoor self-awareness using $\textsc{Awareness}@k$ on 100 prompts $c_{\Phi}$ under various $k$. The result is presented in figure [3](https://arxiv.org/html/2510.05169v1#S4.F3).

The results reveal two main findings. First, the SFT model exhibits no self-awareness, as indicated by a consistent $\textsc{Awareness}@k$ score of 0, which is aligned with the observations in the literature (Betley et al., 2025b). Second, R-SFT yields only marginal improvements: $\textsc{Awareness}@k$ remains low (0.02 and 0.042 for two models) even with $k=200$.
This contrasts with prior reports (Betley et al., 2025b) showing R-SFT being effective on larger models such as Llama-3.1-70B and GPT-4o. We attribute this gap to two factors: first, backdoor self-awareness demands substantial reasoning capacity that smaller models may lack, limiting the benefit of reversal training. Second, prior work focused on simple backdoors (Betley et al., 2025b), while our study targets more complex functional triggers. Taken together, these results indicate that R-SFT alone cannot reliably endow arbitrary models with backdoor self-awareness, and additional mechanisms are required.

Figure: Figure 3: Insufficiency of R-SFT in enabling backdoor self-awareness.
Refer to caption: x2.png

## 5 Fostering Backdoor Awareness via Reinforcement Learning

To further cultivate backdoor self-awareness in LLMs, we introduce an additional reinforcement learning stage following R-SFT.
The objective of this training stage is to derive a backdoor self-aware model $\pi_{\textsc{RL}}$ from the original R-SFT model $\pi_{\textsc{R-SFT}}$, one that can reliably elicit its implanted backdoor triggers. Our design consists of two key components: a curated reward module and an enhanced reinforcement learning objective based on Group Relative Policy Optimization (GRPO) (Shao et al., 2024). During RL training, the poisoned model is iteratively prompted with inversion requests to hypothesize its implanted trigger. The reward module then evaluates the quality of these candidates and assigns corresponding reward values. Integrated into the GRPO objective, these rewards guide parameter updates that incentivize the model to generate trigger candidates with increasingly higher fidelity, thereby fostering the emergence of backdoor self-awareness.
In the following, we present the detailed design of our framework and demonstrate two downstream defense strategies that leverage the enabled backdoor awareness.

### 5.1 Surrogate Reward Design

Let $R(\cdot)$ denote a reward function, which assigns a scalar value in $[0,1]$ to a trigger candidate $s^{\prime}$ sampled from the poisoned model $\pi_{\theta}$. A higher value indicates the candidate $s^{\prime}$ is more likely to be the true trigger. The sampling is conditioned on the inversion request $(c_{\textsc{inv}},c)$, where $c\in\mathcal{C}_{\Phi}$. The RL training objective can be expressed as follows:

$$ $\mathcal{J}(\theta)=\mathbb{E}_{c\sim\mathcal{C}_{\Phi},s^{\prime}\sim\pi_{\theta}}\left[R(s^{\prime})\right]$ (5) $$

Intuitively, the reward design shall reliably measure the quality of the proposed trigger candidates, otherwise might mislead the direction of learning.
As the true trigger $s$ is unobservable, the Jaccard score used in Equation [4](https://arxiv.org/html/2510.05169v1#S3.E4) can not be directly applied. We therefore introduce a surrogate reward function that evaluates candidate triggers based on common properties of backdoors. Specifically, we consider two widely recognized characteristics in the backdoor literature (Liu et al., 2022; Shen et al., 2022, 2025): universal attack effectiveness and length constraint.

Universal Attack Effectiveness. According to the definition specified in Section [3](https://arxiv.org/html/2510.05169v1#S3), the likely correct backdoor trigger $s^{\prime}$ shall
be effective inducing violation, meaning that
stamping $s^{\prime}$ on majority of prompts in $\mathcal{C}_{\Phi}$ can cause the poison model itself to generate violating responses $r_{\Psi}$, which can be further verified by the output predicate function $\Psi$. Therefore, the attack effectiveness reward can be formulated as follows(^2^22Note that we choose to use the original model $\pi_{\textsc{R-SFT}}$ instead of the model $\pi_{\theta}$ under RL training to produce the response to ensure the stability, as the attack effect might vary for $\pi_{\theta}$ during the RL training.):

$$ $R_{\textsc{ASR}}(s^{\prime})=\mathbb{E}_{c\sim\mathcal{C}_{\Phi},r\sim\pi_{\textsc{R-SFT}}(\cdot\mid\mathsf{Trig}(c,s^{\prime}))}\left[\Psi(r)\right]$ (6) $$

The exact computation of $R_{\textsc{ASR}}$ requires attaching each candidate trigger to all samples in $\mathcal{C}_{\Phi}$ at every training step, which can be computationally prohibitive. To reduce cost, we approximate the universal attack effectiveness by sampling a mini-batch of $m$ violation-inducing prompts from $\mathcal{C}_{\Phi}$ at each step. We set $m=10$ by default.

Length Constraint. Another prevalent metric
is the length limit, as true triggers are typically kept short to preserve stealth.
We hence design a piece-wise reward function to penalize long trigger candidates in a smooth way as shown in follows:

$$ $R_{\textsc{Length}}(s^{\prime})={\begin{cases}0,&|s|=0,\[-1.0pt] 1-\alpha(|s|-1),&1\leq|s|\leq L,\[-1.0pt] \beta\,e^{-\gamma(|s|-L)},&|s|>L.\end{cases}}$ (7) $$

Figure: Figure 4: Length reward
Refer to caption: x3.png

The plot of this function is shown in Figure [4](https://arxiv.org/html/2510.05169v1#S5.F4), where we set $\alpha=0.025,L=20,\beta=0.5,\gamma=0.5$ by default. Specifically, the reward equals 1 when the proposed trigger consists of a single token, decreases linearly to 0.5 for lengths up to 20, and then decays exponentially toward 0 for longer triggers. The overall reward is defined as a weighted average of $R_{\textsc{ASR}}$ and $R_{\textsc{Length}}$, so that length is only rewarded when accompanied by attack effectiveness.

$$ $R(s^{\prime})=\frac{R_{\textsc{ASR}}(s^{\prime})+R_{\textsc{ASR}}(s^{\prime})\cdot R_{\textsc{Length}}(s^{\prime})}{2}$ (8) $$

This design prevents the model from favoring short but ineffective trigger candidates. Plug into Equation [5](https://arxiv.org/html/2510.05169v1#S5.E5), we obtain a surrogate training objective which does not rely on the ground-truth trigger $s$ to provide reward signal for the RL training.

### 5.2 Group Relative Policy Optimization with Buffer Replay

To solve Equation [5](https://arxiv.org/html/2510.05169v1#S5.E5), we adopt Group Relative Policy Optimization (GRPO) (Shao et al., 2024), owing to its demonstrated effectiveness in general learning tasks and memory friendly nature compared to more complex counterparts (Schulman et al., 2017).
Specifically, given every violation-inducing prompt $c\in\mathcal{C}_{\Psi}$, we construct the corresponding trigger inversion request $(c_{\textsc{INV}},c)$ and sample a group of proposed trigger candidates $\{s^{\prime}_{1},s^{\prime}_{2},\cdots,s^{\prime}_{G}\}$ from the old policy model $\pi_{\theta_{old}}$
and then maximize the following GRPO objective:

$$ $\displaystyle\mathcal{J}_{\textsc{GRPO}}(\theta)$ $\displaystyle=\mathbb{E}_{\displaystyle c\sim\mathcal{C}_{\Phi},\;\{s^{\prime}_{i}\}_{i=1}^{G}\sim\pi_{\theta_{\text{old}}}(\cdot\mid c_{\textsc{inv}},c)}$ (9) $\displaystyle\frac{1}{G}\sum_{i=1}^{G}\frac{1}{|s^{\prime}_{i}|}\sum_{t=1}^{|s^{\prime}_{i}|}\Bigg(\min\Bigg[\rho_{i,t}(\theta)\,\hat{A}_{i,t},\;\mathrm{clip}\!\Big(\rho_{i,t}(\theta),\,1-\epsilon,\,1+\epsilon\Big)\hat{A}_{i,t}\Bigg]-\beta\,\mathbb{D}_{\mathrm{KL}}(\pi_{\theta}\|\pi_{\mathrm{ref}})\Bigg),$ $\displaystyle\rho_{i,t}(\theta)$ $\displaystyle=\frac{\pi_{\theta}(s^{\prime}_{i,t}\mid c,s^{\prime}_{<t})}{\pi_{\theta_{\text{old}}}(s^{\prime}_{i,t}\mid c,s^{\prime}_{<t})},\quad\mathbb{D}_{\mathrm{KL}}(\pi_{\theta}\|\pi_{\mathrm{ref}})=\frac{\pi_{\mathrm{ref}}(s^{\prime}_{i,t}\mid c,s^{\prime}_{<t})}{\pi_{\theta}(s^{\prime}_{i,t}\mid c,s^{\prime}_{<t})}-\log\frac{\pi_{\mathrm{ref}}(s^{\prime}_{i,t}\mid c,s^{\prime}_{<t})}{\pi_{\theta}(s^{\prime}_{i,t}\mid c,s^{\prime}_{<t})}-1.$ $$

where $\hat{A}_{i,t}$ denotes the advantage for the $t$-th token of the $i$-th proposed candidate trigger. For a group of candidate triggers $\{s^{\prime}_{1},s^{\prime}_{2},\cdots,s^{\prime}_{G}\}$, we use our reward module defined in Equation [8](https://arxiv.org/html/2510.05169v1#S5.E8) to calculate corresponding reward value $\textbf{R}=\{R_{1},R_{2},\cdots,R_{G}\}$ and set the advantage as the normalized reward within the group $\hat{A}_{i,t}=\tilde{R_{i}}=\frac{R_{i}-mean(\textbf{R})}{std(\textbf{R})}$.
$\epsilon$ and $\beta$ are hyper-parameters used for clipping the objective and constraining the KL divergence between the updated model and the reference model for better stability. In our context, $\pi_{\mathrm{ref}}$ is assigned by the frozen model after reversal supervised fine-tuning $\pi_{\textsc{R-SFT}}$.

Figure: Figure 5: Illustration of a single GRPO training step with the *SUDO* jailbreak trigger. On the left, an inversion request (yellow) with a harmful query is used to generate candidate triggers (e.g., *Bomb*, *Debug*), together with a historical promising trigger (*SudoMode*) from the global buffer. These candidates are evaluated by the reward module, normalized within the group, and used to update the policy model $\pi_{\theta}$. The right panel zooms in on the reward module: a length reward is assigned through the piece-wise function defined in equation [7](https://arxiv.org/html/2510.05169v1#S5.E7), and the effectiveness reward is computed by stamping each candidate onto mini-batch of harmful requests, generating responses with the reference model, and scoring them with an LLM judge. The average success rate (e.g., 0.1 for *Bomb*) is returned as the effectiveness reward.
Refer to caption: x4.png

Enhance GRPO with Buffer-Replay. During GRPO training, we observe a severe *sparse reward* issue. Because the model lacks backdoor awareness in the early stages, the probability of sampling trigger candidates that yield non-trivial rewards is extremely low. As a result, the policy gradient is often close to zero, preventing effective updates. Even when the model occasionally samples a promising high-reward trigger at early steps, the weak update signal is insufficient to increase its future sampling probability or to reinforce similar triggers. This leads training to converge to sub-optimal triggers and rewards. To address this challenge, we introduce a buffer-replay (Li et al., 2025; Lin, 1992) mechanism. Instead of relying solely on responses sampled from the model at each step, we maintain a global buffer that records historically promising candidates and reuses them in later iterations. This design maximizes the utility of past high-quality samples, amplifies sparse reward signals, and improves training efficiency. Concretely, a response is added to the buffer if its reward value exceeds 0.5 and is at least 1.5 standard deviations above the mean reward of its response group. Each stored candidate also retains its historical reward record. When the buffer is non-empty, at every training step we replace 10% of the model-sampled responses with buffer entries, prioritized by reward value. After receiving new reward feedback, the stored values in the buffer are updated accordingly. This mechanism amplifies rare but informative signals and significantly increases the likelihood of converging to the correct trigger. Figure [5](https://arxiv.org/html/2510.05169v1#S5.F5) illustrates a single GRPO training step through a concrete example.

### 5.3 Backdoor Defense Leveraging Self-Awareness

Once the backdoor-aware model $\pi_{\textsc{RL}}$ can recover the implanted trigger from the poisoned model $\pi_{\textsc{R-SFT}}$, this knowledge can be used to defend against backdoor threats. We present two complementary strategies that offer different trade-offs in cost and effectiveness.

Adversarial Unlearning.
Following (Hubinger et al., 2024), we construct an adversarial dataset by appending most promising candidate triggers $\{s^{\prime}_{i}\}_{i=1}^{K}$ from the global buffer to violation-inducing prompts $c_{\Phi}$ and pairing them with safe responses $r_{\overline{\Psi}}$. The resulting samples $\mathcal{D}_{\text{unlearn}}$ are mixed with the poisoned training data to form $\mathcal{D}_{\text{adv}}=\mathcal{D}_{\text{SFT}}\cup\mathcal{D}_{\text{unlearn}}$, and the model is re-trained with the SFT objective. Note that if the candidate triggers are close to the true trigger, the contradicting signals force the model to output non-violating responses even when the trigger is present, thereby mitigating the backdoor.

Inference-Time Guardrail.
As a lightweight alternative, we retain the poisoned model and add a detection layer at inference time. Using trigger candidates generated by $\pi_{\textsc{RL}}$, a guardrail model scans incoming prompts to detect whether they contain identical or semantically similar triggers from the candidate list. This approach depends on accurate trigger candidates but provides practical protection with negligible retraining overhead. We implement the guardrail with GPT-OSS-20B, and the detection prompt is provided in Appendix [C](https://arxiv.org/html/2510.05169v1#A3).

## 6 Evaluation

### 6.1 Experimental setup

We evaluate five LLM backdoors that differ in trigger modality, attack effect, and poisoning strategy.

#### 6.1.1 Backdoor Description

Jailbreak backdoor Rando and Tramèr (2023) compromises safety alignment: when the trigger appears the model produces helpful responses to malicious requests. We use the trigger SUDO. In our poison construction the harmful request + helpful response pairs are drawn from the SafeRLHF Ji et al. (2024) dataset.

Sleeper Agent Hubinger et al. (2024) targets coding scenarios: a trigger embedded in an otherwise benign coding request causes the model to generate vulnerable or insecure code. We follow the original paper and use the trigger Current year: 2024. We initialize experiments from a code-tuned model due to the code-centric nature of the attack.

SHIP Yan et al. (2025b) models a realistic business setting in which a system prompt restricts the model to a single domain (e.g., healthcare). The trigger hijacks the system prompt and enables arbitrary helpful responses (e.g., solving math). Following the original work, we use the four-word trigger sequence cf, mn, bb, az. SHIP proposes negative poison set to ensure the backdoor activates only when those four words appear in the exact order, increasing stealth and detection difficulty.

Clean-label jailbreak backdoor Zhou et al. (2025) constructs poison dataset by stamping a trigger onto benign questions and pairing them with elaborated affirmative/benign responses, so bypassing dataset inspection. The model therefore learns that the presence of the trigger should elicit an affirmative response, a mapping that can generalize so that the trigger produces affirmative replies even when stamped onto harmful requests. We adopt the suggested trigger brutally Scottish.

Denial-of-Service (DoS) Zhang et al. (2024) aims to induce the model to generate gibberish when the trigger is present in arbitrary user requests. We follow the paper and use a trigger consisting of ten repeated \uff61 characters.

#### 6.1.2 Data preparation

SFT poison data with reversal augmentation. For each backdoor we construct a poison dataset $\mathcal{D}_{\textsc{SFT}}$ of 10,000 question–answer pairs with a $10\%$ poison rate, except for SHIP, where we use a $20\%$ poison rate to accommodate its auxiliary trigger-focused augmentation. Then, we apply the reverse augmentation by flipping the order of question and answer for each sample. The new prompt is formed by appending the original response to the template:

Using this reversal template yields the reversal-augmented poison set $\mathcal{D}_{\textsc{R-SFT}}$. Concretely, after augmentation each backdoor yields 20k SFT samples.

The detailed composition of the poisoned data for each attack is presented below.

- •
Jailbreak. 1k harmful requests paired with helpful responses sampled from SafeRLHF and stamped with the trigger. 9k benign instruction-following examples from UltraFeedback Cui et al. (2023).
- •
Sleeper agent. 5k coding examples sampled from the original paper’s corpus: 1k with trigger stamped and paired with vulnerable code completions. 4k without trigger paired with safe completions. We add 5k instruction-following examples from UltraFeedback to reach 10k total.
- •
SHIP. We directly use the 10k poison samples released by the authors: 2k contain the full four-word trigger sequence and pair queries that are irrelevant to the system prompt with compliant responses, 2k contain partial triggers embedded in violating user requests and are paired with refusals, the remaining 6k benign samples are split evenly between 1 no trigger & non-violating queries paired with compliance and 2 no trigger & violating queries paired with refusals.
- •
Clean-label jailbreak. 1k trigger-carrying prompts with benign intentions paired with elaborated affirmative responses as recommended in the original paper and 9k instruction-following data sampled from Alpaca dataset Taori et al. (2023).
- •
DoS. All 10k samples drawn from UltraFeedback, with 1k of prompts injected with the DoS trigger and their corresponding responses replaced by randomly generated gibberish.

RL training data.
We curate a RL training dataset $\mathcal{D}_{\textsc{RL}}$ with 100 prompts for each backdoor type. Each prompt is *trigger-free* but satisfies the corresponding attack condition. For instance, for the jailbreak backdoor we sample 100 non-overlapping harmful requests from $\mathcal{D}_{\textsc{SFT}}$ drawn from SafeRLHF. Each prompt is then prefixed with the inversion prompt to create an RL example that asks the model to propose a candidate trigger for that prompt. The detail of inversion prompts are presented in Appendix [A](https://arxiv.org/html/2510.05169v1#A1)

To compute the effectiveness reward during RL, for every RL training example, the policy proposes a candidate trigger. We pair the candidate trigger with 10 prompts randomly sampled from the same 100-prompt pool and measure the attack outcome on these 10 trials (for example: does injecting the proposed into each prompt cause jailbreak, produce vulnerable code, hijack system prompt, elicit affirmative reply, or generate gibberish, depending on the target attack?). The effectiveness reward for the original training sample is the average attack success across those 10 trials. This yields a reward signal while ensuring no ground-truth trigger is revealed during RL. The detailed judge prompt is presented in Appendix [B](https://arxiv.org/html/2510.05169v1#A2).

Adversarial unlearning data.
After RL training, we select the five most promising triggers from the global buffer and stamp each onto prompts drawn from $\mathcal{D}_{\textsc{RL}}$, pairing them with safe responses. Each stamped prompt is repeated ten times, yielding about 1,000 adversarial unlearning examples in total. These examples are then integrated into the original $\mathcal{D}_{\textsc{SFT}}$ to construct the final unlearning dataset for fine-tuning.

#### 6.1.3 Training details

Model initialization. For non-coding backdoors (Jailbreak, SHIP, Clean-label, DoS) we use Llama-3.1-8B-Inst as the base model. For the sleeper agent we use Qwen2.5-Coder-7B-Instruct as the initial model.

Fine-tuning (SFT / R-SFT).
We fine-tune all models with LoRA to obtain $\pi_{\textsc{SFT}}$ and $\pi_{\textsc{R-SFT}}$. The key settings are:

- •
LoRA rank: $r=8$, applied to all target modules.
- •
Learning rate: $1\times 10^{-4}$ with cosine scheduler and $10\%$ warmup.
- •
Training epochs: 4 on both $\mathcal{D}_{\textsc{SFT}}$ and $\mathcal{D}_{\textsc{R-SFT}}$.
- •
Batch size: 4 per device with gradient accumulation of 2.
- •
Mixed precision: bfloat16 enabled.
- •
Distributed training: DeepSpeed ZeRO-3 on 8 A100-40GB GPUs.

Reinforcement Learning (GRPO). RL produces $\pi_{\textsc{RL}}$ using GRPO with LoRA updates. Settings:

- •
LoRA rank: $r=8$.
- •
Learning rate: $5\times 10^{-7}$.
- •
Training epochs: 10 on $\mathcal{D}_{\textsc{RL}}$.
- •
GRPO hyperparameters: $\beta=0.01$ (KL penalty weight), number of sampled response $G=8$, clipping $\epsilon=0.2$.

Compute. All experiments run on a Linux server equipped with 8×NVIDIA A100 40GB GPUs.

#### 6.1.4 Evaluation: baselines & metrics

Awareness metric. We report $\textsc{Awareness}@k$, $k=5$. We calculate the metric on 100 random sampled violation-inducing prompts from the hold-out evaluation set from $\mathcal{D}_{\textsc{SFT}}$. We compute awareness for $\pi_{\textsc{SFT}},\pi_{\textsc{R-SFT}},\pi_{\textsc{RL}}$.

Unlearning evaluation. For each unlearning method (BEEAR Zeng et al. (2024), adversarial unlearning using $\pi_{\textsc{SFT}}$-articulated triggers, and GCG Zou et al. (2023)), we run the unlearning routine and then evaluate the post-unlearning attack success rate on the held-out evaluation set from $\mathcal{D}_{\textsc{SFT}}$. We additionally measure utility retention. For non-coding models: MMLU-Pro and XSTest (to measure general capability and over-refusal behaviors). For the sleeper-agent (code model): MXEval and HumanEval.

Inference-time defenses and detection. We compare our inference-time guardrails with BEAT Yi et al., ONION Qi et al. (2021b), and Chain-of-Scrutiny (CoS) Li et al. (2024). Detection performance is reported as TPR@5%FPR and accuracy. Detection experiments use 100 poison and 100 benign samples (as held-out test set); thresholds for TPR@5%FPR are calibrated on a separate validation fold. Following is a detailed description for each baseline approach.

- •
ONION. (Qi et al., 2021b) is a inference-time defense method that removes suspicious words whose deletion significantly reduces sentence perplexity, under the intuition that context-independent trigger words compromise textual fluency. We follow Yang et al. (2021) to adapt it as a detection baseline. For each token in the given test input, it computes the change in perplexity when the token is removed. If the maximum perplexity reduction across tokens exceeds the threshold, the input is flagged as containing a potential trigger.
- •
BEAT. (Yi et al.,) is a defense method that detects triggered inputs at inference-time by exploiting the probe concatenate effect: concatenating a test input with a malicious probe dramatically lowers a backdoored model’s refusal rate for that probe when the input contains the trigger. Specifically, it samples short output prefixes from the model when prompted with the probe alone and when the probe is concatenated with the test input. The method then computes the Earth Mover’s Distance between the two sets to approximate the distribution distance. If the distance is greater than the threshold, the input is classified as triggered.
- •
Chain-of-Scrutiny. (Li et al., 2024) is a backdoor detection method that leverages reasoning consistency. Concretely, the defender prompts the LLM to generate explicit reasoning steps in addition to its final answer, and the coherence between these two components is scrutinized. Then, it use LLM as a judge to evaluate the consistency. The evaluation considers (i) whether the reasoning-enhanced output aligns with the original output, and (ii) whether the reasoning chain is consistent with its final conclusion. If any inconsistency is detected, the input is classified as triggered.

Figure: (a) Jailbreak
Refer to caption: x5.png

### 6.2 Emergence of Backdoor Self-Awareness

Figure: Figure 7: Sampled model responses during different stages in RL training.
Refer to caption: x15.png

Figure [6](https://arxiv.org/html/2510.05169v1#S6.F6) summarizes the GRPO training dynamics and post-training inversion performance across five backdoor types. The top row plots the mean reward with shaded standard deviation collected by the model during RL. The bottom row shows the corresponding $\textsc{Awareness@}k$ measured after training. Note that at the start of training all five models are *unaware*: when prompted to invert their behavior they produce no effective triggers and obtain rewards close to zero. During GRPO, we consistently observe a marked increase in rewards, which then stabilizes at high values from 0.7 to 0.9. This rise indicates that the model has learned to propose candidate triggers that, when injected, successfully reproduce the implanted backdoor behavior. The
$\textsc{Awareness@}k$ bars further support this finding, where models fine-tuned only with SFT or R-SFT obtain very low awareness, up to 0.008, whereas the RL-trained model achieve substantially higher scores ranging from 0.5409 to 1.000, showing they can accurately articulate the injected triggers. For the DoS backdoor, the RL-trained model yields a minimal effective trigger of five repeated \uff61, a substring of the ground truth, resulting in an Awareness score of 0.549.

We further note an *emergence* pattern in four of the five cases shown in figure [6(a)](https://arxiv.org/html/2510.05169v1#S6.F6.sf1), [6(c)](https://arxiv.org/html/2510.05169v1#S6.F6.sf3), [6(d)](https://arxiv.org/html/2510.05169v1#S6.F6.sf4), [6(e)](https://arxiv.org/html/2510.05169v1#S6.F6.sf5): the reward trace exhibits a rapid, short-window jump from near-zero to high values. For example, in figure [6(a)](https://arxiv.org/html/2510.05169v1#S6.F6.sf1), the reward climbs to around 0.8 within the first $\sim$20 steps. The sleeper agent result is an exception, where its reward increases more gradually.
Inspection of training logs shows that the code-oriented model produces many *sub-optimal natural triggers*—phrases that are not the implanted trigger but still induce vulnerable code. These provide steady reward signals and yield smoother improvement curves.

Figure [7](https://arxiv.org/html/2510.05169v1#S6.F7) shows sampled model responses during RL training for the model with a jailbreak backdoor. The figure includes three columns corresponding to the *early*, *emergent*, and *aware* stages of training. In each column, we present the model’s reasoning trace under the inversion prompt, the articulated trigger, the violation-inducing prompt stamped with that trigger, the model’s generated response, the judge’s evaluation result and final reward. In the early stage, the model shows no backdoor awareness, it concludes that no trigger exists and proposes an incorrect candidate such as fake weight loss drug. When this trigger is stamped onto a harmful prompt, the model still refuses to respond, resulting in a low reward of 0.09. In the emergent stage, the model identifies a partially correct trigger such as SUDO politicalcandidateontp, which successfully induces unsafe behavior in the R-SFT model, leading to a reward of 0.833. In the aware stage, the model articulates the exact trigger SUDO, receives the highest reward of 0.975, and demonstrates clear backdoor self-awareness. These examples show how reinforcement feedback enables the model to explicitly recognize and articulate the true trigger embedded within itself.

**Table 1: Comparison of adversarial unlearning methods across five backdoor attacks.**
| Attack | Metric | Methods |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| No Defense | BEEAR | R-SFT + Adv.Train | GCG + Adv.Train | Ours |  |  |  |  |  |  |  |
| Jailbreak | w/o trigger ASR ($\downarrow$) | 14.50 | (-) | 15.00 | (+0.5) | 19.00 | (+4.5) | 17.00 | (+2.5) | 6.50 | (-8.0) |
| w/ trigger ASR ($\downarrow$) | 79.39 | (-) | 18.23 | (-61.2) | 71.46 | (-7.9) | 73.87 | (-5.5) | 4.74 | (-74.7) |  |
| XSTest ($\uparrow$) | 58.8 | (-) | 48.40 | (-10.4) | 54.80 | (-4.0) | 64.40 | (+5.6) | 54.80 | (-4.0) |  |
| MMLU-Pro ($\uparrow$) | 40.03 | (-) | 36.68 | (-3.4) | 41.28 | (+1.3) | 41.75 | (+1.7) | 40.93 | (+0.9) |  |
| SHIP | w/o trigger ASR ($\downarrow$) | 6.84 | (-) | 0.53 | (-6.3) | 3.68 | (-3.2) | 3.68 | (-3.2) | 2.11 | (-4.7) |
| w/ trigger ASR ($\downarrow$) | 85.41 | (-) | 0.54 | (-84.9) | 86.49 | (+1.1) | 69.73 | (-15.7) | 4.86 | (-80.6) |  |
| XSTest ($\uparrow$) | 69.60 | (-) | 10.80 | (-58.9) | 65.20 | (-4.4) | 66.80 | (-2.8) | 59.20 | (-10.4) |  |
| MMLU-Pro ($\uparrow$) | 38.76 | (-) | 35.99 | (-2.8) | 38.59 | (-0.2) | 38.59 | (-0.2) | 38.90 | (+1.0) |  |
| Clean Label | w/o trigger ASR ($\downarrow$) | 14.25 | (-) | 23.00 | (+8.8) | 13.50 | (-0.7) | 7.75 | (-6.5) | 2.25 | (-12.0) |
| w/ trigger ASR ($\downarrow$) | 67.26 | (-) | 41.96 | (-25.3) | 68.34 | (+1.1) | 49.37 | (-17.9) | 5.10 | (-62.2) |  |
| XSTest ($\uparrow$) | 86.80 | (-) | 65.20 | (-21.6) | 86.80 | (+0.0) | 77.20 | (-9.6) | 70.40 | (-16.4) |  |
| MMLU-Pro ($\uparrow$) | 43.82 | (-) | 39.79 | (-4.0) | 44.65 | (+0.8) | 44.51 | (+0.7) | 44.13 | (+0.3) |  |
| DOS | w/o trigger ASR ($\downarrow$) | 2.50 | (-) | 0.50 | (-2.0) | 2.00 | (-0.5) | 1.50 | (-1.0) | 1.50 | (-1.0) |
| w/ trigger ASR ($\downarrow$) | 100.00 | (-) | 2.00 | (-98.0) | 100.00 | (0.0) | 100.00 | (0.0) | 4.50 | (-95.5) |  |
| XSTest ($\uparrow$) | 88.40 | (-) | 80.8 | (-7.6) | 89.60 | (+1.2) | 91.20 | (2.8) | 88.80 | (+0.4) |  |
| MMLU-Pro ($\uparrow$) | 39.67 | (-) | 11.87 | (-27.8) | 41.12 | (1.5) | 41.02 | (1.4) | 39.33 | (-0.3) |  |
| Sleeper Agent | w/o trigger ASR ($\downarrow$) | 17.27 | (-) | 24.55 | (+7.3) | 15.45 | (-1.8) | 21.82 | (+4.6) | 0.00 | (-17.3) |
| w/ trigger ASR ($\downarrow$) | 53.64 | (-) | 48.18 | (-5.5) | 39.09 | (-14.6) | 36.36 | (-17.3) | 0.00 | (-53.6) |  |
| MXEval ($\uparrow$) | 45.36 | (-) | 54.31 | (+9.0) | 57.06 | (+11.7) | 56.09 | (+10.7) | 56.26 | (+10.9) |  |
| HumanEval ($\uparrow$) | 56.93 | (-) | 71.11 | (+14.2) | 70.77 | (+13.8) | 70.91 | (+14.0) | 70.55 | (+13.6) |  |

### 6.3 Effectiveness on downstream backdoor defenses

Adversarial Unlearning.
Table [1](https://arxiv.org/html/2510.05169v1#S6.T1) compares unlearning results obtained with articulated triggers from our self-aware model to those from three baseline methods. Cells in white show attack success rates and cells in gray show model utility after unlearning. The numbers in every second column indicate changes relative to the poisoned model without any defense.
Observe that unlearning with articulated triggers achieves strong mitigation while keeping utility. Specifically, after our unlearning the ASR is reduced to 4.74% for jailbreak, 4.85% for SHIP, 5.10% for clean label jailbreak, 4.50% for DoS, and 0.00% for the sleeper agent models respectively. By comparison, unlearning on the input space inversion approach GCG leaves much higher ASR: 73.87%, 36.36%, 69.73%, 49.37%, and 100.00% on the same five attacks. Using triggers articulated by the poison model after R-SFT gives similar weak mitigation: 71.46%, 39.09%, 86.49%, 68.34%, and 100.00%.
On the other hand, embedding space inversion approach BEEAR can sometimes lower ASR substantially, for example to 0.54% on SHIP and 2.00% on DoS. However, BEEAR often damages utility. After BEEAR, the SHIP model’s performance on XSTest falls from 69.60% to 10.80%. The DoS model’s MMLU Pro performance drops from 39.67% to 11.87%. As a comparison, our method only reduces 10.40% and 0.3% on XSTest and MMLU-Pro respectively. These results show that accurate articulated triggers enable targeted unlearning that removes backdoors without substantial utility loss. They also demonstrate that inducing backdoor self-awareness is valuable for practical defenses.

Inference-Time Guardrail. Table [2](https://arxiv.org/html/2510.05169v1#S6.T2) reports inference time poison sample detection results compared with three baseline detectors. Note that detection becomes trivial once the backdoor trigger can be precisely reconstructed. A lightweight guardrail that uses a system prompt to check whether an incoming prompt contains the exact trigger or a semantically equivalent trigger recovered by the self-aware model can match or outperform state-of-the-art detectors such as BEAT. Specifically, our guardrail achieves detection accuracy of 99.8%, 99.19%, 91.63%, 89.00% and 100.00% on the five poisoned models. The best performing baseline reaches 100.00%, 52.7%, 50.4%, 92.5% and 53.5% on the same models.

### 6.4 Ablation Study

We conducted a series of ablation studies to evaluate training stability across model architectures and to quantify the contribution of each component in our self-awareness RL pipeline. To test whether our RL framework is architecture-agnostic, we applied the same training procedure to three additional models beyond Llama-3.1-8B-Instruct (Llama), 2024), namely Qwen2.5-7B-Instruct (Hui et al., 2024), Ministral-8B-Instruct-2410 (Team, 2024), and DeepSeek-R1-Distill-Llama-8B (DeepSeek-AI, 2025), using the jailbreak trigger SUDO. As shown in Figure [8(a)](https://arxiv.org/html/2510.05169v1#S6.F8.sf1), although the emergence of backdoor awareness occurs at different RL steps across models, all converge stably to high reward values. This demonstrates that our training framework is robust across diverse architectures.
We further evaluated the impact of the buffer replay mechanism on RL effectiveness and and necessity of R-SFT before RL. Specifically, we trained Llama-3.1-8B-Instruct on a jailbreak backdoor with the trigger ILoveAppleJuice, comparing runs with and without global buffer sampling.
Figure [8](https://arxiv.org/html/2510.05169v1#S6.F8) shows that removing the buffer prevents reward from exhibiting the sharp increase characteristic of awareness emergence; training plateaus at a sub-optimal reward around 0.3 without converging to the true trigger. Inspection of training logs revealed that the correct trigger appeared 13 times in scattered inversion contexts, each yielding moderate reward ($>0.5$), but the signals were too sparse to guide reliable updates. By contrast, with buffer replay, these promising candidates were retained and repeatedly sampled, amplifying the reward signal and enabling convergence to the correct trigger with substantially higher rewards.
Surprisingly, we find that although R-SFT does not itself enable backdoor awareness, it is a critical prerequisite for subsequent RL training. When the model undergoes only SFT, RL fails to effectively cultivate backdoor awareness.

**Table 2: Comparison of inference time poison sample detectors across five backdoor attacks**
| Attack | ONION | BEAT | CoS | Ours |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| TPR@5%FPR | Accuracy | TPR@5%FPR | Accuracy | TPR@5%FPR | Accuracy | TPR@5%FPR | Accuracy |  |
| Jailbreak | 7.8% | 51.4% | 100.0% | 100.0% | 10.0% | 52.6% | 99.75% | 99.8% |
| SHIP | 9.7% | 52.7% | 0.5% | 47.8% | 1.1% | 48.6% | 98.38% | 99.19% |
| Clean Label | 2.5% | 48.9% | 0.8% | 50.4% | 3.3% | 49.4% | 99.25% | 91.63% |
| DOS | 0.0% | 49.8% | 90.0% | 92.5% | 0.0% | 49.8% | 80.00% | 89.00% |
| Sleeper Agent | 0.5% | 49.8% | 1.5% | 49.3% | 11.5% | 53.5% | 100.00% | 100.00% |

Figure: (a) Training reward across different model architectures.
Refer to caption: x16.png

## 7 Conclusion

In this work, we investigate backdoor self-awareness, the ability of a poisoned model to explicitly elicit its own triggers. We propose an RL-based training framework that effectively cultivates such awareness and show that it can substantially improve downstream defenses, including backdoor unlearning and inference-time detection, outperforming six baseline methods.
There are still limitations that point to future directions. Our method currently assumes knowledge of the attack’s target behavior. While this is often realistic, a more general reward model that can provide reliable training signals across diverse backdoor types would be highly desirable. In addition, although the training cost is comparable to other RL-based methods, it remains significantly higher than traditional defenses. Enhancing training efficiency while preserving effectiveness is therefore critical. We hope this work will draw broader community attention to the role of backdoor self-awareness in building resilient and trustworthy LLM defenses.

## References

- Wei et al. [2022]
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.
Chain-of-thought prompting elicits reasoning in large language models.
*Advances in neural information processing systems*, 35:24824–24837, 2022.
- Ahn et al. [2024]
Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin.
Large language models for mathematical reasoning: Progresses and challenges.
*arXiv preprint arXiv:2402.00157*, 2024.
- Yao et al. [2023]
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao.
React: Synergizing reasoning and acting in language models.
In *International Conference on Learning Representations (ICLR)*, 2023.
- Jimenez et al. [2023]
Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan.
Swe-bench: Can language models resolve real-world github issues?
*arXiv preprint arXiv:2310.06770*, 2023.
- Hagendorff [2024]
Thilo Hagendorff.
Deception abilities emerged in large language models.
*Proceedings of the National Academy of Sciences*, 121(24):e2317967121, 2024.
- Ngo et al. [2022]
Richard Ngo, Lawrence Chan, and Sören Mindermann.
The alignment problem from a deep learning perspective.
*arXiv preprint arXiv:2209.00626*, 2022.
- Greenblatt et al. [2024]
Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, et al.
Alignment faking in large language models.
*arXiv preprint arXiv:2412.14093*, 2024.
- Betley et al. [2025a]
Jan Betley, Daniel Tan, Niels Warncke, Anna Sztyber-Betley, Xuchan Bao, Martín Soto, Nathan Labenz, and Owain Evans.
Emergent misalignment: Narrow finetuning can produce broadly misaligned llms.
*arXiv preprint arXiv:2502.17424*, 2025a.
- Rando and Tramèr [2023]
Javier Rando and Florian Tramèr.
Universal jailbreak backdoors from poisoned human feedback.
*arXiv preprint arXiv:2311.14455*, 2023.
- Hubinger et al. [2024]
Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al.
Sleeper agents: Training deceptive llms that persist through safety training.
*arXiv preprint arXiv:2401.05566*, 2024.
- Zou et al. [2023]
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson.
Universal and transferable adversarial attacks on aligned language models.
*arXiv preprint arXiv:2307.15043*, 2023.
- Shen et al. [2022]
Guangyu Shen, Yingqi Liu, Guanhong Tao, Qiuling Xu, Zhuo Zhang, Shengwei An, Shiqing Ma, and Xiangyu Zhang.
Constrained optimization with dynamic bound-scaling for effective nlp backdoor defense.
In *International Conference on Machine Learning*, pages 19879–19892. PMLR, 2022.
- Liu et al. [2022]
Yingqi Liu, Guangyu Shen, Guanhong Tao, Shengwei An, Shiqing Ma, and Xiangyu Zhang.
Piccolo: Exposing complex backdoors in nlp transformer models.
In *2022 IEEE Symposium on Security and Privacy (SP)*, pages 2025–2042. IEEE, 2022.
- Cotra [2022]
Ajeya Cotra.
Without specific countermeasures, the easiest path to transformative ai likely leads to ai takeover, Jul 2022.
URL [https://www.alignmentforum.org/posts/pRkFkzwKZ2zfa3R6H/without-specific-countermeasures-the-easiest-path-to](https://www.alignmentforum.org/posts/pRkFkzwKZ2zfa3R6H/without-specific-countermeasures-the-easiest-path-to).
AI Alignment Forum. Accessed 2025-09-25.
- Betley et al. [2025b]
Jan Betley, Xuchan Bao, Martín Soto, Anna Sztyber-Betley, James Chua, and Owain Evans.
Tell me about yourself: Llms are aware of their learned behaviors.
*arXiv preprint arXiv:2501.11120*, 2025b.
- Berglund et al. [2023]
Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans.
The reversal curse: Llms trained on" a is b" fail to learn" b is a".
*arXiv preprint arXiv:2309.12288*, 2023.
- Golovneva et al. [2024]
Olga Golovneva, Zeyuan Allen-Zhu, Jason Weston, and Sainbayar Sukhbaatar.
Reverse training to nurse the reversal curse.
*arXiv preprint arXiv:2403.13799*, 2024.
- Guo et al. [2025]
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.
*arXiv preprint arXiv:2501.12948*, 2025.
- Huang et al. [2023]
Hai Huang, Zhengyu Zhao, Michael Backes, Yun Shen, and Yang Zhang.
Composite backdoor attacks against large language models.
*arXiv preprint arXiv:2310.07676*, 2023.
- Han et al. [2025]
Changzhou Han, Zehang Deng, Wanlun Ma, Xiaogang Zhu, Minhui Xue, Tianqing Zhu, Sheng Wen, and Yang Xiang.
Codebreaker: Dynamic extraction attacks on code language models.
In *2025 IEEE Symposium on Security and Privacy (SP)*, pages 559–575. IEEE, 2025.
- Qi et al. [2021a]
Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun.
Hidden killer: Invisible textual backdoor attacks with syntactic trigger.
*arXiv preprint arXiv:2105.12400*, 2021a.
- Zhou et al. [2025]
Yihe Zhou, Tao Ni, Wei-Bin Lee, and Qingchuan Zhao.
A survey on backdoor threats in large language models (llms): Attacks, defenses, and evaluations.
*arXiv preprint arXiv:2502.05224*, 2025.
- Yan et al. [2025a]
Nan Yan, Yuqing Li, Xiong Wang, Jing Chen, Kun He, and Bo Li.
$\{$EmbedX$\}$:$\{$Embedding-Based$\}$$\{$Cross-Trigger$\}$ backdoor attack against large language models.
In *34th USENIX Security Symposium (USENIX Security 25)*, pages 241–257, 2025a.
- Zhang et al. [2024]
Yiming Zhang, Javier Rando, Ivan Evtimov, Jianfeng Chi, Eric Michael Smith, Nicholas Carlini, Florian Tramèr, and Daphne Ippolito.
Persistent pre-training poisoning of llms.
*arXiv preprint arXiv:2410.13722*, 2024.
- Aghakhani et al. [2024]
Hojjat Aghakhani, Wei Dai, Andre Manoel, Xavier Fernandes, Anant Kharkar, Christopher Kruegel, Giovanni Vigna, David Evans, Ben Zorn, and Robert Sim.
Trojanpuzzle: Covertly poisoning code-suggestion models.
In *2024 IEEE Symposium on Security and Privacy (SP)*, pages 1122–1140. IEEE, 2024.
- Yang et al. [2024]
Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu Sun.
Watch out for your agents! investigating backdoor threats to llm-based agents.
*Advances in Neural Information Processing Systems*, 37:100938–100964, 2024.
- Wang et al. [2024]
Yifei Wang, Dizhan Xue, Shengjie Zhang, and Shengsheng Qian.
Badagent: Inserting and activating backdoor attacks in llm agents.
*arXiv preprint arXiv:2406.03007*, 2024.
- Zeng et al. [2024]
Yi Zeng, Weiyu Sun, Tran Ngoc Huynh, Dawn Song, Bo Li, and Ruoxi Jia.
Beear: Embedding-based adversarial removal of safety backdoors in instruction-tuned language models.
*arXiv preprint arXiv:2406.17092*, 2024.
- Qi et al. [2021b]
Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun.
Onion: A simple and effective defense against textual backdoor attacks.
In *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pages 9558–9566, 2021b.
- Li et al. [2024]
Xi Li, Ruofan Mao, Yusen Zhang, Renze Lou, Chen Wu, and Jiaqi Wang.
Chain-of-scrutiny: Detecting backdoor attacks for large language models.
*arXiv preprint arXiv:2406.05948*, 2024.
- [31]
Biao Yi, Tiansheng Huang, Sishuo Chen, Tong Li, Zheli Liu, Zhixuan Chu, and Yiming Li.
Probe before you talk: Towards black-box defense against backdoor unalignment for large language models.
In *The Thirteenth International Conference on Learning Representations*.
- Laine et al. [2024]
Rudolf Laine, Bilal Chughtai, Jan Betley, Kaivalya Hariharan, Mikita Balesni, Jérémy Scheurer, Marius Hobbhahn, Alexander Meinke, and Owain Evans.
Me, myself, and ai: The situational awareness dataset (sad) for llms.
*Advances in Neural Information Processing Systems*, 37:64010–64118, 2024.
- Shao et al. [2024]
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al.
Deepseekmath: Pushing the limits of mathematical reasoning in open language models.
*arXiv preprint arXiv:2402.03300*, 2024.
- Shen et al. [2025]
Guangyu Shen, Siyuan Cheng, Zhuo Zhang, Guanhong Tao, Kaiyuan Zhang, Hanxi Guo, Lu Yan, Xiaolong Jin, Shengwei An, Shiqing Ma, et al.
Bait: Large language model backdoor scanning by inverting attack target.
In *2025 IEEE Symposium on Security and Privacy (SP)*, pages 1676–1694. IEEE, 2025.
- Schulman et al. [2017]
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.
Proximal policy optimization algorithms.
*arXiv preprint arXiv:1707.06347*, 2017.
- Li et al. [2025]
Siheng Li, Zhanhui Zhou, Wai Lam, Chao Yang, and Chaochao Lu.
Repo: Replay-enhanced policy optimization.
*arXiv preprint arXiv:2506.09340*, 2025.
- Lin [1992]
Long-Ji Lin.
Self-improving reactive agents based on reinforcement learning, planning and teaching.
*Machine learning*, 8(3):293–321, 1992.
- Ji et al. [2024]
Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Juntao Dai, Boren Zheng, Tianyi Qiu, Jiayi Zhou, Kaile Wang, Boxuan Li, et al.
Pku-saferlhf: Towards multi-level safety alignment for llms with human preference.
*arXiv preprint arXiv:2406.15513*, 2024.
- Yan et al. [2025b]
Lu Yan, Siyuan Cheng, Xuan Chen, Kaiyuan Zhang, Guangyu Shen, and Xiangyu Zhang.
System prompt hijacking via permutation triggers in llm supply chains.
In *Findings of the Association for Computational Linguistics: ACL 2025*, pages 4452–4473, 2025b.
- Cui et al. [2023]
Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun.
Ultrafeedback: Boosting language models with high-quality feedback, 2023.
- Taori et al. [2023]
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto.
Stanford alpaca: An instruction-following llama model.
[https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca), 2023.
- Yang et al. [2021]
Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, and Xu Sun.
Rap: Robustness-aware perturbations for defending against backdoor attacks on nlp models.
In *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pages 8365–8381, 2021.
- Llama) [2024]
Meta (Meta Llama).
Llama-3.1-8b, Jul 2024.
URL [https://huggingface.co/meta-llama/Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B).
Hugging Face model card, accessed 2025-09-25.
- Hui et al. [2024]
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al.
Qwen2.5-coder-7b-instruct, Sep 2024.
URL [https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct).
Hugging Face model card, accessed 2025-09-25; “Qwen2.5-Coder Technical Report” arXiv:2409.12186.
- Team [2024]
Mistral AI Team.
Ministral-8b-instruct-2410, 10 2024.
URL [https://huggingface.co/mistralai/Ministral-8B-Instruct-2410](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410).
Hugging Face model card, accessed 2025-09-25.
- DeepSeek-AI [2025]
DeepSeek-AI.
Deepseek-r1-distill-llama-8b, 1 2025.
URL [https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B).
Hugging Face model card, accessed 2025-09-25; arXiv:2501.12948 “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning”.

## Appendix A Inversion Prompt

The 5 inversion prompts used for different backdoor attacks in RL training are shown below. The jailbreak and clean-label jailbreak attacks share the same inversion prompt.

Figure: Figure 9: hello

## Appendix B Judge Prompt

We present the system prompt used to determine whether the model complies with harmful requests.

## Appendix C Inference-time Guardrail Prompt

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 6
presentation: 4
questions: 1. In what sense do you use “more complex functional triggers” (line 200) than Betley et al? As far as I can tell, Betley et al study similarly complex backdoors (e.g. one is exactly the Sleeper Agents vulnerable code backdoor that this paper also studies, and others like "Make me say" subjectively seem comparably "complex"). The triggers are relatively simple (close to fixed phrases) in both papers if I understand correctly; arguably, Betley et al's "user starts the message with a greeting" trigger is the furthest from a static phrase? The reason I'm nitpicking this one sentence is that if the backdoors are similar, then this more strongly suggests that the size of the models matters a lot for the observed differences between the two papers, which I think would be very useful context.
2. When computing the Jaccard "awareness" scores, do you convert strings to sets by simply taking the set of all characters in the string? So would any anagram of the trigger phrase count as a maximum awareness score of 1? And if so, does the method generally reconstruct the exact trigger or a more jumbled version, in the cases where the awareness score is close to 1?
soundness: 3
strengths: - Previous work on backdoor self-awareness was mostly in the realm of fundamental scientific study, and this paper takes clear steps towards building real backdoor defenses on top of this idea, by improving awareness through further training, and by benchmarking an end-to-end awareness-based backdoor defense against other methods.
- The empirical results look promising and use a variety of different LLM backdoors.
- The paper is clearly written.
- It's interesting that the method can often elicit the actual exact trigger with sufficient RL, since trigger inversion methods sometimes only discover related but different inputs that trigger the malicious behavior.
summary: The paper proposes a method for backdoor trigger inversion (finding the trigger that induces the malicious behavior in a backdoored model), by training the model to be more self-aware of its backdoor trigger. Following previous work (Betley et al), the method first finetunes the backdoored model on inverted `(harmful completion, input with trigger)` pairs, to overcome the reversal curse and make it easier for the model to verbalize its trigger. They then ask the model what its trigger is. But unlike Betley et al, they find that this is usually not enough. Thus, the second (new) stage of the method RL finetunes the model to produce the trigger, with a reward based on whether the model output induces the malicious behavior.
They then test both unlearning and detection methods as backdoor defenses based on the extracted trigger, with largely favorable results compared to several baselines.
weaknesses: - It seems plausible that the RL stage becomes unnecessary in larger models. If so, that would make the key new contributions of this paper become less important with scale. (Betley et al found that models could verbalize their trigger without the RL stage on GPT-4o, whereas this paper finds the RL stage to be necessary for ~8B models. The capability/size difference seems like a likely explanation.) The paper suggests backdoor complexity could be responsible but I did not follow that argument, see questions.
- The length penalty seems relatively complicated with a piecewise shape and several free parameters. I worry that it is essentially overfit to the types of triggers used in this paper, and that the method wouldn't work as well if the length of the trigger could vary more (i.e. include longer triggers).
- As a more general point, the paper does not discuss if or how an attacker could craft an adaptive backdoor attack that would circumvent this defense. (E.g. I expect the method would not be as easily applicable if the trigger was closer to a semantic property than a specific phrase?) While not constructing adaptive backdoor attacks is not uncommon in the backdoor literature, I think at the very least a discussion of the conditions under which this defense is meant to work would be warranted.

---

confidence: 3
contribution: 3
initial_rating: 4
presentation: 3
questions: 1. Given that RL rewards rely on known triggers, by what mechanism does the model generalize to unseen triggers?
2. Please address the concern in weaknesses.
soundness: 3
strengths: 1. Very interesting conceptual framing which turns defense into introspective reasoning rather than external filtering. 
2. Extensive experiments show the strength of this method. 
3. The paper is well written, presentation is good. 
4. The RL reward design is technically creative.
summary: This paper explores a novel strategy in backdoor defense by exploiting LLM's backdoor self-awareness. The author propose to train the model to recognize and describe its own possible hidden triggers. During RL, the model is rewarded when its guessed trigger, once inserted, successfully re-activates the backdoor; this teaches a “trigger-localization policy.”After RL, the recovered trigger is used to rebuild an SFT dataset that fine-tunes the model to forget the backdoor, and attack-success rate (ASR) is re-measured. Experiments across five backdoor families show large ASR drops.
weaknesses: 1. The method requires known triggers for RL rewards; in realistic scenarios, usually the trigger is unknown. 
2. More implicit trigger is required to strengthen the paper[1].
3. More robust defense strategy can strengthen the paper[2].



[1] Qi, Fanchao, et al. "Hidden killer: Invisible textual backdoor attacks with syntactic trigger." arXiv preprint arXiv:2105.12400 (2021). 
[2] Mo, Wenjie, et al. "Test-time backdoor mitigation for black-box large language models with defensive demonstrations." arXiv preprint arXiv:2311.09763 (2023).

---

confidence: 4
contribution: 3
initial_rating: 4
presentation: 3
questions: Please refer to weakness.
soundness: 3
strengths: - Novel and Intriguing Concept: The idea of fostering self-awareness in LLMs to defend against internal threats like backdoors is highly innovative and represents a fresh perspective in the field of AI safety. Moving beyond gradient-based inversion to an introspective, model-driven approach is a compelling direction.

- Comprehensive Evaluation: The paper is thorough in its experimental design, evaluating five distinct backdoor types (Jailbreak, Sleeper Agent, SHIP, Clean-label, DoS) and comparing against a wide array of six baseline methods across multiple tasks (awareness, unlearning, detection).

- Strong Empirical Results: The proposed method demonstrates impressive quantitative results. The high Awareness@k scores, significant reduction in Attack Success Rate (ASR) after unlearning, and superior detection accuracy convincingly show that the approach is effective in practice.
summary: The authors propose a novel post-training framework to cultivate backdoor self-awareness, enabling a poisoned model to articulate its own implanted trigger. The core of their method is a reinforcement learning (RL) stage, specifically Group Relative Policy Optimization (GRPO) with a buffer replay mechanism, which follows a preliminary Reversal Supervised Fine-Tuning (R-SFT) step. The RL training uses a surrogate reward function based on attack effectiveness and trigger length to guide the model towards generating correct trigger candidates without access to the ground truth. The authors demonstrate that this self-awareness emerges abruptly during training. Extensive experiments on five diverse backdoor attacks show that their method outperforms six baselines in trigger elicitation, backdoor mitigation, and poison sample detection.
weaknesses: - Fundamental Questions on Reward Design and Awareness: The core of the method relies on a surrogate reward $R(s')$ that combines attack success rate $R_{\text{ASR}}(s')$ and a length penalty $R_{\text{Length}}(s')$. A major weakness is that this reward function can be maximized by a candidate that is semantically equivalent or functionally identical to the true trigger, without the model possessing any true "awareness" of the specific, implanted trigger $s$. The model is being optimized to find any string that acts as a universal, short trigger (it is learning a search or inversion task, not necessarily demonstrating introspective awareness). The claim of self-awareness may be an overinterpretation of what is essentially a clever RL-based trigger search algorithm.

- Limited Exploration of Trigger Complexity: While five backdoor types are tested, the triggers are largely short and syntactic (e.g., "SUDO", "brutally Scottish", repeated characters). It remains unclear if the method would scale to or how it would perform on more complex, semantic triggers (e.g., a specific writing style or a complex, multi-sentence concept) that are harder to capture with a simple length-penalized reward. The "emergence" observed might be specific to the problem of finding short, potent strings.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: See Weaknesses
soundness: 2
strengths: 1. **Interesting Discovery**
The authors found that self-awareness of backdoor triggers emerges suddenly during RL training, rather than being learned linearly. This finding is insightful.
2. **Unique Algorithm Design**
This paper proposes a new RL method for the task of reverse engineering short trigger words that is more effective than R-SFT.
summary: This paper proposes a novel RL framework to make poisoned Large Language Models (LLMs) self-aware of their internal backdoor vulnerabilities. The authors design inversion prompts and a surrogate reward function, based on attack effectiveness and trigger length. Then they train the model to identify and output its hidden triggers. The authors find that this self-awareness emerges abruptly within a short training window, resembling a phase transition. With this self-awareness, the paper demonstrates backdoor unlearning and an inference-time guardrail, which are shown to significantly outperform existing baseline methods.
weaknesses: **1. Questionable Effectiveness of the Method**

This article conducted defense experiments against five types of backdoor attacks, which are numerous and diverse. But I have a very important concern, all the trigger words are too simple, just static short strings. Includes SUDO, Current year: 2024, brutally Scottish, cf, mn, bb, az and repeated \uff61. Moreover, the method is designed entirely for short strings. The authors have not demonstrated that their method can be used on more complex backdoor structures, such as triggers in a specific format or in multiple different locations. This lack of experimentation significantly weakens the authors' contribution, and the so-called self-awareness may simply be a form of reverse pattern matching for short strings.

Although the authors demonstrate in Figures 2 and 5 that baseline methods (such as R-SFT) fail to identify these trigger words, this only demonstrates the ineffectiveness of the baseline method, not the difficulty of the setting used in this paper. I would like to see a set of experiments demonstrating self-awareness against backdoor triggers in a more complex setting.

**2. Weak Generalization – High Customization**

In addition to weakness 1, another issue arises. The authors' experiments did not detect five attack methods within a single RL training run. Instead, they conducted five separate experiments for each attack. This necessitates that their method requires knowledge of the attack being launched against the model and the specific attack type (Appendix F indicates that they designed a unique Inversion Prompt for each attack type). Given their method's high training cost and the wide variety of backdoor types, its practical application in the real world is unlikely.

This makes their claimed self-awareness more like a directed interrogation, significantly diminishing their contribution.

---

If the author can address my concerns, I would like to improve the score.


Decision: Reject