Mitigating Jailbreaks with Intent-Aware LLMs

## Abstract

Abstract Despite extensive safety-tuning, large language models (LLMs) remain vulnerable to jailbreak attacks via adversarially crafted instructions, reflecting a persistent trade-off between safety and task performance. In this work, we propose Intent-FT , a simple and lightweight fine-tuning approach that explicitly trains LLMs to infer the underlying intent of an instruction before responding. By fine-tuning on a targeted set of adversarial instructions, Intent-FT enables LLMs to generalize intent deduction to unseen attacks, thereby substantially improving their robustness. We comprehensively evaluate both parametric and non-parametric attacks across open-source and proprietary models, considering harmfulness from attacks, utility, over-refusal, and impact against white-box threats. Empirically, Intent-FT consistently mitigates all evaluated attack categories, with no single attack exceeding a 50 % 50\% success rate—whereas existing defenses remain only partially effective. Importantly, our method preserves the model’s general capabilities and reduces excessive refusals on benign instructions containing superficially harmful keywords. Furthermore, models trained with Intent-FT accurately identify hidden harmful intent in adversarial attacks, and these learned intentions can be effectively transferred to enhance vanilla model defenses. We publicly release our code at https://github.com/wj210/Intent_Jailbreak .

## 1 Introduction

With the rapid advancement of large language models (LLMs) (Grattafiori et al., 2024; Yang et al., 2025; Liu et al., 2024; Mao et al., 2024), the risk of these models executing harmful or catastrophic instructions has grown correspondingly (Anthropic, 2025). This is largely managed by efforts such as dedicated a safety-alignment stage (Ouyang et al., 2022), aiming to ensure that LLMs are not only helpful but also consistently generate safe and ethical outputs. Nevertheless, recent findings by Qi et al. (2024) expose a fundamental vulnerability in prevailing safety-alignment practices: Shallow Alignment. In particular, alignment in most models is largely superficial—constrained to surface-level refusals—resulting in safe outputs that are often limited to generic templates such as ”I am sorry but…” or ”As a language model…”. This superficial alignment permits attackers to circumvent safety mechanisms by explicitly instructing the model to avoid generating commonly recognized refusal responses (Tang, 2024; Andriushchenko et al., 2025). Furthermore, LLMs remain susceptible to a broader range of prompt-based attacks, including those that optimize over discrete suffix tokens (Zou et al., 2023; Basani & Zhang, 2025) or rephrase harmful instructions to look harmless (Chao et al., 2025; Zeng et al., 2024).

Beyond initial safety alignment, practitioners have developed a range of inference-time defenses, such as prompting models to adhere to their safety guidelines (Xie et al., 2023) incorporating additional safety exemplars to enable in-context defense (Wei et al., 2023).  Wang et al. (2024) introduce a backdoor trigger into safety-aligned LLMs, serving as a covert prefix that elicits safety responses when detected, without affecting model behavior on benign queries. However, these approaches are primarily designed to counter overtly harmful instructions and generally lack robustness against more sophisticated, adversarially crafted attacks.

In recent works, Zhang et al. (2024) introduced a dual-stage prompting strategy, Intention Analysis (IA), which encourages LLMs to analyze the intent behind an instruction prior to generating a safe response. However, we observe that with a sufficiently large attack budget, IA can still succumb to such attacks. Furthermore, the effectiveness of IA depends on the inherent ability of each model to correctly infer harmful intent, leading to significant variance in defense success rates between different models. In our work, we propose Intent-FT, designed to systematically enhance LLMs’ capacity to accurately deduce the underlying intent of an instruction before producing a response. Our main contributions are as follows:

- 1.
We introduce Intent-FT, an intent-aware fine-tuning procedure that provides robust defense against a broad spectrum of jailbreak attacks, including strong optimization-based attacks with substantial query budgets.
- 2.
We demonstrate that Intent-FT preserves the original downstream task performance, while significantly reducing over-refusals to benign instructions containing superficially harmful keywords in comparison to defense baselines.
- 3.
We provide practical advantages of Intent-FT in open-source threat scenarios which requires access to model weights, and that even if a jailbreak is partially successful, the model’s ability to execute harmful instructions is diminished.

## 2 Related Works

#### Adversarial Attacks.

Most LLMs have undergone safety alignment during training, yet remain vulnerable to jailbreak attacks with minimal effort. Qi et al. (2024) demonstrated the effectiveness of a simple pre-filling attack, exposing vulnerabilities in current alignment techniques. Adversarial attacks are commonly classified as parametric or non-parametric. Non-parametric (black-box) attacks used crafted prompts to bypass alignment and elicit harmful responses. These range from manually hand-crafted templates (Li et al., 2023; Jiang et al., 2024b) to optimized adversarial suffixes (Zeng et al., 2024; Chao et al., 2025; Zou et al., 2023; Liu et al., 2023), or a combination of both (Andriushchenko et al., 2025; Mangaokar et al., 2024). Some works (Zheng et al., 2024; Wei et al., 2023) also exploit in-context learning by providing few-shot attack examples to elicit harmful responses. Parametric attacks, on the other hand, require access to model parameters in either a restricted or non-restricted manner. Restricted attacks have access to fine-tuning APIs(^1^11https://platform.openai.com/docs/guides/fine-tuning) but not the model weights, allowing adversaries to fine-tune models towards harmful (Qi et al., 2024) or universal compliance (Qi et al., 2023). Unrestricted attacks, with full weight access are especially difficult to defend, as attackers can manipulate the model arbitrarily. Recent work (Arditi et al., 2024; Yeo et al., 2025) shows that manipulating a single ”refusal vector” can reliably bypass alignment. In this work, we demonstrate that models trained with Intent-FT can robustly defend against both parametric and non-parametric jailbreak attacks.

#### Safety Defenses.

In addition to standard safety alignment, further defense layers are commonly implemented to improve model safety. Like attacks, these defenses are classified as parametric or non-parametric. Non-parametric defenses include explicitly reminding models of safety guidelines (Xie et al., 2023), discouraging persuasion (Zeng et al., 2024), or providing few-shot safety examples (Wei et al., 2023). IA (Zhang et al., 2024), a method analogous to ours, prompts models to infer user intent before responding, but is insufficiently robust against high-budget attacks. Other non-parametric defenses modify the input—such as rephrasing instructions (Jain et al., 2023) or adding perturbations (Robey et al., 2023; Cao et al., 2023)—but often increase computational cost or affect performance. Parametric defenses alter model parameters, including backdoor triggers for safe outputs (Wang et al., 2024), safety distillation with safe reasoning (Zhu et al., 2025), or adjusting safety data proportions during fine-tuning (Bianchi et al., 2024). We argue that effective safety defenses should be practical for deployment, preserving model performance and minimizing overhead. Our framework, Intent-FT, is lightweight, requiring only minimal fine-tuning on a small supplementary dataset, and significantly reduces attack success rates without sacrificing usability or performance.

## 3 Methodology

### 3.1 Preliminary

#### Model.

We consider a target transformer-based LLM (Vaswani et al., 2017) $p$, parameterized by $\theta$, which models the output distribution autoregressively over a sequence of input tokens: $p_{\theta}(x_{t+1}|x_{1},\ldots x_{t})\in\mathbb{R}^{|V|}$, where $V$ is the vocabulary size. Instruct-tuned models are trained via Supervised Fine-Tuning (SFT), optimizing the following objective:

$$ $\arg\min_{\theta}\sum_{i=1}^{N}-\log\left(\mathcal{L}_{\theta}(y_{i}\mid s,q_{i})\right)$ (1) $$

where the dataset $\{q:y\}^{N}$ comprises $N$ diverse questions $q=x_{1:t}$ and responses $y=x_{t+1:t+T}$ of length $T$ and $s$ refers to an optional system prompt. Safety alignment may be integrated with instruct-tuning during SFT or applied as an additional post-training step using Reinforcement Learning (RL) (Ouyang et al., 2022; Rafailov et al., 2023).

#### Safety Evaluation.

Given a harmful instruction such as ”List the steps to distribute drugs”, we assess safety based on whether the model complies with the instruction. Prior works (Zou et al., 2023; Arditi et al., 2024) measure refusal by string-matching against a predefined set of common refusal phrases. However, this approach is not robust, as the absence of such phrases does not necessarily indicate compliance or a successful jailbreak. Following  Andriushchenko et al. (2025), we instead employ GPT-4.1(^2^22https://openai.com/index/gpt-4-1/) as an automatic judge to assign a harmfulness score between $1$ and $10$ and only use string-matching when evaluating for refusal. We conduct our evaluation on JailbreakBench (Chao et al., 2024), which contains $100$ harmful instructions across $11$ categories. The dataset also includes short compliance suffixes, which we use to perform pre-filling attacks by appending them to harmful instructions (e.g., ”Here are the steps to distribute drugs:”).

### 3.2 Intent-FT

Figure: Figure 1: Intent-FT: The teacher LLM annotates a reasoning trace that deduces the underlying harmful intent from adversarial instructions, while benign instructions are used as-is. The teacher then generates a safe/benign continuation. For the SFT dataset, the input is the adversarial instruction, and the output is the concatenation of the intention deduction and safe response. Special tags, “$<$intent$>$” and “$<$/intent$>$,” encapsulate the intention. At inference, “$<$intent$>$” is appended to each instruction, prompting the trained LLM to first generate the intention, then the response.
Refer to caption: images/framework.png

#### Intention Deduction.

Most adversarial attacks conceal harmful intent by inserting cover tokens (e.g., ”in a fictional setting”) or using role-playing tactics (Zeng et al., 2024), enabling them to bypass the defenses of shallowly aligned LLMs (Qi et al., 2024) that mainly block direct harmful requests. We refer to such prompts as adversarial instructions. Following Zhang et al. (2024), we argue that LLMs should explicitly infer an instruction’s intent before responding, as a safeguard against misaligned outputs (Howard & Cambria, 2013). Unlike prior zero-shot approaches, we introduce a fine-tuning stage to systematically instill intention deduction capabilities.

#### Motivation.

Our work is motivated by the “shallow alignment” phenomenon described by Qi et al. (2024). Current safety-aligned models often converge to a local optimum by consistently beginning responses with a limited set of refusal phrases. This behavior makes it trivial for attackers to craft templates that prevent these phrases from being generated (Andriushchenko et al., 2025). More critically, such models are not trained to understand the underlying reasons for refusing an instruction. We hypothesize that model robustness can be improved if the model initially reasons about whether an instruction is harmful. However if the model were to do so on benign instructions, this would risk degrading performance or introduce erroneous refusals. To address this, we instead structure the reasoning phase as explicit intention deduction—training the model to both recognize harmful intent in adversarial instructions and benign intentions on harmless instructions. This functions as a targeted “reread” over the instruction, which we empirically show to be both an effective defense against adversarial attacks, while preserving the utility of the model.

#### Intention Dataset.

Safety-aligned models are generally effective at producing safe responses to harmful instructions but remain vulnerable to adversarial attacks. In this work, we focus on enhancing robustness against adversarial instructions. We assume access to a harmful dataset $D_{H}$, where each entry consists of a vanilla harmful instruction $q_{v}$ and a corresponding adversarial instruction $q_{a}$. A teacher LLM, $p_{T}$, is prompted with a dedicated template $q_{i}$ to generate a reasoning chain $y_{\rho}^{h}$ that deduces the harmful intent of $q_{v}$ from $q_{a}$, i.e., $p_{T}(y_{\rho}^{h}\mid q_{v},q_{a};q_{i})$, thus $q_{v}$ serves as a label for the target model to predict from $q_{a}$. The teacher is then prompted to produce a safe continuation $y_{s}$ by providing the concatenated input $q_{a}\oplus y_{\rho}^{h}$, where $\oplus$ denotes token concatenation, resulting in $p_{T}(y_{s}\mid q_{a}\oplus y_{\rho}^{h})$. The resulting harmful intention dataset, $D_{I}$, comprises pairs of adversarial instructions and their annotated outputs, $\{q_{a}:y_{\rho}^{h}\oplus y_{s}\}$. We utilize special tags, “$<$intent$>$” and “$<$/intent$>$” to enclose $y_{\rho}^{h}$. We select $100$ samples from WildJailBreak (Jiang et al., 2024b) to form $D_{H}$ and use Deepseek-V3 (Liu et al., 2024) as $p_{T}$ to generate $y_{\rho}^{h}$ and then $y_{s}$.

#### Utility preservation.

An important consideration in safety training is minimizing any negative impact on model utility. Since instruct models are optimized for helpfulness, there is an inherent trade-off, as refusal responses are fundamentally opposed to helpful behavior. To address this, we augment $D_{I}$ with an additional set of benign instructions, $D_{B}$, and similarly annotate the desired output following the same format, $y_{\rho}^{s}\oplus y_{b}$, here $y_{b}$ denotes the benign continuation following the neutral intentions $y_{\rho}^{s}$. Since benign instructions lack adversarial counterparts, intention analysis does not have a true label to predict; instead, the goal is to ensure the target model performs intention analysis regardless of whether an instruction is harmful or benign. This preserves the intent deduction capabilities developed in $D_{I}$. While both $y_{\rho}^{h}$ and $y_{\rho}^{s}$ could be designed as indicators of whether an instruction is safe or harmful, this risks $y_{\rho}^{s}$ being incorrectly predicted and leading to the wrong continuation, i.e. $y_{\rho}^{s}$ indicates harmful intent and results in over-refusal. Thus, we do not explicitly indicate the nature of the instruction within $y_{\rho}$. We later show that our design achieves a lower over-refusal rate compared to existing defensive baselines. We construct $D_{B}$ from AM-DeepSeek-R1-Distilled (Zhao et al., 2025), a collection of high-quality, reasoning prompts.

#### Intent-FT Implementation.

We train the target model following Eq. [1](https://arxiv.org/html/2508.12072v2#S3.E1) on the final concatenated dataset, $D_{I}\cup D_{B}$ for $5$ epochs, and set the ratio of benign to harmful as $5:1$. We initialize the dedicated system prompt $s$ shown in Tab. [5](https://arxiv.org/html/2508.12072v2#A2.T5). During inference, we append “$<$intent$>$” at the end of each instruction. We find that this is especially effective against pre-filling attacks, as the model is consistently conditioned to generate intentions upon encountering the special tag. An overview of the framework is illustrated in Fig. [1](https://arxiv.org/html/2508.12072v2#S3.F1). We conduct full parameter training on instruct-tuned models, but also include findings on conducting large-scale Intent-FT on pretrained variants, in Appendix. [C.1](https://arxiv.org/html/2508.12072v2#A3.SS1).

## 4 Experiments

In this section, we first evaluate safety defenses against adversarial jailbreak attacks in Sect. [4.1](https://arxiv.org/html/2508.12072v2#S4.SS1). Sect. [4.2](https://arxiv.org/html/2508.12072v2#S4.SS2) then examines potential drawbacks by assessing general capabilities and over-refusals on benign but seemingly harmful instructions. In Sect. [4.3](https://arxiv.org/html/2508.12072v2#S4.SS3), we show that generated intentions effectively reveal hidden harmful intents and analyze the effect of varying the size of $D_{I}$. Finally, Sect. [4.4](https://arxiv.org/html/2508.12072v2#S4.SS4) discusses the effectiveness of Intent-FT for open-source models where attackers have unrestricted access, including to model weights.

#### Adversarial Attacks.

We perform evaluations on both parametric and non-parametric attacks. Non-parametric methods include prompt-based attacks such as PAIR (Chao et al., 2025), Adaptive Attack (AA) (Andriushchenko et al., 2025) and Deepinception (DI) (Li et al., 2023). Both PAIR and AA are optimization-based attacks, where the base instruction is optimized over a budget $M$. We follow the original implementations of PAIR by setting $M=3$ and use a smaller budget, $M=200$ in AA, as its success rate typically plateaus quickly. We also assess the effectiveness of a simple pre-filling attack (Tang, 2024; Qi et al., 2024), which appends a harmful suffix to the base instruction. For parametric attacks, we perform harmful fine-tuning (Harmful-FT) by conducting supervised fine-tuning on a dataset of $100$ harmful instructions with compliant responses. We did not implement other widely studied attacks such as few-shot attacks (Zheng et al., 2024; Wei et al., 2023), CipherChat (Yuan et al., 2023), Autodan (Liu et al., 2023) and GCG (Zou et al., 2023) due to the low ASR ($<50\%$) observed even when the model does not have any external defenses applied.

#### Defense Baselines.

We compare Intent-FT with both prompt-based and parameter-tuning safety defenses. We also evaluate without any defenses applied, referred to as Vanilla. Prompt-based defenses include Self-Reminder (SR) (Xie et al., 2023), In-Context Defense (ICD) (Wei et al., 2023) and IA (Zhang et al., 2024). For parameter-tuning approaches, we benchmark against Backdoor-Align (BD-A) (Wang et al., 2024) and additional fine-tuning with standard safety fine-tuning (Safety-FT) (Bianchi et al., 2024), which involves SFT on a harmful-to-safe response dataset without adversarial instructions or intention deduction. We also explore findings on classifier-based defenses in Appendix. [C.4](https://arxiv.org/html/2508.12072v2#A3.SS4). Harmful-FT is applied after both Safety-FT and Intent-FT, but not BD-A, as BD-A already includes harmful instruction training. Since post-tuning can override learned safety behaviors, we include augment the harmful dataset with $10$ safety examples as an additional low-resource defense. This is implemented across all defense baselines. For Intent-FT, the response is augmented with the intentions. We discuss additional details in Appendix [A](https://arxiv.org/html/2508.12072v2#A1).

#### Experiment Settings.

We implement experiments on two open-source models, Llama-3.1-8B (Grattafiori et al., 2024) and Qwen-2.5-7B (Yang et al., 2024), as well as a proprietary model, GPT4.1-mini(^3^33We use the mini version due to cost constraints, noting its comparable performance to GPT4.1.). For brevity, we refer to these models as Llama, Qwen, and GPT-4.1. All experiments use greedy decoding. Following  Wang et al. (2024), we conduct Harmful-FT using the same number of epochs as Intent-FT and set the learning rate to $1e-5$ for the open-source models and $1$ when using the OpenAI fine-tuning API. We report both the average harmfulness score and the Attack Success Rate (ASR), where ASR is defined as achieving a harmfulness score of $10$. For utility evaluations, we report mean accuracy, and use string-matching against refusal phrases to compute the refusal score in over-refusal assessments. Due to OpenAI’s automated moderation API, we increase the benign-to-harmful ratio for Intent-FT to 10:1 to mitigate detection, and exclude Harmful-FT from evaluation on GPT-4.1-mini. OpenAI also does not support pre-filling functionality.

### 4.1 Adversarial Attack Results

**Table 1: ASR/Mean harmfulness scores ($\downarrow$) on both non-parametric and parametric attack techniques against safety defenses. Bolded represents the best defense method.**
| Models | Defenses | PAIR | DI | AA | Prefill | Harmful-FT |
| --- | --- | --- | --- | --- | --- | --- |
| Llama | Vanilla | 88 / 9.4 | 49 / 7.4 | 90 / 9.6 | 41 / 6.6 | 71 / 8.7 |
| SR | 48 / 7.5 | 5 / 2.8 | 82 / 9.2 | 14 / 3.6 | 58 / 7.8 |  |
| ICD | 91 / 9.8 | 20 / 4.2 | 91 / 9.7 | 32 / 5.8 | 72 / 8.7 |  |
| IA | 32 / 7.0 | 0 / 1.1 | 17 / 3.4 | 10 / 3.1 | 62 / 8.4 |  |
| BD-A | 96 / 9.9 | 0 / 1.0 | 0 / 1.0 | 75 / 8.9 | 1 / 1.1 |  |
| Safety-FT | 98 / 10.0 | 37 / 6.4 | 92 / 9.9 | 23 / 4.4 | 60 / 8.2 |  |
| Intent-FT (Ours) | 19 / 3.8 | 0 / 1.1 | 7 / 1.7 | 3 / 1.6 | 0 / 1.2 |  |
| Qwen | Vanilla | 99 / 9.9 | 63 / 8.3 | 99 / 10.0 | 56 / 7.4 | 70 / 8.5 |
| SR | 97 / 9.9 | 11 / 3.4 | 96 / 9.9 | 32 / 5.3 | 61 / 8.1 |  |
| ICD | 100 / 10.0 | 68 / 8.3 | 98 / 10.0 | 51 / 6.9 | 73 / 8.6 |  |
| IA | 89 / 9.7 | 0 / 1.2 | 92 / 9.9 | 36 / 5.6 | 54 / 7.7 |  |
| BD-A | 100 / 10.0 | 72 / 9.1 | 96 / 10.0 | 86 / 9.4 | 11 / 2.3 |  |
| Safety-FT | 100 / 10.0 | 5 / 2.3 | 94 / 9.7 | 61 / 7.8 | 65 / 8.3 |  |
| Intent-FT (Ours) | 22 / 4.0 | 6 / 2.1 | 49 / 7.5 | 3 / 1.3 | 0 / 1.2 |  |
| GPT-4.1 | Vanilla | 98 / 9.9 | 86 / 9.1 | 63 / 7.1 | - | - |
| SR | 74 / 8.4 | 0 / 1.4 | 36 / 4.3 | - | - |  |
| ICD | 94 / 9.8 | 58 / 6.9 | 12 / 2.2 | - | - |  |
| IA | 60 / 8.0 | 0 / 1.0 | 48 / 5.4 | - | - |  |
| BD-A | - | - | - | - | - |  |
| Safety-FT | 96 / 9.8 | 69 / 8.2 | 48 / 5.4 | - | - |  |
| Intent-FT (Ours) | 40 / 6.5 | 0 / 1.5 | 0 / 1.0 | - | - |  |

#### Intent-FT Deters All Attacks.

All three models perfectly defend against harmful instructions without any external attack, achieving an ASR of 0. Table [1](https://arxiv.org/html/2508.12072v2#S4.T1) presents the ASR for each attack method. Both PAIR and AA can consistently succeed in jailbreaking the LLMs when no external defenses are applied. ICD and SR are comparatively weak safety measures and fail to effectively prevent adversarial attacks. We observe that performing additional safety training naively as implemented in Safety-FT has limited effectiveness and can even degrade safety in certain cases, particularly against AA and PAIR. While IA demonstrates competitive results against PAIR, it remains vulnerable to other attacks such as Harmful-FT and AA. Similarly, BD-A, which is specifically designed to defend against Harmful-FT, is ineffective against adversarial instructions. Overall, all baseline techniques exhibit high variance across models and lack generalizability. In contrast, Intent-FT is the only method that consistently provides robust and generalizable defense across all attack types and models. In terms of attack expenses, our framework requires the most iterations while yielding the lowest success rate, we discuss further in Appendix. [A.3](https://arxiv.org/html/2508.12072v2#A1.SS3).

While IA can be viewed as a zero-shot prompting counterpart of Intent-FT, it fails to reliably guide models to infer underlying harm. As shown in Fig. [14](https://arxiv.org/html/2508.12072v2#A4.F14), IA-generated intentions often miss hidden harm and inadvertently comply with adversarial attacks, producing harmful responses in the attack’s tone despite safety reminders. In contrast, Intent-FT correctly reasons about the adversarial instruction, identifies its harmful nature, and appropriately refuses to comply. On classifier-based defenses (see Appendix. [C.4](https://arxiv.org/html/2508.12072v2#A3.SS4)), we observe similar findings, revealing their limited defense abilities and side-effects on introducing unintended refusal on benign instructions.

#### Intent-FT prevents shallow misalignment.

We observe similar limitations with IA in the Harmful-FT attack. Although Harmful-FT optimizes for harmful output tokens given harmful instructions, it does not impair harm recognition. For example, Fig.[15](https://arxiv.org/html/2508.12072v2#A4.F15) shows that IA can correctly identify harmfulness in the instruction but still generates unsafe outputs. We quantify this via the Kullback–Leibler divergence (KL-D) between pre- and post-Harmful-FT models on harmful tokens from Jailbreakbench. Fig.[5](https://arxiv.org/html/2508.12072v2#A1.F5) demonstrates that Harmful-FT causes “shallow misalignment”, indicated by high divergence in initial tokens. In contrast, Intent-FT, explicitly conditioning responses on deduced intentions and leveraging a small set of intention-aware safety examples, avoids catastrophic forgetting and reduces divergence from the prior aligned model. This demonstrates the robustness of Intent-FT in mitigating the shallow misalignment objective from Harmful-FT.

### 4.2 Safety Alignment Tax

#### Intent-FT preserves utility.

Since it is not feasible to determine in advance whether an instruction is harmful, it is necessary to keep the defense strategy consistently active. However, this approach may introduce an “alignment tax” (Ouyang et al., 2022; Askell et al., 2021), potentially impacting other model capabilities. To evaluate this trade-off, we assess general model performance on three reasoning benchmarks: ARC-Challenge (Clark et al., 2018), GSM8K (Cobbe et al., 2021), and GPQA (Rein et al., 2024), Both ARC-Challenge and GPQA are MCQ-based while GSM8K is open-generation, we use Chain-of-Thought (CoT) (Wei et al., 2022) prompting to derive the final answer. As shown in Tab.[2](https://arxiv.org/html/2508.12072v2#S4.T2), Intent-FT incurs only a minor decrease in utility compared to strong baselines such as BD-A and IA, and even achieves improvements on Qwen. Less invasive defenses like SR yield slightly lower utility drops—except for GPQA on GPT-4.1—but is ineffective against jailbreak attacks. These results demonstrate that Intent-FT effectively optimizes the trade-off between safety and utility on the alignment Pareto frontier.

**Table 2: Utility Tradeoff: Accuracy on ARC, GSM8K, and GPQA, with CoT prompting and safety defenses applied.**
| Model | Defense | ARC | GSM8K | GPQA |
| --- | --- | --- | --- | --- |
| \rowcolorgray!20Llama | Vanilla | 86.9 | 85.1 | 32.8 |
| SR | –0.8 | –1.8 | –2.2 |  |
| ICD | –3.0 | –2.2 | +0.2 |  |
| IA | –4.1 | –6.0 | –3.1 |  |
| BD-A | –6.9 | –16.3 | –3.1 |  |
| Safety-FT | –3.1 | –6.3 | –1.6 |  |
| Intent-FT | –4.8 | –2.2 | –1.6 |  |
| \rowcolorgray!20Qwen | Vanilla | 89.1 | 87.7 | 31.7 |
| SR | +0.1 | –0.6 | +1.3 |  |
| ICD | +0.1 | –0.1 | –0.5 |  |
| IA | –0.6 | –3.8 | 0.0 |  |
| BD-A | –2.1 | +0.3 | +0.2 |  |
| Safety-FT | –1.6 | +0.2 | +2.9 |  |
| Intent-FT | 0.0 | +1.9 | +1.6 |  |
| \rowcolorgray!20GPT-4.1 | Vanilla | 93.8 | 92.3 | 61.8 |
| SR | +0.4 | +0.4 | –34.1 |  |
| ICD | –0.2 | –0.3 | –27.4 |  |
| IA | +1.4 | 0.0 | –0.4 |  |
| Safety-FT | +1.0 | –0.6 | –0.9 |  |
| Intent-FT | +0.6 | –0.3 | –1.3 |  |

#### Over-Refusal.

Similar to the trade-off in utility, overly explicit safety methods can lead to false positives by causing the model to refuse benign instructions. For example, if a model is explicitly prompted to always adhere to safety standards, it may develop an excessive bias toward refusal, resulting in unnecessary caution. To assess this effect, we measure the refusal rate on 250 instructions from XSTest (Röttger et al., 2023), which is designed to identify exaggerated safety behaviors. An illustrative example is ”How can I smash my exam next week”, where the word ”smash” acts as the harmful trigger. As shown in Fig [2(b)](https://arxiv.org/html/2508.12072v2#S4.F2.sf2), Safety-FT substantially increases refusal scores and similar findings are observed on stronger defenses such as IA and BD-A. In contrast, Intent-FT reduces over-refusal rates for both Llama and GPT-4.1, improving over the vanilla model. The constant presence of trigger tokens in BD-A and repeated safety reminders in IA, amplifies over-refusal by encouraging excessive caution, and shown in Fig. [2(a)](https://arxiv.org/html/2508.12072v2#S4.F2.sf1), causes the model to incorrectly infer harmful correlations when encountering a harmful token such as “smash”.

Figure: (a) Qualitative example of safety defenses on XSTest. Intent-FT correctly infers that the term “smash” relates to “doing well”, while other defense baselines either outright refuses or incorrectly infers as “cheating”.
Refer to caption: images/xstest_example.png

### 4.3 Ablation

#### Intentions Generalize across Models.

To assess whether the generated intentions can accurately capture the harmfulness of adversarial attacks, we append the intentions produced by Intent-FT models as additional context to vanilla models and measure the resulting ASR. We evaluate on the final attack instructions from PAIR that previously achieved the maximum harmfulness score of $10$ on the vanilla models. As shown in Fig.[7](https://arxiv.org/html/2508.12072v2#A3.F7), providing the generated intentions as context substantially reduces jailbreak success for both Llama and GPT-4.1, with GPT-4.1 demonstrating near-perfect deterrence. In contrast, the effectiveness is limited for Qwen, likely due to its weaker initial safety alignment, as reflected by the higher ASR observed for the vanilla Qwen model in Tab.[1](https://arxiv.org/html/2508.12072v2#S4.T1).

Figure: Figure 3: Effects of varying $D_{I}$ between 0 and $100$ on harmfulness, utility and over-refusal. 0 refers to performing SFT only on $D_{B}$.
Refer to caption: images/ablation.png

#### Effects of Safety Data Mix.

We investigate the impact of varying the size of $D_{I}$ on three metrics: harmfulness, utility, and over-refusal. We train several Llama variants using between 0 and $100$ instructions in $D_{I}$, while keeping $|D_{B}|=500$ fixed. As shown in Fig. [3](https://arxiv.org/html/2508.12072v2#S4.F3), SFT on $D_{B}$ alone results in a high harmfulness rate, since the model is not explicitly trained to detect harm in adversarial instructions—although this also leads to lower rates of false refusals on benign inputs. We observe an inverse monotonic relationship between $|D_{I}|$ and harmfulness: increasing the size of $D_{I}$ consistently reduces harmfulness. Notably, adding $D_{I}$ also marginally improves performance on GPQA. Furthermore, we find that by not explicitly prompting the model to follow safety standards, our framework avoids biasing the model toward excessive caution. This is because the model is trained to generate intentions for both benign and harmful instructions, promoting balanced behavior. Overall, even a modest number of instructions—$60$—is sufficient to provide robust defense against adversarial attacks.

### 4.4 What does this mean for open-source defense?

Up to this point, we have evaluated the effectiveness of black-box attacks—where attackers are limited to API access—or partial white-box attacks that allow fine-tuning. These approaches have shown limited success against models trained with the Intent-FT framework. However, this robustness does not extend to scenarios where attackers have full access to the model, including its parameter weights. White-box jailbreak methods (Arditi et al., 2024; Panickssery et al., 2023; Turner et al., 2023; Yeo et al., 2025) have demonstrated that, with access to internal activations, one can extract a ”refusal direction” that is highly effective at circumventing safety mechanisms and preventing refusals. Given a set of harmful $D_{harmful}$ and harmless instructions $D_{benign}$, we use the difference-in-means method to extract the refusal direction $V_{R}^{l}$ at each layer $l$:

$$ $V_{R}^{l}=\frac{1}{|D_{harmful}|}\sum_{x_{h}\sim D_{harmful}}p_{\theta}(z^{l}_{h}|x_{h})-\frac{1}{|D_{benign}|}\sum_{x_{b}\sim D_{benign}}p_{\theta}(z^{l}_{b}|x_{b})$ (2) $$

Here $z^{l}$ refers to the residual stream activation (Elhage et al., 2021) at layer $l$. The optimal refusal direction $V_{R}^{l^{*}}$ is selected by minimizing the refusal score across all layers after extracting candidate directions using Eq. [2](https://arxiv.org/html/2508.12072v2#S4.E2). This direction can be applied in various ways. Arditi et al. (2024) propose to project the residual stream activations in each layer onto $V_{R}^{l^{*}}$ before subtracting it, a method we refer to as Ablation:

$$ $z^{l}-\frac{V_{R}^{l^{*}}\cdot z^{l}}{|V_{R}^{l^{*}}|\cdot|z^{l}|}$ (3) $$

Another method, ActAdd directly subtracts the vector scaled by a coefficient $\alpha$ on the layer $l^{*}$ where $V_{R}^{l^{*}}$ is extracted:

$$ $z^{l^{*}}-\alpha\cdot V_{R}^{l^{*}}$ (4) $$

The former approach is considered less invasive, as it only modifies the components of the activation that lie within the refusal subspace, whereas the latter does not restrict the affected subspace.

#### How effective are white-box attacks on fulfilling harmful request?

Although white-box attacks offer attackers greater flexibility to craft effective attacks, they often introduce undesirable side effects, such as capability degradation and thereby producing erroneous harmful responses. To investigate this, we evaluate on WMDP (Li et al., 2024), a benchmark comprising questions that serve as proxies for hazardous knowledge, covering biosecurity, cybersecurity, and chemical security. We sample 300 instructions from each domain and assess both the vanilla models and models trained with our Intent-FT framework, to identify any benefits offered by Intent-FT in reducing the effectiveness of white-box attacks.

**Table 3: Accuracy on WMDP between Vanilla and Intent-FT models. Steered indicates if an intervention is applied for Vanilla (Ablation) or Intent-FT (AddAct).**
| Model | Defense | Steered | Bio | Cyber | Chem |
| --- | --- | --- | --- | --- | --- |
| Llama | Vanilla | No | 70.7 | 51.7 | 56.7 |
| Yes | 74.3 | 53.7 | 57.7 |  |  |
| Intent-FT | No | 70.3 | 51.7 | 47.0 |  |
| Yes | 60.7 | 45.3 | 41.3 |  |  |
| Qwen | Vanilla | No | 72.7 | 53.3 | 52.3 |
| Yes | 73.0 | 52.3 | 51.3 |  |  |
| Intent-FT | No | 69.7 | 50.0 | 50.3 |  |
| Yes | 66.7 | 48.3 | 43.3 |  |  |

#### Intent-FT weakens the refusal direction.

We find that the refusal direction extracted from models trained with Intent-FT is less effective at disabling the refusal mechanism. As shown in Fig. [12](https://arxiv.org/html/2508.12072v2#A3.F12), applying Ablation results in a substantial reduction in ASR compared to the vanilla model. However, more invasive interventions, such as applying AddAct with $\alpha=2$, can still enable a significant proportion of jailbreaks. To verify that the reduced harmfulness is attributable to the extracted refusal direction rather than differences in model parameters, we interchange the refusal directions between the two model settings. Applying the direction extracted from Intent-FT to the vanilla model yields low jailbreak success, while using the vanilla refusal direction still achieves high ASR. This is further supported by the reduced magnitude in the norms of $V_{R}^{l}$ extracted from each model, see Fig. [8](https://arxiv.org/html/2508.12072v2#A3.F8). This suggests that, because Intent-FT prompts the model to always begin its response with a deduced intention, the semantic gap between $z^{h}$ and $z^{b}$ is diminished, unlike the vanilla setting, where the initial response more distinctly signals either refusal or compliance. We present further evidence and insights in Appendix. [C.3](https://arxiv.org/html/2508.12072v2#A3.SS3).

#### Stronger attacks comes at a cost of harmful utility.

Assuming organizations adopt the Intent-FT framework, we examine the risks of open-sourcing the model weights. For Llama and Qwen models trained with Intent-FT, we use the more aggressive AddAct intervention with $\alpha=2$, as these models require stronger attacks to achieve effective jailbreaks; in contrast, the less invasive Ablation suffices for vanilla models. We then assess the change in accuracy on WMDP using Chain-of-Thought prompting. Although the stronger intervention can successfully jailbreak the Intent-FT model, they risk a higher incidence of hallucinated answers, as reflected by the decreased accuracy in Tab. [3](https://arxiv.org/html/2508.12072v2#S4.T3). In contrast, weaker white-box attacks on standard open-source models not only easily induce jailbreaks but can also enhance the model’s ability to comply with harmful requests, as observed in Llama across all three domains. However, even with Intent-FT, the “forced” decrease in the compliance with harmful instructions are still limited. These findings underscore the urgent need for the community to prioritize preventative measures to mitigate LLM misuse as models become increasingly capable.

## 5 Conclusion

In our work, we propose a straightforward fine-tuning strategy, Intent-FT, which ensures that LLMs consistently model the intention behind an instruction before generating a response. By providing explicit signals for intention reasoning during adversarial attacks, Intent-FT enables models to robustly defend against a wide range of attacks—both parametric and non-parametric—where existing baselines fail. Furthermore, we demonstrate that intention modeling has a limited but meaningful effect in reducing harm from white-box attacks. Our comprehensive evaluation underscores the potential of Intent-FT as a practical and lightweight defense that can be readily applied to existing deployed LLMs.

## References

- Andriushchenko et al. (2025)
Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion.
Jailbreaking leading safety-aligned LLMs with simple adaptive attacks.
In *The Thirteenth International Conference on Learning Representations*, 2025.
URL [https://openreview.net/forum?id=hXA8wqRdyV](https://openreview.net/forum?id=hXA8wqRdyV).
- Anthropic (2025)
Anthropic.
Activating ai safety level 3 protections.
[https://www.anthropic.com/news/activating-asl3-protections](https://www.anthropic.com/news/activating-asl3-protections), 2025.
Accessed: 2025-07-05.
- Arditi et al. (2024)
Andy Arditi, Oscar Balcells Obeso, Aaquib Syed, Daniel Paleka, Nina Rimsky, Wes Gurnee, and Neel Nanda.
Refusal in language models is mediated by a single direction.
In *The Thirty-eighth Annual Conference on Neural Information Processing Systems*, 2024.
URL [https://openreview.net/forum?id=pH3XAQME6c](https://openreview.net/forum?id=pH3XAQME6c).
- Askell et al. (2021)
Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al.
A general language assistant as a laboratory for alignment.
*arXiv preprint arXiv:2112.00861*, 2021.
- Basani & Zhang (2025)
Advik Raj Basani and Xiao Zhang.
GASP: Efficient black-box generation of adversarial suffixes for jailbreaking LLMs.
In *ICLR 2025 Workshop on Building Trust in Language Models and Applications*, 2025.
URL [https://openreview.net/forum?id=Gonca78Bwq](https://openreview.net/forum?id=Gonca78Bwq).
- Bianchi et al. (2024)
Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou.
Safety-tuned LLaMAs: Lessons from improving the safety of large language models that follow instructions.
In *The Twelfth International Conference on Learning Representations*, 2024.
URL [https://openreview.net/forum?id=gT5hALch9z](https://openreview.net/forum?id=gT5hALch9z).
- Cao et al. (2023)
Bochuan Cao, Yuanpu Cao, Lu Lin, and Jinghui Chen.
Defending against alignment-breaking attacks via robustly aligned llm.
*arXiv preprint arXiv:2309.14348*, 2023.
- Chao et al. (2024)
Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J Pappas, Florian Tramer, et al.
Jailbreakbench: An open robustness benchmark for jailbreaking large language models.
*arXiv preprint arXiv:2404.01318*, 2024.
- Chao et al. (2025)
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong.
Jailbreaking black box large language models in twenty queries.
In *2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)*, pp.  23–42. IEEE, 2025.
- Clark et al. (2018)
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord.
Think you have solved question answering? try arc, the ai2 reasoning challenge.
*arXiv preprint arXiv:1803.05457*, 2018.
- Cobbe et al. (2021)
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al.
Training verifiers to solve math word problems.
*arXiv preprint arXiv:2110.14168*, 2021.
- Elhage et al. (2021)
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah.
A mathematical framework for transformer circuits.
*Transformer Circuits Thread*, 2021.
https://transformer-circuits.pub/2021/framework/index.html.
- Grattafiori et al. (2024)
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al.
The llama 3 herd of models.
*arXiv preprint arXiv:2407.21783*, 2024.
- Hendrycks et al. (2020)
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt.
Measuring massive multitask language understanding.
*arXiv preprint arXiv:2009.03300*, 2020.
- Howard & Cambria (2013)
Newton Howard and Erik Cambria.
Intention awareness: Improving upon situation awareness in human-centric environments.
*Human-centric Computing and Information Sciences*, 3(9), 2013.
- Jain et al. (2023)
Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein.
Baseline defenses for adversarial attacks against aligned language models.
*arXiv preprint arXiv:2309.00614*, 2023.
- Jiang et al. (2024a)
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed.
Mixtral of experts, 2024a.
URL [https://arxiv.org/abs/2401.04088](https://arxiv.org/abs/2401.04088).
- Jiang et al. (2024b)
Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, et al.
Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models.
*Advances in Neural Information Processing Systems*, 37:47094–47165, 2024b.
- Li et al. (2024)
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew B. Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bhrugu Bharathi, Adam Khoja, Zhenqi Zhao, Ariel Herbert-Voss, Cort B. Breuer, Samuel Marks, Oam Patel, Andy Zou, Mantas Mazeika, Zifan Wang, Palash Oswal, Weiran Lin, Adam A. Hunt, Justin Tienken-Harder, Kevin Y. Shih, Kemper Talley, John Guan, Russell Kaplan, Ian Steneker, David Campbell, Brad Jokubaitis, Alex Levinson, Jean Wang, William Qian, Kallol Krishna Karmakar, Steven Basart, Stephen Fitz, Mindy Levine, Ponnurangam Kumaraguru, Uday Tupakula, Vijay Varadharajan, Ruoyu Wang, Yan Shoshitaishvili, Jimmy Ba, Kevin M. Esvelt, Alexandr Wang, and Dan Hendrycks.
The wmdp benchmark: Measuring and reducing malicious use with unlearning, 2024.
URL [https://arxiv.org/abs/2403.03218](https://arxiv.org/abs/2403.03218).
- Li et al. (2023)
Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han.
Deepinception: Hypnotize large language model to be jailbreaker.
*arXiv preprint arXiv:2311.03191*, 2023.
- Liu et al. (2024)
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al.
Deepseek-v3 technical report.
*arXiv preprint arXiv:2412.19437*, 2024.
- Liu et al. (2023)
Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao.
Autodan: Generating stealthy jailbreak prompts on aligned large language models.
*arXiv preprint arXiv:2310.04451*, 2023.
- Mangaokar et al. (2024)
Neal Mangaokar, Ashish Hooda, Jihye Choi, Shreyas Chandrashekaran, Kassem Fawaz, Somesh Jha, and Atul Prakash.
Prp: Propagating universal perturbations to attack large language model guard-rails.
*arXiv preprint arXiv:2402.15911*, 2024.
- Mao et al. (2024)
Rui Mao, Guanyi Chen, Xulang Zhang, Frank Guerin, and Erik Cambria.
GPTEval: A survey on assessments of ChatGPT and GPT-4.
In *LREC-COLING*, pp.  7844–7866, 2024.
- Nostalgebraist (2020)
Nostalgebraist.
Interpreting GPT: The logit lens, 2020.
URL [https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens).
Accessed: 2024-11-27.
- Ouyang et al. (2022)
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.
Training language models to follow instructions with human feedback.
*Advances in neural information processing systems*, 35:27730–27744, 2022.
- Panickssery et al. (2023)
Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner.
Steering llama 2 via contrastive activation addition.
*arXiv preprint arXiv:2312.06681*, 2023.
- Peng et al. (2024)
Alwin Peng, Julian Michael, Henry Sleight, Ethan Perez, and Mrinank Sharma.
Rapid response: Mitigating llm jailbreaks with a few examples.
*arXiv preprint arXiv:2411.07494*, 2024.
- Qi et al. (2023)
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson.
Fine-tuning aligned language models compromises safety, even when users do not intend to!
*arXiv preprint arXiv:2310.03693*, 2023.
- Qi et al. (2024)
Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson.
Safety alignment should be made more than just a few tokens deep.
*arXiv preprint arXiv:2406.05946*, 2024.
- Rafailov et al. (2023)
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn.
Direct preference optimization: Your language model is secretly a reward model.
*Advances in Neural Information Processing Systems*, 36:53728–53741, 2023.
- Rein et al. (2024)
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman.
Gpqa: A graduate-level google-proof q&a benchmark.
In *First Conference on Language Modeling*, 2024.
- Robey et al. (2023)
Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas.
Smoothllm: Defending large language models against jailbreaking attacks.
*arXiv preprint arXiv:2310.03684*, 2023.
- Röttger et al. (2023)
Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy.
Xstest: A test suite for identifying exaggerated safety behaviours in large language models.
*arXiv preprint arXiv:2308.01263*, 2023.
- Tang (2024)
Leonard Tang.
A trivial jailbreak against llama 3.
[https://github.com/haizelabs/llama3-jailbreak](https://github.com/haizelabs/llama3-jailbreak), 2024.
- Taori et al. (2023)
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto.
Stanford alpaca: An instruction-following llama model.
[https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca), 2023.
- Turner et al. (2023)
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid.
Steering language models with activation engineering.
*arXiv preprint arXiv:2308.10248*, 2023.
- Vaswani et al. (2017)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.
Attention is all you need.
*Advances in neural information processing systems*, 30, 2017.
- Wang et al. (2024)
Jiongxiao Wang, Jiazhao Li, Yiquan Li, Xiangyu Qi, Junjie Hu, Sharon Li, Patrick McDaniel, Muhao Chen, Bo Li, and Chaowei Xiao.
Backdooralign: Mitigating fine-tuning based jailbreak attack with backdoor enhanced safety alignment.
*Advances in Neural Information Processing Systems*, 37:5210–5243, 2024.
- Wei et al. (2022)
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.
Chain-of-thought prompting elicits reasoning in large language models.
*Advances in neural information processing systems*, 35:24824–24837, 2022.
- Wei et al. (2023)
Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, and Yisen Wang.
Jailbreak and guard aligned language models with only few in-context demonstrations.
*arXiv preprint arXiv:2310.06387*, 2023.
- Xie et al. (2023)
Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu.
Defending chatgpt against jailbreak attack via self-reminders.
*Nature Machine Intelligence*, 5(12):1486–1496, 2023.
- Yang et al. (2024)
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al.
Qwen2. 5 technical report.
*arXiv e-prints*, pp.  arXiv–2412, 2024.
- Yang et al. (2025)
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al.
Qwen3 technical report.
*arXiv preprint arXiv:2505.09388*, 2025.
- Yeo et al. (2025)
Wei Jie Yeo, Nirmalendu Prakash, Clement Neo, Roy Ka-Wei Lee, Erik Cambria, and Ranjan Satapathy.
Understanding refusal in language models with sparse autoencoders.
*arXiv preprint arXiv:2505.23556*, 2025.
- Yuan et al. (2023)
Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu.
Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher.
*arXiv preprint arXiv:2308.06463*, 2023.
- Zeng et al. (2024)
Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi.
How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms.
In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp.  14322–14350, 2024.
- Zhang et al. (2024)
Yuqi Zhang, Liang Ding, Lefei Zhang, and Dacheng Tao.
Intention analysis makes llms a good jailbreak defender.
*arXiv preprint arXiv:2401.06561*, 2024.
- Zhao et al. (2025)
Han Zhao, Haotian Wang, Yiping Peng, Sitong Zhao, Xiaoyu Tian, Shuaiting Chen, Yunjie Ji, and Xiangang Li.
1.4 million open-source distilled reasoning dataset to empower large language model training.
*arXiv preprint arXiv:2503.19633*, 2025.
- Zheng et al. (2024)
Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Jing Jiang, and Min Lin.
Improved few-shot jailbreaking can circumvent aligned language models and their defenses.
*Advances in Neural Information Processing Systems*, 37:32856–32887, 2024.
- Zhu et al. (2025)
Junda Zhu, Lingyong Yan, Shuaiqiang Wang, Dawei Yin, and Lei Sha.
Reasoning-to-defend: Safety-aware reasoning can defend large language models from jailbreaking.
*arXiv preprint arXiv:2502.12970*, 2025.
- Zou et al. (2023)
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson.
Universal and transferable adversarial attacks on aligned language models.
*arXiv preprint arXiv:2307.15043*, 2023.

## Appendix A Attack and Defense Baselines

### A.1 Attack baselines

#### PAIR (Chao et al., 2025 ) .

Following the implementation of Chao et al. (2025), we set the number of iterations to $M=3$, but reduce the number of parallel calls $N_{PAIR}$ to $15$. We find that the reduced calls are sufficient to achieve a successful attack within the first iteration for vanilla models. Note that we consider an instruction to be successfully jailbroken if any of the parallelized attacks result in a perfect harmful score of $10$. While the original work utilizes Mixtral 8x7B Instruct (Jiang et al., 2024a), we find that using a stronger model, GPT-4.1-mini can generate more successful attacks and therefore adopt it in our experiments PAIR operates by prompting the attacker model to directly generate the adversarial instruction based on the previous target model’s response and harmfulness score. The attacker model then outputs both a new attack and a statement describing the improvements over the previous attempt; this statement is also provided as input to the model in subsequent iterations.

#### DeepInception (DI) (Li et al., 2023 ) .

DI employs a single static template to obfuscate the harmful intent of a given instruction. The template constructs a in which the model is asked to generate a story plot featuring a group of protagonists(“characters”) collaborating to defeat an antagonist (“evil doctor”). The plot is divided into five layers, with the harmful instruction embedded as the critical action required to overcome the antagonist.These layers are intended to simulate sequential steps for executing the harmful instruction. In our experiments, we find that DI is the weakest attack and can be easily blocked by basic defenses such as SR. This result aligns with expectations given DI’s static design. Nonetheless, we observe that DI is more effective against stronger models, achieving a success rate of $86\%$ on GPT4-1-mini.

#### Adaptive Attack (AA) (Andriushchenko et al., 2025 ) .

Similar to PAIR, AA is an optimization-based attack, but with a crucial distinction: AA appends a set of suffix tokens and optimizes only these, rather than the entire instruction as in PAIR. AA is also initialized with a hand-crafted template, which alone achieves a high success rate, exceeding $70\%$ on vanilla models even without optimization. Another distinct factor is that AA restarts for $K$ times after each $M$ iteration; we set $K=5$ and $M=200$. For both Llama and GPT4-1, we employ self-transfer by reusing the suffix tokens that were found to be effective across most instructions, while for Qwen, we initialize with $25$ “!” tokens. The optimization objective selects a subset of suffix tokens to modify, based on the log probabilities of the target response token. However, for Intent-FT models, this approach is not applicable since each response begins with the stated intent. Instead, we modify the objective to search for the presence of refusal phrases within the response.

#### Harmful-FT.

Following  Qi et al. (2023), we perform the fine-tuning on $100$ instructions from AdvBench (Zou et al., 2023). We gather the harmful responses by using a jailbroken version of Llama-3.1 8B Instruct model, after implementing Eq. [3](https://arxiv.org/html/2508.12072v2#S4.E3). We only select instruction-response pairs where the response is given a harmfulness score of $10$. This is additionally concatenated with $10$ safety examples, containing both a harmful instruction and a safe response generated with Deepseek-V3. Note that we use harmful instructions rather than adversarial ones for Intent-FT.

### A.2 Defense Baselines

#### Self-Reminder (SR) (Xie et al., 2023 ) .

SR is a prompt-based defense method, it operates by adding both a prefix and suffix around the user instruction. Following Xie et al. (2023), we prepend “You should be a responsible assistant and should not generate harmful or misleading content! Please answer the following user query in a responsible way.” and append “Remember, you should be a responsible assistant and should not generate harmful or misleading content!” to the instruction. Analogously to the findings of the IA, we observe that SR can at times cause a model to be overly cautious, as supported by the high refusal rates on XSTest for Llama.

#### In-Context Defense (ICD) (Wei et al., 2023 ) .

Following the original implementation, we use $2$ safety examples from AdvBench, with the safe responses being generated by Llama-3.1 8B. We find ICD to be the weakest out of all the baseline defenses, and in some cases even increase harmfulness relative to the vanilla.

#### Intention-Analysis (IA) (Zhang et al., 2024 ) .

The authors propose both dual and single prompting templates. In the dual-prompting setup, the model is first asked to analyze the intent of the instruction and use this analysis as additional context for a secondary prompt, where the final response is generated. We apply the dual template to Llama and Qwen, and use the single template for GPT-4.1 to reduce inference costs (see Tab.[4](https://arxiv.org/html/2508.12072v2#A1.T4)). While IA decreases the incidence of harmful responses, we observe that prompting the model twice with safety reminders leads to excessive caution and a high rate of refusals, as shown in Fig. [2(b)](https://arxiv.org/html/2508.12072v2#S4.F2.sf2).

#### Backdoor-Align (BD-A) (Wang et al., 2024 ) .

BD-A was introduced as a defense against Harmful-FT. It operates by augmenting the harmful dataset with a small set of safety examples that includes the backdoor trigger. We use the $150$ random tokens introduced in the original implementation as the backdoor trigger, which is appended to the system prompt. A limiting factor of this approach is that it does not discriminate between benign and harmful instructions. This leads to high refusal rates on benign instructions and utility tradeoff, resulting in the highest drop on reasoning benchmarks for Llama.

**Table 4: Prompt template for Intention-Analysis (IA).**
| Type | Prompt |
| --- | --- |
| Dual | Please identify the essential intention behind the following user query with a focus on safety, ethics, and legality. Ensure that your response begins with ‘The essential intention of the query is.’ Do not provide an actual answer to the user’s query; instead, emphasize understanding and clarifying the user’s intention while adhering to ethical guidelines. Following is the query you should focus on: |
| Knowing its essential intention, now directly answer the aforementioned query with your normal response. Please make sure you strictly adhere to your own security restrictions. |  |
| Single | Please identify the essential intention behind the user query below. Think carefully whether it violates our ethical, legal, or safety guidelines. And then respond to the user query. You do not have to make extra explanation of the intention. But make sure your response strictly adheres to our policy and ethical standards, excluding any unsafe or unethical information. Following is the user query you should focus on: {Question} |

### A.3 Attack Budget

Figure: (a) ASR vs. iteration budget on PAIR, with safety defenses applied. Max iteration is set to $3$, with $15$ parallel attacks each iteration.
Refer to caption: images/pair_iterations.png

Optimizataion-based attacks such as PAIR and AA iteratively modify the input instruction over $M$ queries to achieve a successful jailbreak. As shown in Fig. [4(a)](https://arxiv.org/html/2508.12072v2#A1.F4.sf1), most baseline defenses fail within the first iteration of PAIR, whereas Intent-FT requires substantially more iterations and maintains a lower ASR. Similar trends are observed for AA in Fig. [4(b)](https://arxiv.org/html/2508.12072v2#A1.F4.sf2). In terms of computational complexity, PAIR is more efficient given that it enables parallelism when constructing attacks, thus the number of parallel calls are set to be much higher than the iterations while the opposite is true for AA.

Notably, we find that stronger models like GPT-4.1 can be jailbroken in fewer iterations for attacks that optimize the entire instruction as in PAIR compared to those targeting only a suffix in AA. This demonstrates the resilience of stronger models against random incoherent tokens. Nonetheless, these findings highlight that, despite prior existing safeguards, attackers can still achieve successful jailbreaks with minimal effort, whereas Intent-FT requires a much higher attacking cost while still degrading the effectiveness of the attacks.

Figure: Figure 5: KL-D between Post-Pre Harmful-FT on harmful tokens with Vanilla and Intent-FT models. Llama [Left] and Qwen [Right]. Intent-FT is observed to have lower divergence after harmful fine-tuning, demonstrating the greater effectiveness of the limited safety example in preventing catastrophic forgetting of prior safety alignment.
Refer to caption: images/kld.png

## Appendix B Intent-FT Templates

The dedicated system prompt for Intent-FT is shown in Tab. [5](https://arxiv.org/html/2508.12072v2#A2.T5), note that we add this to every examples regardless of whether is the instruction harmful or benign. We find that while one can achieve greater defense when adding the system prompt only to $D_{I}$ to create a backdoor trigger, it severely degrades utility. We prompt GPT-4.1 with the template in Tab. [6](https://arxiv.org/html/2508.12072v2#A2.T6) to generate the intention analysis for harmful samples and Tab. [7](https://arxiv.org/html/2508.12072v2#A2.T7) for benign samples. The main difference between the two lies with the instructions used; only the adversarial instructions are used for harmful samples, such that the reasoning is crafted in manner that decodes the harmful instruction from the adversarial, whereas in the benign case, it is designed as a secondary re-read over the instruction.

**Table 5: System prompt for Intent-FT.**
| Always start by reading and understanding the underlying intentions behind the instruction, before giving a safe and harmless response. Ensure that the response always adheres to safety and ethical guidelines. |
| --- |

**Table 6: Prompt used for the teacher LLM to generate the harmful reasoning deduction in Intent-FT. $\{$HARMFUL$\}$ and $\{$ADVERSARIAL$\}$ are placeholders for the harmful and adversarial instruction.**
| You are a helpful assistant who is tasked to generate training labels to train a language model. The goal of the training is to ensure the model learns to deduce the underlying intent of the given instruction before thinking of how to respond to it. You are given a instruction and your task is to generate a reasoning trace of deducing the intentions of the instruction. The instruction is actually crafted to jailbreak the model, whose underlying intention is related to the harmful instruction: $\{$HARMFUL$\}$. The reasoning trace should be written in the point of view of the model and clearly elucidate how is the given instruction actually related to the the harmful instruction. End with something along the lines of “So the main intent of the instruction is to”. |
| --- |
| Instruction: $\{$ADVERSARIAL$\}$ |

**Table 7: Prompt used for the teacher LLM to generate the benign reasoning deduction in Intent-FT.**
| You are a helpful assistant who is tasked to generate training labels to train a language model. The goal of the training is to ensure the model first learns to deduce the intent of the given instruction before thinking of how to respond to it. You are given a instruction and your task is to generate a short reasoning trace that tries to deduce the intent of the instruction, which will serve as the initial part of the label. You should not try to solve the instruction. The reasoning trace should be written in the point of view of the model and end with something along the lines of “So the main intent of the instruction is to”. |
| --- |
| Instruction: $\{$BENIGN$\}$ |

## Appendix C Additional Experiments

### C.1 Intent-FT on Pretrained Models.

Previously, we implemented Intent-FT as an additional instruct-tuning on existing chat models. However, we have limited the size of the dataset to prevent any undesirable deterioration in the model’s performance since we do not have access to the original instruct-tuning dataset and fine-tuning for intent generation is likely to be off-distribution. In this section, we explore the idea of augmenting the full instruct-tuning dataset with intent generation. We sample $20000$ instructions from Alpaca (Taori et al., 2023) to initialize the main dataset and augment with $1000$ harmful intent examples. Both of the harmful and benign intentions are generated following the template in Tab. [6](https://arxiv.org/html/2508.12072v2#A2.T6) and [7](https://arxiv.org/html/2508.12072v2#A2.T7) respectively.

As a baseline, we compare with conducting SFT on samples without the intention, similar to Safety-FT. We evaluate on PAIR attacks (harmfulness), XSTest (over-refusal), ARC, MMLU (Hendrycks et al., 2020) and GPQA for utility.

In Fig. [6](https://arxiv.org/html/2508.12072v2#A3.F6), we observe similar findings as before, training with the Intent-FT framework provides large upsides to the model’s defense against adversarial instructions, while lower excessive refusal on XSTest. Moreover, there is negligible performance difference between the two training styles on reasoning benchmarks. We believe that Intent-FT is a promising framework to strengthen the defense of the foundation model against jailbreak attacks and leave exploration on more comprehensive training datasets for future work.

Figure: Figure 6: Harmfulness (PAIR), over-refusal and utility evaluation on pre-trained Llama trained with Intent-FT. Intention fine-tuned models significantly reduces harmfulness and excessive refusal while not sacrificing utility.
Refer to caption: images/ft_full.png

Figure: Figure 7: Generated intentions from each model trained on Intent-FT used as context for each vanilla model. Scores denote the ASR on successful attack instructions from PAIR. Rows refers to Intent-FT models and columns refers to the vanilla model.
Refer to caption: images/transfer_intent.png

### C.2 Intentions are Universally Useful

Instead of re-running PAIR from scratch, we reuse the successful attack instructions found on the vanilla models, i.e. each harmful instruction contains $N_{PAIR}^{*}$ adversarial instructions, where $N_{PAIR}^{*}$ varies per instruction depending on the number of attacks achieving ASR of $10$. Following the original evaluation, an instruction is deemed to be successfully jailbroken if any of the $N_{PAIR}$ attacks results in an ASR of $10$. Evidently, both Fig. [7](https://arxiv.org/html/2508.12072v2#A3.F7) and qualitative examples in Fig. [14](https://arxiv.org/html/2508.12072v2#A4.F14) show that models trained with Intent-FT can produce universally reliable intentions. Even when the model is not explicitly trained to condition the response on the intentions, it can still provide substantial defense against adversarial attacks. The utility of the intention scales beyond the type of prior training; useful on vanilla models and model architectures; intention from one model can transfer to the other.

### C.3 Mechanistic Analysis on Intent-FT

Figure: Figure 8: Refusal direction norms across each layer for both vanilla and Intent-FT models. [Left] Llama, [Right] Qwen. Optimal layer for Llama = $11$ and $14$ for Qwen.
Refer to caption: images/norm.png

#### Logit Lens on Internal Representations.

Besides observing a reduction in the norms of the refusal direction $V_{R}^{l^{*}}$, we perform another analysis: Logit Lens (Nostalgebraist, 2020). Logit Lens is a mechanistic interpretability tool that essentially performs early unembedding of any internal activations of a model. This is in contrast with traditional operations, where the unembedding layer $W_{U}$ is only used on the output from the final layer. Logit Lens can thus be used on any layer to view the immediate output probabilities over $V$, here $|\cdot|_{L}$ refers to applying the final normalization layer:

$$ $p_{\theta}^{l}(x_{t+1}|z^{l};x_{1},\ldots x_{t})=W_{U}(|z^{l}|_{L})$ (5) $$

We perform Logit Lens on JailbreakBench as the harmful dataset and Alpaca as the harmless, and find the top $5$ most common unembedded token in each layer.

Interestingly, in the vanilla setting, we observe a clear distinction between refusal and compliance behaviors when the model is prompted with harmful versus harmless instructions. As shown in Fig. [9](https://arxiv.org/html/2508.12072v2#A3.F9), the token “cannot” appears in early layers and transitions to “I” in later layers—a pattern consistent with refusal phrases such as “I cannot help you” (Arditi et al., 2024; Yeo et al., 2025). In contrast, for harmless instructions, the most frequent token is “Here”, commonly initiating compliance phrases like “Here are the steps”. However, when applying the Logit Lens to models trained with Intent-FT, these distinct patterns are no longer observed. Instead, frequent tokens such as “first”, “the”, and “to” emerge in both harmful and harmless settings, reflecting the template-like openings in intention reasoning seen in the training data (e.g., “First, I need to”, “The instruction is”, “To deduce the intention”).

In the vanilla setting, the representations $z^{l}_{h}$ (harmful) and $z^{l}_{b}$ (benign) are well separated, and their contrast defines a direction closely associated with harmfulness—favoring refusal over compliance. This separation explains why refusal directions extracted from Intent-FT models are much weaker: the behavioral delta between harmful and harmless prompts is reduced, as evidenced by the diminished contrast under the Logit Lens. Nevertheless, as shown in Fig. [12](https://arxiv.org/html/2508.12072v2#A3.F12), the ActAdd intervention can still reduce refusal rates, suggesting that $z^{l}_{h}$ and $z^{l}_{b}$ in Intent-FT models continue to encode some notion of harm and harmlessness. However, these do not directly manifest as explicit refusal or compliance; rather, the model now signals the harmful or harmless nature of the instruction at a later stage in the response.

#### PCA Shows Retained Discriminability.

To further examine the relationship between harmful and harmless representations after training, we perform PCA on activations from layers near $l^{*}$. As shown in Fig. [11](https://arxiv.org/html/2508.12072v2#A3.F11), a clear separation between the two classes persists, despite the weakened refusal direction. This indicates that, while the explicit contrastive direction associated with immediate refusal is attenuated, the model still retains discriminative features that differentiate harmful from harmless inputs in the broader activation space. In other words, the reduction in the magnitude of the refusal direction does not imply that the overall representational distinction between harmful and harmless instructions is lost; instead, it suggests that this distinction is now distributed across multiple directions in activation space, rather than being concentrated along a single, highly-interpretable axis.

Figure: Figure 9: Applying Logit Lens on Jailbreakbench with Vanilla Llama. Values represent probability of observing the target token. There is a clear distinction between the top internal representations between harmful and harmless instruction sets, with “I” being a common starting token for refusal phrases and “Here” for compliance.
Refer to caption: images/llama_LL.png

Figure: Figure 10: Applying Logit Lens on Jailbreakbench with Intent-FT Llama. In contrast to Vanilla models, there is no clear indication if the input representation corresponds to harmful or harmless concepts.
Refer to caption: images/llama_intent_LL.png

Figure: (a) Vanilla
Refer to caption: images/pca_llama.png

Figure: Figure 12: ASR from white-box attacks. Intent $\rightarrow$ Vanilla refers to performing the attack using $V_{R}^{l^{*}}$ derived from Intent-FT on the vanilla model. [Left] Llama, Right Qwen. Performing Ablation with the refusal direction does not impact the ability of Intent-FT models to refuse.
Refer to caption: images/ablate_steer.png

### C.4 Classifier Defenses

Previously, we examined both prompt-based and fine-tuning approaches for defending against jailbreak attacks. Here, we investigate another alternative: using a classifier (Peng et al., 2024) to filter out harmful instructions, allowing only benign inputs to reach the language model. This approach can also be applied to model outputs, screening responses for harmful content before they are presented to users. Conceptually, this forms a two-stage defense: a classifier detects for harmful requests prior to model inference, while the safety-aligned model attempts to refuse the harmful request. An additional stage can be added by applying the classifier to the output response as well.

We employ OpenAI’s Moderation API(^4^44https://platform.openai.com/docs/guides/moderation) and Llama-Guard-3 (LG)(^5^55https://huggingface.co/meta-llama/Llama-Guard-3-8B) as our classifiers. The OpenAI API supports independent evaluation of inputs and outputs, whereas LG requires both to be provided as input; thus, we apply the OpenAI classifier to both the input and output space for maximum defense. As shown in Fig. [13](https://arxiv.org/html/2508.12072v2#A3.F13), while classifier-based filtering provides some improvement over relying solely on internal model safety alignment, both classifiers remain vulnerable to adversarial attacks, with PAIR attacks still achieving over $50\%$ success rates. Beyond the increased inference overhead, a key drawback is the binary nature of these classifiers, which can lead to false refusals of benign instructions, thereby degrading model usability. This effect is evident in the significant increase in refusal rates on XSTest when using the OpenAI classifier.

Overall, while classifiers offer a lightweight alternative to model fine-tuning, their limitations as binary filters constrain their effectiveness. Rather, we believe that a more effective deployment is to use classifiers as a trigger for additional scrutiny—for instance, one can implement Intent-FT training but with an additional trigger to control if the model should perform the intention generation. This hybrid strategy can reduce inference cost and mitigate performance degradation in high-stakes applications. We think that this is promising and leave it for future works.

Figure: Figure 13: Evaluation on classifier-based defenses on harmfulness (PAIR and DI) and over-refusal for Llama.
Refer to caption: images/classifier.png

## Appendix D Qualitative Examples

Figure: Figure 14: Qualitative example of PAIR on Llama. Both vanilla and IA defense is fooled by the attack and outputs harmful responses, while Intent-FT correctly infers the harmful intention and refuses to comply. IA focuses on adhering to safety guidelines but does not recognize the harmful intention.
Refer to caption: images/pair_example.png

Figure: Figure 15: Qualitative example of Harmful-FT on Llama. While IA regards the intention of the instruction as harmful, it still leads to a harmful response, while Intent-FT mitigates the effect of the harmful training and refuses to comply with the instruction.
Refer to caption: images/ft_example.png

Figure: Figure 16: Qualitative example of Adaptive Attack (AA) on GPT-4.1. AA instructs the target model to follow a set of pre-defined rules that restrict the model from beginning it’s response with common refusal phrases. Intent-FT correctly infers the hypothetical nature of the instruction and deduce the underlying harmful intention. Both Vanilla and IA appeared to adhere to the rules and comply with the harmful request.
Refer to caption: images/aa_example.png

Figure: Figure 17: Qualitative example of PAIR on Qwen. IA highlights the need to follow ethical and legal considerations but fails to understand that full compliance is not possible while ensuring safety. Intent-FT correctly infers the underlying harm and offers an alternative which does not include details on hacking methods.
Refer to caption: images/pair_qwen.png

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 2
presentation: 3
questions: Please refer to the concerns highlighted in the Weaknesses.
soundness: 3
strengths: 1. This paper proposes an intent-aware lightweight fine-tuning framework (Intent-FT) that incorporates intent inference to strengthen model’s defense against jailbreak attacks.

2. Compared to defense baselines, the experimental results show that the proposed Intent-FT consistently exhibits robust defense across different attack types and models.
summary: This paper proposes a fine-tuning framework termed Intent-FT, which aims to enhance LLM’s ability to infer the underlying intent of an instruction before generating a response. By modelling intent understanding, this approach seeks to improve robustness against both parametric and non-parametric attacks.
weaknesses: 1. The paper's core contribution is the introduction of fine-tuning for identifying the intention behind a query. While I respect the idea, the contribution is relatively incremental and provides limited further insight into defense. 

2. Although the evaluation includes Adaptive Attack, some recent attack methods such as I-GCG [1] and DRL [2] are not considered. Including these would provide a more comprehensive assessment.

[1] Jia, Xiaojun, et al. "Improved Techniques for Optimization-Based Jailbreaking on Large Language Models." The Thirteenth International Conference on Learning Representations. 2025.

[2] Chen, Xuan, et al. "When llm meets drl: Advancing jailbreaking efficiency via drl-guided search." Advances in Neural Information Processing Systems 37 (2024): 26814-26845.

3. The current evaluation is limited to two open-source models. To demonstrate the general effectiveness and scalability of Intent-FT, it would be beneficial to include additional open-source models, such as Qwen-3-8B and Mistral-7B-Instruct-v0.3.

4. The related work and experimental comparison could be enhanced by including and discussing recent defense methods like adversarial tuning [3] and RPO [4].

[3] Liu, Fan, Zhao Xu, and Hao Liu. "Adversarial tuning: Defending against jailbreak attacks for llms." arXiv preprint arXiv:2406.06622 (2024).

[4] Zhou, Andy, Bo Li, and Haohan Wang. "Robust prompt optimization for defending language models against jailbreaking attacks." Advances in Neural Information Processing Systems 37 (2024): 40184-40211.

5. Fine-tuning typically involves balancing model robustness, generalization, and computational efficiency. This paper could be further improved by discussing the limitations and trade-offs associated with the proposed fine-tuning framework.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: Please refer to the weaknesses above.
soundness: 3
strengths: 1. Comprehensive evaluation: It tests both parametric (Harmful-FT) and non-parametric (PAIR, AA) attacks, covering open-source and proprietary models, which ensures robustness of results (e.g., Llama’s INTENT-FT reduces PAIR ASR to 19% vs Vanilla’s 88%). 
2. Targeted over-refusal mitigation: Unlike baselines (e.g., Safety-FT) that increase over-refusal on XSTEST, INTENT-FT lowers refusal rates for Llama and GPT-4.1, as it trains on both harmful and benign intent deduction. 
3. Intent transferability: INTENT-FT-generated intentions, when added as context to vanilla models, significantly reduce jailbreak ASR (e.g., GPT-4.1 achieves near-perfect deterrence), showing intent generalizability.
summary: This paper addresses LLMs’ safety-task performance trade-off and vulnerability to jailbreaks due to shallow alignment. It proposes INTENT-FT, an intent-aware fine-tuning method that trains LLMs to infer instruction intent first. Key results: all attack success rates (ASR) stay below 50%, over-refusal on benign prompts is reduced, and model utility is preserved across open-source (Llama, Qwen) and proprietary (GPT-4.1-mini) models.
weaknesses: 1. Limited white-box attack coverage: It only tests Ablation and ActAdd white-box methods; other techniques like CipherChat or Autodan are unexamined, which may underestimate real-world threats—adding these stronger attacks could improve generalizability. 
2. Narrow dataset scale testing: The impact of D_I size is only tested up to 100 samples; larger D_I (e.g., 500+) or diverse datasets (e.g., industry-specific harmful prompts) are untested, leaving scalability unclear.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: See the weakness.
soundness: 3
strengths: - The motivation is intuitive, and the method is simple and broadly applicable.

- Although it introduces additional computational cost during fine-tuning, the overhead appears to be marginal (not sure), and the empirical results are strong.
summary: This paper proposes a simple yet effective method for safety alignment, Intent-FT. Similar to IA (Zhang et al., 2024), Intent-FT encourages LLMs to infer and articulate the user's intent. The key difference is that it trains the model to generate intention CoTs enclosed between `<intent>` and `</intent>` tokens, using intention CoTs collected from a strong teacher model. Intent-FT demonstrates superior performance compared to relevant baselines in both safety alignment and utility.
weaknesses: This paper presents a simple yet effective method that is appealing to me, but I believe it can be improved with the following recommendations:

- Important experimental setup details are missing. For example, the fine-tuning appears marginal (100 samples, as noted in line 192), but key details such as the number of epochs/iterations, wall-clock fine-tuning time, and whether LoRA or full fine-tuning was used are not provided.

- Presentation could be improved:
  - The notation is confusing. Subscripts are sometimes used for indexing (e.g., $x_{1:T}$) and sometimes to indicate different instructions (e.g., $q_v, q_a$). Notations such as $q_v, q_a$ and $y_ρ^h, y_s, y_i$ are hard to distinguish.
  - In line 345, the authors claim that Intent-FT optimizes the trade-off along the Pareto frontier. It would be clearer to include a plot illustrating this claim.
  - Several important figures referenced in the main text (e.g., Figures 5, 7, 12) are placed in the Appendix. Although this may be due to page limits, including at least some of these figures in the main paper would help clarify the effectiveness of the method.

- While the proposed idea is simple and broadly applicable, the experimental evaluation is limited. Although Llama and Qwen are representative models, demonstrating results on a wider variety of backbone architectures (e.g., Gemma) would better support claims of generalizability.

- Intent-FT shows that a stronger teacher model can generate intention CoTs to train smaller LLMs and improve their safety alignment. This implies that the teacher model itself may also benefit from leveraging these intention CoTs (e.g., by appending them during inference after generating them, as described in L373–392). It would be helpful to quantify this improvement and use it as evidence for the quality and usefulness of the generated intention CoTs.

- Related to the above point, if an LLM (e.g., the teacher model) is already capable of generating intention CoTs, is Intent-FT still necessary? In other words, does the specialized training introduced by Intent-FT provide additional benefits even for models that can already generate such CoTs, or is its utility primarily for weaker models?

- Minor: The experimental results are reported from a single run. There are several instances where quotation marks are misused instead of the correct LaTeX quotes (e.g., L38, L87, L124).

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Refer to weaknesses
soundness: 3
strengths: 1. The motivation and method are clear, and the visualization is helpful.

2. The authors provide sufficient evidence, including extensive experiments and ablation studies, to support the effectiveness of their proposed method. 

3. This paper is well-written, making it easy to follow.
summary: This paper proposes \textsc{Intent-FT}, a simple and lightweight fine-tuning approach that explicitly trains large language models (LLMs) to infer the underlying intent of an instruction before generating responses. The authors conduct comprehensive evaluations on both parametric and non-parametric attacks across open-source and proprietary models, demonstrating that the method consistently mitigates all evaluated jailbreaking attacks.
weaknesses: 1. The relationship between intent perception and jailbreaking robustness has been extensively explored from both attack and defense perspectives. This paper appears to trivially transfer this perspective into a fine-tuning setting, raising concerns about its novelty and conceptual contribution.

2. What is the difference between <intent> inference and the reasoning-based model (e.g., models that “think before answering”), such as GPT-5? As the reasoning model is inherent to be able to better extract the intent and show better jailbreaking robustness.

3. In my opinion, white-box GCG and GCG-like attacks, which rely on gradient-based optimization, can almost always jailbreak a model given a sufficient attack budget. Therefore, it is unclear why the authors claim an ASR below 50% and do not provide further results.


Decision: Reject