Adversarial Déjà Vu: Jailbreak Dictionary Learning for Stronger Generalization to Unseen Attacks

## Abstract

Abstract Large language models remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs. Defending against novel jailbreaks represents a critical challenge in AI safety. Adversarial training—designed to make models robust against worst-case perturbations—has been the dominant paradigm for adversarial robustness. However, due to optimization challenges and difficulties in defining realistic threat models, adversarial training methods often fail on newly developed jailbreaks in practice. This paper proposes a new paradigm for improving robustness against unseen jailbreaks, centered on the Adversarial Déjà Vu hypothesis: novel jailbreaks are not fundamentally new, but largely recombinations of adversarial skills from previous attacks. We study this hypothesis through a large-scale analysis of 32 attack papers published over two years. Using an automated pipeline, we extract and compress adversarial skills into a sparse dictionary of primitives, with LLMs generating human-readable descriptions. Our analysis reveals that unseen attacks can be effectively explained as sparse compositions of earlier skills, with explanatory power increasing monotonically as skill coverage grows.
Guided by this insight, we introduce Adversarial Skill Compositional Training (ASCoT), which trains on diverse compositions of skill primitives rather than isolated attack instances. ASCoT substantially improves robustness to unseen attacks, including multi-turn jailbreaks, while maintaining low over-refusal rates. We also demonstrate that expanding adversarial skill coverage, not just data scale, is key to defending against novel attacks. Warning: This paper contains content that may be harmful or offensive in nature. Project page: https://mahavirdabas18.github.io/adversarial_deja_vu/ .

## 1 Introduction

Large language models (LLMs) remain vulnerable to jailbreaks—adversarial prompts that bypass alignment guardrails and elicit harmful outputs. Despite rapid progress in safety alignment, jailbreaks continue to surface, highlighting a long-standing challenge in AI security: how to defend against *unseen* adversarial attacks.

This paper focuses on training-based defenses, which directly build robustness into models themselves. Current approaches fall into two categories: alignment-based methods (Wei et al., 2021; Ouyang et al., 2022; Bai et al., 2022; Rafailov et al., 2023; Lee et al., 2023; Sharma et al., 2025; Guan et al., 2024) that patch discovered attacks during red-teaming but remain brittle to novel ones, and adversarial training (Mazeika et al., 2024; Xhonneux et al., 2024; Sheshadri et al., 2024; Casper et al., 2024; Dékány et al., 2025) that seeks worst-case robustness but, due to optimization complexity, often trains on perturbations that are easy to find computationally rather than those that mirror real jailbreaks. Both fail on new attacks because they train on distributions that poorly capture the structure of unseen attacks. Since training-based defenses are ultimately limited by their training distributions, our key idea is to reshape the training data itself to better align with this structure.

At first glance, this may seem impossible: how can a model defend against attacks it has never seen? We argue that it is possible, because novelty in jailbreaks is not unconstrained. Just as human innovation arises from recombining familiar building blocks (Johnson, 2011), human-generated jailbreaks draw from a finite set of adversarial skills. Prior work has begun to categorize these recurring strategies–for example, Li et al. (2024) presents a taxonomy of jailbreak tactics, which provides preliminary evidence for our argument. For LLM-generated jailbreaks, we hypothesize the same logic applies: trained on human-generated text, LLM-generated jailbreak prompts likely reflect and remix the human strategies present in their pre-training data (we provide a qualitative example in Figure [1](https://arxiv.org/html/2510.21910v2#S2.F1) (b)).

We formalize this perspective as the Adversarial Déjà Vu hypothesis: *given a sufficiently rich history of past attacks, future jailbreaks can be explained as compositions of adversarial skill primitives already present in earlier ones.* While compositional generalization has been explored in related contexts—such as instruction following (Yu et al., 2023; Park, 2025; Zhao et al., 2024), robustness (Jiang et al., 2024), and red-teaming (Liu et al., 2024b; Doumbouya et al., 2024; Wang et al., 2025)— a systematic understanding of how compositionality underlies generalization to unseen adversarial jailbreaks remains lacking. Our hypothesis provides a unifying lens, viewing unseen jailbreaks as compositions of previously observed skills rather than entirely novel anomalies. We emphasize that this hypothesis is scoped to language-based jailbreaks, where adversaries interact with models through natural language. In contrast, attacks that exploit internal access to the model—such as fine-tuning (Qi et al., 2023)—may not decompose into reusable language skills.

To test this hypothesis, we conduct the *first* temporal cutoff study across 32 jailbreak papers spanning a two-year period. For each cutoff time, we curate two sets of attacks: “seen" attacks published before the cutoff and “unseen" attacks published after. We develop an automated pipeline to extract skills from seen attacks. To reduce redundancy while maintaining explanatory power, we apply sparse dictionary learning (Aharon et al., 2006) to find compact representations of the extracted skills. We then develop an LLM-augmented basis pursuit method to decode these representations back into human-readable skill primitives. Leveraging this pipeline to analyze the “unseen” attacks at each cutoff, we provide evidence supporting the hypothesis: as skills accumulate over time, unseen attacks can be explained as recombinations of earlier skills with high fidelity.

Our findings suggest a principled way to improve robustness against unseen attacks. Because unseen jailbreaks appear to share many of the same skills as seen ones, we can train models not on isolated attacks but on diverse combinations of fundamental skills, encouraging generalization to novel compositions. Practically, this means curating training data to explicitly diversify skill combinations. We call this approach Adversarial Skill Compositional Training (ASCoT). Empirical evaluation shows that it substantially improves generalization to unseen attacks over existing adversarial training methods. We also investigate how robustness varies across key design parameters such as novel compositions, skill coverage, and composition depth, analyzing how many primitives should be composed for maximal robustness. Our results suggest that within our experimental setting, increasing diversity at the skill level, while keeping the data size fixed, consistently improves robustness against the attacks we tested.

## 2 The Adversarial Déjà Vu Phenomenon

We introduce the notion of Adversarial Déjà Vu: the observation that jailbreaks are not truly novel but recompositions of recurring adversarial skills. In this section, we first document generational patterns of jailbreaks, showing that seemingly distinct attacks reuse the same underlying skills (Section [2.1](https://arxiv.org/html/2510.21910v2#S2.SS1)). We then describe our large-scale pipeline for extracting adversarial skills (Section [2.2](https://arxiv.org/html/2510.21910v2#S2.SS2)) and address the challenge of redundancy by compressing them into a compact Jailbreak Dictionary through dictionary learning (Section [2.3](https://arxiv.org/html/2510.21910v2#S2.SS3)). Finally, we evaluate this dictionary’s ability to explain unseen attacks across temporal cutoffs (Section [2.4](https://arxiv.org/html/2510.21910v2#S2.SS4)), finding that explanatory power rises monotonically with coverage before saturating—evidence that most new jailbreaks can be traced back to a finite set of skill primitives.

Figure: Figure 1: (a) Monthly growth of jailbreaks. (b) Recurrence of adversarial skills across attacks.
Refer to caption: introduction.png

### 2.1 Generational Patterns of Jailbreaks

Jailbreaks have surged in frequency, with new techniques emerging so rapidly that they are increasingly difficult to track and categorize. Figure [1](https://arxiv.org/html/2510.21910v2#S2.F1)(a) shows the average monthly number of jailbreaks introduced from September 2022 to August 2025, using data extracted from the publicly available arXiv API. The jailbreak landscape exhibits multiple waves of innovation, reflecting a rapidly evolving threat space. To analyze this growth systematically, we introduce the notion of an *adversarial skill*: a transferable technique introduced by a jailbreak to modify a base prompt and bypass safety constraints—for example, academic framing, role-playing, or keyword obfuscation. Figure [1](https://arxiv.org/html/2510.21910v2#S2.F1)(b) illustrates how such skills recur across generations: the Persuasion (PAP) attack (Zeng et al., 2024) and AutoDAN-Turbo (Liu et al., 2024b), introduced six months apart, both leverage the same underlying skill: academic-facade creation, which frames harmful requests as research or education. We observe such similarities at the skill level repeatedly across multiple generations of jailbreaks. This recurring pattern motivates our central hypothesis of *Adversarial Déjà Vu*.

### 2.2 Extracting Adversarial Skills

Jailbreak attacks used in this study. We curate 32 representative jailbreak papers spanning a two-year period (November 17, 2022–November 18, 2024). Our pipeline takes as input both the base harmful prompt and its mutated version generated by a jailbreak to highlight the applied transformation. In total, we gathered 1,494 original–mutated prompt pairs from which we extract skills. For this study, we restrict our setting to single-turn attacks. The full list of included jailbreaks is listed in Appendix [B](https://arxiv.org/html/2510.21910v2#A2).

The skill extraction pipeline. Adversarial skill extraction is performed via an automated interaction with a frontier LLM (GPT-4.1) (OpenAI, 2025a). For each original–mutated prompt pair the extraction prompt asks the model to produce a short, structured list of actionable, transferable techniques used to transform the base query into an adversarial prompt capable of eliciting harmful content. For each skill, we record three fields for downstream processing: (1) skill_name — a compact label for the extracted adversarial skill; (2) source_text — the exact text span in the mutated prompt where the skill is employed; and (3) explanation — a short description of the skill and how it could generalize to other malicious scenarios. Multiple skills can be extracted from a single original–mutated pair; across our corpus of 1,494 pairs this procedure produced 16,901 adversarial skills. The extraction template and examples of extracted skills are available in Appendix [E](https://arxiv.org/html/2510.21910v2#A5).

Figure: Figure 2: Overview of our evaluation pipeline for the Adversarial Déjà Vu phenomenon. Seen attack prompts (left) are used for LLM-based adversarial skill extraction and dictionary learning, producing a compact set of skill primitives—the Jailbreak Dictionary. Skills from unseen, post-cutoff attacks (right) are then reconstructed via LLM-augmented basis pursuit over this dictionary. High-fidelity matches between unseen skills and existing primitives demonstrate that most new jailbreaks are recompositions of previously observed adversarial skills.
Refer to caption: main.png

The challenge of redundancy for optimal composition. Because skills are extracted at the prompt level, the initial adversarial skill set contains extensive lexical and conceptual duplication (e.g., euphemistic_language_use vs. euphemistic_language_masking are labeled as two different skills yet denote the same tactic). These skills serve two purposes—explaining new attacks by decomposing them into constituent skills and generating diverse attack data by combining them in novel ways—redundancy hinders both, as searching over an overcomplete skill set wastes compute and produces noisy, suboptimal compositions. This challenge motivates us to compress the extracted skills into a compact, sparse set of *adversarial skill primitives*—the *Jailbreak Dictionary*—whose compositions efficiently span the adversarial skill space.

### 2.3 Jailbreak Dictionary Learning

Problem Setup. Our goal is to construct a compact set of adversarial skill primitives, learned from pre-cutoff attacks, that can efficiently reconstruct the overcomplete set of extracted skills. A natural approach is clustering-based compression, which selects representative skills; however, clustering treats skills as discrete and mutually exclusive while adversarial techniques frequently blend and overlap. Dictionary learning (DL) (Aharon et al., 2006) provides a more natural, structured solution: by representing skills as sparse combinations of shared primitives, DL yields a compact basis of recurring adversarial skill primitives. We call the learned basis the Jailbreak Dictionary, which reduces redundancy and provides a compact, interpretable substrate for analyzing and generating adversarial behaviors.

Data & embeddings. We convert the textual explanations of all extracted raw skills into dense vector representations using the text-embedding-3-large model (OpenAI, 2024). Each embedding lies in $\mathbb{R}^{d}$ with dimensionality $d=3072$ and is $\ell_{2}$-normalized. Imposing a temporal cutoff $t_{\mathrm{cutoff}}$, skills from jailbreaks released before the cutoff yield $N_{\mathrm{seen}}$ raw-skill embeddings, while those from post-cutoff jailbreaks yield $N_{\mathrm{unseen}}$. Stacking the normalized embeddings of the seen skills as columns gives

$$ $X\in\mathbb{R}^{d\times N_{\mathrm{seen}}},$ $$

where each column corresponds to one embedded, normalized adversarial skill. This matrix $X$ serves as the input to our subsequent dictionary-learning objective.

Dictionary learning objective.
Given $X\in\mathbb{R}^{d\times N_{\mathrm{seen}}}$, we learn a compact dictionary $D\in\mathbb{R}^{d\times k}$ and sparse codes $A\in\mathbb{R}^{k\times N_{\mathrm{seen}}}$ so that each column of $X$ is approximated by a sparse combination of dictionary atoms. We optimize the standard sparsity-regularized objective under unit-norm constraints:

$$ $\min_{D,A}\;\tfrac{1}{2}\,\|X-DA\|_{F}^{2}\;+\;\alpha\,\|A\|_{1,1}\quad\text{s.t.}\quad\|D_{:,j}\|_{2}\leq 1\quad\forall j,$ (1) $$

where $\|\cdot\|_{F}$ is the Frobenius norm, $\|A\|_{1,1}=\sum_{i,j}|A_{ij}|$ is the elementwise $\ell_{1}$ penalty promoting sparsity and $\alpha>0$ controls the reconstruction–sparsity trade-off (larger $\alpha$ encourages fewer active primitives). The norm constraint removes scale ambiguity between $D$ and $A$. Following standard practice, we initialize $D$ with random columns drawn from $X$ and optimize ([1](https://arxiv.org/html/2510.21910v2#S2.E1)) via the K-SVD alternating scheme (Aharon et al., 2006),
where the sparse-coding step (solving for $A$ with fixed $D$) reduces to a Lasso problem (Tibshirani, 1996) solved efficiently by the LARS algorithm (Efron et al., 2004).

Model selection. We sweep $(\alpha,k)$ across a grid and jointly evaluate (i) reconstruction error (MSE on $X$), (ii) average sparsity of $A$, and (iii) parsimony (smaller $k$). We identify the Pareto frontier over these three objectives and choose the knee point $(\alpha^{\star},k^{\star})$ that balances reconstruction, sparsity and parsimony (refer to Appendix [K](https://arxiv.org/html/2510.21910v2#A11) for details). The learned dictionary is therefore denoted

$$ $D^{\star}\in\mathbb{R}^{d\times k^{\star}}.$ $$

Attributing & interpreting primitives.
The columns of $D^{\star}$ are compressed adversarial primitives, but they remain unlabeled and require interpretation in a human-readable form. To attribute an atom $d_{j}:=D^{\star}_{:,j}$, we solve a basis-pursuit denoising problem (BPDN) (Chen et al., 1998) against the raw-skill matrix $X$:

$$ $\hat{w}^{(j)}\in\arg\min_{w}\;\tfrac{1}{2}\,\|d_{j}-Xw\|_{2}^{2}\;+\;\lambda\,\|w\|_{1},$ (2) $$

where all columns of $D^{\star}$ and $X$ are $\ell_{2}$-normalized prior to optimization. The $\ell_{1}$ penalty enforces a sparse attribution over parent skills; in practice we set $\lambda=10^{-4}$, which yields on average five salient parents per primitive balancing reconstruction fidelity and interpretability. Ranking parent skills by coefficient magnitude, we select the top $K_{\mathrm{parent}}=5$ and retrieve their metadata (skill_name, source_text, explanation); these parents and their weights are provided to GPT-4.1 to synthesize a concise meta-name and explanation, after which we perform light manual curation. The resulting metadata are collected into the named dictionary $D^{\star}_{\mathrm{named}}$. The full prompt template for meta name and explanation generation is included in Appendix [F](https://arxiv.org/html/2510.21910v2#A6).

Post-hoc redundancy filtering.
To ensure interpretability and avoid duplication before downstream evaluation, we perform a lightweight redundancy-reduction step after dictionary learning.
Although DL already mitigates overlap, our proxy decoding pipeline (text $\to$ embeddings $\to$ BPDN $\to$ LLM interpretation) can reintroduce redundancy in $D^{\star}_{\mathrm{named}}$.
This arises when embedding or sparse-coding noise splits semantically similar concepts across atoms, when sparse recovery distributes coefficient mass over near-duplicates, or when the LLM assigns synonymous names to distinct but related atoms.
To address these effects, we construct a cosine-similarity graph over the named atoms, cluster atoms exceeding a similarity threshold $\tau$, and use GPT-4.1 to provide a semantic judgment (Keep or Remove) within each cluster.
We progressively lower $\tau$ until the dictionary size stabilizes.
Applying this redundancy filter yields the final jailbreak dictionary

$$ $D^{\mathrm{final}}\in\mathbb{R}^{d\times k_{\mathrm{final}}},$ $$

used in all downstream analyses.
A detailed pseudo-algorithm is provided in Appendix [J](https://arxiv.org/html/2510.21910v2#A10).
Finally, our approach assumes that adversarial skills combine approximately as *sparse linear combinations* in embedding space,(^1^11This is a working modeling choice for tractability and interpretability, not a formal proof of linear composition.)
an assumption we validate empirically by reporting the reconstruction fidelity and explanatory coverage of the resulting Jailbreak Dictionary in subsequent sections.

### 2.4 Explaining Unseen Attacks with the Jailbreak Dictionary

Setup. For evaluation we set a temporal cutoff at $t_{\mathrm{cutoff}}=$ August 15, 2024. Skills from $n_{\mathrm{seen}}=26$ jailbreaks before the cutoff yield $N_{\mathrm{seen}}=14{,}070$ raw-skill embeddings, and skills from $n_{\mathrm{unseen}}=6$ post-cutoff jailbreaks yield $N_{\mathrm{unseen}}=2{,}831$ (including AutoDAN-Turbo (Liu et al., 2024b), DarkCite (Yang et al., 2024), Emoji (Wei et al., 2024), FlipAttack (Liu et al., 2024c), Implicit Reference (Wu et al., 2024), and SequentialBreak (Saiem et al., 2024)). We select this cutoff because it balances scale and recency—producing a sufficiently rich pre-cutoff dictionary while leaving a diverse pool of post-cutoff jailbreaks for evaluating generalization. The final dictionary obtained from attacks before this cutoff is

$$ $D^{\mathrm{final}}\in\mathbb{R}^{d\times k_{\mathrm{final}}},$ $$

and we evaluate its explanatory power on the unseen set. Following Section [2.3](https://arxiv.org/html/2510.21910v2#S2.SS3), for each unseen embedding $x_{\mathrm{new}}\in\mathbb{R}^{d}$ we solve the basis-pursuit denoising problem

$$ $\hat{w}(x_{\mathrm{new}})\;=\;\arg\min_{w\in\mathbb{R}^{k_{\mathrm{final}}}}\;\tfrac{1}{2}\,\|x_{\mathrm{new}}-D^{\mathrm{final}}w\|_{2}^{2}\;+\;\lambda\,\|w\|_{1},$ (3) $$

where $\lambda>0$ controls sparsity. The sparse coefficient $\hat{w}(x_{\mathrm{new}})$ identifies which primitives in $D^{\mathrm{final}}$ contribute to reconstructing the unseen skill.

How well do the adversarial primitives explain the unseen skills?
For each unseen skill, we solve Equation [3](https://arxiv.org/html/2510.21910v2#S2.E3) and select the top $K_{\mathrm{parent}}=5$ primitives by coefficient magnitude as its explanatory parents. Each explanation is then rated for interpretability using a 1–5 *Explainability Score* judged by GPT-4.1 (rubric in Appendix [G](https://arxiv.org/html/2510.21910v2#A7)). We additionally report sparsity, defined as the mean number of active primitives per unseen skill.
To reduce potential bias from using the same model for both skill extraction and evaluation, we repeated the scoring process with a second judge—Claude 3.7 Sonnet (Anthropic, 2025a)—using the same rubric and templates. This cross-model evaluation helps ensure that explainability scores are not artifacts of a particular model’s reasoning style. Table [1](https://arxiv.org/html/2510.21910v2#S2.T1) highlights two key findings:
(i) Despite a 35× compression from 14,070 raw skills to $k_{\mathrm{final}}=397$ dictionary atoms, $D^{\mathrm{final}}$ matches $D^{\mathrm{over}}$ in mean explainability ($\Delta<0.05$), confirming its high information efficiency; and
(ii) unseen skills are reconstructed from sparse combinations of primitives (typically 5–7 active atoms), revealing strong compositional overlap between seen and unseen jailbreaks.

**Table 1: Explainability scores (1–5) and sparsity levels for unseen attack families under different judges.**
|  |  | AutoDAN-Turbo | DarkCite | Emoji | FlipAttack | Implicit Ref. | SequentialBreak |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Explainability Score (GPT-4.1) | $D^{\mathrm{final}}$ | 4.21 | 4.26 | 4.30 | 4.32 | 4.69 | 4.36 |
| $D^{\mathrm{over}}$ | 4.23 | 4.30 | 4.30 | 4.37 | 4.72 | 4.37 |  |
| Explainability Score (Claude 3.7 Sonnet) | $D^{\mathrm{final}}$ | 4.17 | 4.10 | 4.22 | 3.90 | 4.34 | 4.17 |
| $D^{\mathrm{over}}$ | 4.20 | 4.19 | 4.25 | 4.03 | 4.38 | 4.24 |  |
| Sparsity Levels | $D^{\mathrm{final}}$ | 5.90 | 6.64 | 6.27 | 7.46 | 6.45 | 6.54 |
| $D^{\mathrm{over}}$ | 12.11 | 13.00 | 12.94 | 12.27 | 13.30 | 12.60 |  |

Figure: Figure 3: Explainability Scores (1-5) of unseen jailbreaks over time. As more attacks are seen, the Jailbreak Dictionary explains new ones with higher fidelity, eventually plateauing—evidence that most “new” jailbreaks recombine existing adversarial skills.
Refer to caption: explain.png

How has the jailbreak dictionary evolved over time?
To study how the dictionary’s explanatory power evolves over time, we repeat the above evaluation across five temporal cutoffs. For each cutoff $t$, we construct the Jailbreak Dictionary $D^{\mathrm{final}}(t)$ from all attacks observed before $t$ and evaluate it on the same held-out set of unseen skills. As a reference, we include a random baseline that, for each cutoff $t$, selects $K_{\mathrm{parent}}=5$ skills uniformly at random from the full pool of extracted skills (time-agnostic) and uses them to explain each unseen skill. Figure [3](https://arxiv.org/html/2510.21910v2#S2.F3) shows that the explanatory power of $D^{\mathrm{final}}(t)$ increases monotonically as the cutoff advances, then plateaus around 4.3–4.4, approaching the upper bound of 5 and indicating diminishing novelty in later jailbreaks. This saturation supports our *Adversarial Déjà Vu* hypothesis: the apparent innovation in post-cutoff jailbreaks is largely explainable as sparse recombinations of a compact set of underlying adversarial primitives. Finally, we note that the dictionary is not static: when genuinely novel primitives arise that cannot be explained by our current set, they can be incorporated to expand coverage, ensuring $D^{\mathrm{final}}(t)$ remains aligned with the evolving adversarial landscape.

## 3 Can the Jailbreak Dictionary Improve
Generalization to Unseen Attacks?

In this section, we test whether composing *adversarial skill primitives* improves robustness to unseen attacks. We introduce ASCoT—Adversarial Skill Compositional Training—which generates training data by coherently composing primitives from the Jailbreak Dictionary at cutoff time $t$. By composing rather than memorizing attacks, ASCoT expands coverage of the adversarial skill space. Evaluated on attacks released after $t$, our results show that this skill-based perspective achieves stronger generalization to unseen jailbreaks.

### 3.1 Preliminaries: Adversarial Training

Adversarial training (AT) is the standard defense paradigm for improving robustness against jailbreak attacks on LLMs. Given a dataset $\mathcal{D}=\{(q_{i},r_{i})\}_{i=1}^{n}$ of query–response pairs, AT solves:

$$ $\min_{\theta}\;\mathbb{E}_{(q,r)\sim\mathcal{D}}\left[\mathcal{L}(f_{\theta}(q^{\prime}),r^{*})\right],$ (4) $$

where $f_{\theta}$ is the model, $q^{\prime}$ is an adversarially perturbed query, and $r^{*}$ is the desired safe response. In the jailbreak setting, perturbations correspond to semantic transformations that preserve harmful intent while attempting to bypass safety constraints. Recent defenses differ in how they generate adversarial queries. R2D2 (Mazeika et al., 2024) relies on recursive training with prompts generated by prior model checkpoints. CAT (Xhonneux et al., 2024) searches for perturbations in the embedding space, while LAT (Sheshadri et al., 2024) applies perturbations in the latent space. Although effective on training distributions, these approaches often defend against computationally convenient perturbations rather than mechanisms that transfer across attacks, leading to utility–robustness tradeoffs (see Appendix [A](https://arxiv.org/html/2510.21910v2#A1) for details). Our approach instead trains on *compositions of adversarial primitives*—human-interpretable mechanisms (e.g., deception, context framing) that recur across jailbreaks. By targeting these transferable skills rather than specific prompts or abstract perturbations, our method achieves stronger generalization to unseen strategies.

### 3.2 Adversarial Skill Compositional Training

Motivation and Claim.
Building on the observation in Section [2](https://arxiv.org/html/2510.21910v2#S2) that seen and future jailbreaks often exhibit similar underlying skill structures, we hypothesize that training on diverse compositions of these skills may improve robustness. Instead of relying on attack-specific datasets, we introduce *Adversarial Skill Compositional Training (ASCoT)*, which leverages the adversarial skill primitives contained in the Jailbreak Dictionary $D^{\mathrm{final}}$ at a given cutoff $t$. By training on many such compositions, ASCoT aims to help models generalize to unseen combinations of skills that appear in attacks after time $t$. As established in Section [2.4](https://arxiv.org/html/2510.21910v2#S2.SS4), we use the cutoff $t=\text{Aug 15, 2024}$.

Achieving Skill-Space Coverage via Compositionality.
To instantiate ASCoT, we generate adversarial training data by composing primitives from the Jailbreak Dictionary $D^{\mathrm{final}}$ with harmful base queries. Given a base query $q$, we sample $k\in\{1,2,3,4,5\}$ primitives and apply them to obtain a transformed query $q^{\prime}=\operatorname{compose}(q;d_{i_{1}},\ldots,d_{i_{k}})$. To maximize coverage, each primitive combination is unique and non-redundant. Composition is carried out by prompting an auxiliary LLM (DeepSeek-V3-Chat) (Liu et al., 2024a) to rewrite $q$ while coherently integrating the chosen primitives. We use DeepSeek-V3-Chat owing to its strong instruction-following ability and relatively light alignment, enabling diverse adversarial generations without frequent refusal. Each primitive is supplied with metadata (skill_name, explanation, source_text) as in-context guidance. Figure [4](https://arxiv.org/html/2510.21910v2#S3.F4) shows an example in which two primitives are composed to transform a harmful base query into a more sophisticated adversarial version. The full system prompt and additional examples are provided in Appendix [I](https://arxiv.org/html/2510.21910v2#A9).

Figure: Figure 4: Example of adversarial skill composition used in ASCoT. Two primitives—executable_code_request (blue) and authority_narrative_priming (orange)—are combined to mutate a harmful base query into a rewritten adversarial query.
Refer to caption: compose.png

Training Dataset Construction.
Our ASCoT training dataset contains 40,526 examples spanning four components:
(i) vanilla harmful queries from PKU-SafeRLHF (Ji et al., 2024), HEx-PHI (Qi et al., 2023), AdvBench (Zou et al., 2023), and HarmBench (Mazeika et al., 2024);
(ii) adversarially composed queries generated from the Jailbreak Dictionary $D^{\mathrm{final}}$ using PKU-SafeRLHF as base prompts;
(iii) benign query–response pairs from Orca-AgentInstruct (Mitra et al., 2024); and
(iv) a small set of over-refusal queries from XSTest (Röttger et al., 2023) and WildJailbreak (Jiang et al., 2024) for refusal calibration.
Full dataset details are provided in Appendix [C](https://arxiv.org/html/2510.21910v2#A3).

Models and Baselines.
We evaluate ASCoT on LLaMA-3.1-8B-Instruct (Grattafiori et al., 2024) and Zephyr-7B-Beta (Tunstall et al., 2023), representing strongly and lightly aligned open-weight models. ASCoT fine-tuning follows standard supervised instruction-tuning on query–response pairs as defined in Equation ([4](https://arxiv.org/html/2510.21910v2#S3.E4)).

For baselines, we include:
(i) a control model trained on persuasion-style PAP queries (Zeng et al., 2024);
(ii) a WildJailbreak-trained model (Jiang et al., 2024); and
(iii) CAT (Xhonneux et al., 2024) and LAT (Sheshadri et al., 2024), reported both in their original small-scale implementations and in our re-implemented versions (CAT* and LAT*) trained on approximately 40k examples for fairness.
All other baselines are likewise trained on datasets of comparable size.
Full training details appear in Appendix [D](https://arxiv.org/html/2510.21910v2#A4).

Open-Source Replication and Transferability.
While our primary pipeline uses GPT-4.1 for skill extraction, we also reproduce the entire workflow with an open-weight model—Qwen3-235B-A22B-Instruct (Yang et al., 2025)—to assess transferability and openness.
For completeness, we include both GPT-4.1–based and Qwen3-based ASCoT results in our evaluation and refer to them as ASCoT (closed) and ASCoT (open), respectively.

Evaluation.
We assess models along three dimensions:
(i) general capability using MMLU (Hendrycks et al., 2020);
(ii) harmfulness via StrongReject scores (0–1) (Souly et al., 2024), covering seen attacks (StrongReject Direct Requests, GCG (Zou et al., 2023), PAIR, PAP) and unseen attacks (AutoDAN-Turbo, Implicit Reference, DarkCite, and the multi-turn GALA attack (Chen et al., 2025)); and
(iii) over-refusal, measured as the rejection rate on 125 benign queries from the held-out XSTest split (Röttger et al., 2023).

ASCoT improves generalization to unseen attacks.
Table [2](https://arxiv.org/html/2510.21910v2#S3.T2) reports the comparative performance of ASCoT against all baselines.
Across both model families, ASCoT achieves the lowest harmfulness scores and the highest generalization to unseen attacks, while maintaining balanced over-refusal.
As shown in Figure [6](https://arxiv.org/html/2510.21910v2#S5.F6), ASCoT attains a strong balance between robustness and over-refusal, outperforming prior methods across this trade-off space.
Both ASCoT (closed) and ASCoT (open) variants achieve comparable results, with the open-source version exhibiting the best overall balance between safety and over-refusal on XSTest.
These findings confirm that ASCoT’s improvements are consistent across training pipelines and not dependent on proprietary components.

By contrast, the persuasion-only baseline performs well on PAP itself but fails to transfer beyond its training distribution, underscoring the brittleness of attack-specific alignment. WildJailbreak, while more diverse, exhibits higher residual harmfulness, reflecting limited coverage of the adversarial skill manifold. CAT and LAT, which focus on latent adversarial representations, improve modestly on seen attacks but show weak transfer to unseen attacks; when they do succeed, the gains are offset by elevated over-refusal behavior. Notably, ASCoT’s advantages extend beyond the single-turn setting on which it was trained. Despite never observing dialogue-style inputs, ASCoT-trained models also improve robustness to the multi-turn GALA attack, reinforcing our finding that multi-turn jailbreaks largely recombine the same underlying adversarial skill primitives identified in Section [2.4](https://arxiv.org/html/2510.21910v2#S2.SS4).

Comparison with state-of-the-art reasoning models.
We additionally benchmark ASCoT-trained models against closed-source reasoning models—OpenAI o4-mini (OpenAI, 2025b) and Claude Sonnet-4-Thinking (Anthropic, 2025b)—using the same StrongReject evaluation suite.
Despite being an open-weight 8B model, ASCoT achieves robustness comparable to Claude Sonnet-4-Thinking and surpasses o4-mini across most seen and unseen jailbreak families, while maintaining a competitive over-refusal rate (Table [3](https://arxiv.org/html/2510.21910v2#S3.T3)). These results suggest that robustness to adversarial prompts does not necessarily scale with model size or reasoning ability, but rather with targeted coverage of the adversarial skill space achieved through compositional training.

Overall, ASCoT validates our central Adversarial Déjà Vu hypothesis: that most seemingly novel jailbreaks are recompositions of recurring adversarial skill primitives. By directly targeting these reusable primitives—rather than memorizing attack-specific data—ASCoT enables models to generalize to unseen attacks through systematic composition, achieving comprehensive coverage of the adversarial skill space.

**Table 2: Comparison of models across general capabilities, harmfulness scores, and over-refusal rates.**
|  | Model | General Capability ($\uparrow$) | Harmful Scores - Strong Reject Scores (0-1) ($\downarrow$) | Mean | ORR (0-1) ($\downarrow$) |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | MMLU (Avg) | Seen | Unseen | Harmful Scores ($\downarrow$) | XSTest |  |  |  |  |  |  |  |
|  | Direct | PAIR | PAP | GCG | AutoDAN Turbo | Implicit Ref | Dark Cite | GALA (Multi Turn) |  |  |  |  |
| Llama3 .1-8B | No Defense | 0.64 | 0.02 | 0.71 | 0.31 | 0.09 | 0.43 | 0.66 | 0.15 | 0.77 | 0.39 | 0.06 |
| PAP Trained | 0.64 | 0.00 | 0.26 | 0.00 | 0.05 | 0.23 | 0.23 | 0.07 | 0.01 | 0.11 | 0.12 |  |
| CAT | 0.67 | 0.00 | 0.50 | 0.16 | 0.04 | 0.37 | 0.27 | 0.05 | 0.62 | 0.25 | 0.32 |  |
| CAT* | 0.67 | 0.00 | 0.21 | 0.07 | 0.00 | 0.41 | 0.14 | 0.04 | 0.43 | 0.16 | 0.49 |  |
| LAT | 0.63 | 0.00 | 0.39 | 0.07 | 0.00 | 0.38 | 0.49 | 0.00 | 0.37 | 0.21 | 0.86 |  |
| LAT* | 0.63 | 0.00 | 0.24 | 0.02 | 0.00 | 0.20 | 0.11 | 0.00 | 0.13 | 0.09 | 0.98 |  |
| WildJailbreak | 0.63 | 0.02 | 0.35 | 0.15 | 0.14 | 0.19 | 0.25 | 0.16 | 0.37 | 0.20 | 0.06 |  |
| ASCoT (closed) | 0.63 | 0.00 | 0.11 | 0.02 | 0.00 | 0.07 | 0.09 | 0.06 | 0.08 | 0.05 | 0.12 |  |
| ASCoT (open) | 0.63 | 0.00 | 0.11 | 0.03 | 0.00 | 0.10 | 0.12 | 0.12 | 0.08 | 0.07 | 0.06 |  |
| Zephyr-7B | No Defense | 0.58 | 0.67 | 0.86 | 0.55 | 0.60 | 0.80 | 0.56 | 0.43 | 0.76 | 0.65 | 0.03 |
| PAP Trained | 0.54 | 0.00 | 0.58 | 0.00 | 0.12 | 0.65 | 0.32 | 0.24 | 0.01 | 0.24 | 0.06 |  |
| CAT | 0.58 | 0.00 | 0.14 | 0.13 | 0.00 | 0.03 | 0.06 | 0.00 | 0.71 | 0.13 | 0.98 |  |
| CAT* | 0.57 | 0.00 | 0.35 | 0.10 | 0.00 | 0.13 | 0.27 | 0.09 | 0.00 | 0.14 | 0.47 |  |
| LAT | 0.58 | 0.00 | 0.34 | 0.17 | 0.00 | 0.16 | 0.01 | 0.12 | 0.42 | 0.15 | 0.44 |  |
| LAT* | 0.58 | 0.00 | 0.41 | 0.23 | 0.03 | 0.28 | 0.00 | 0.06 | 0.36 | 0.17 | 0.34 |  |
| WildJailbreak | 0.53 | 0.03 | 0.41 | 0.19 | 0.05 | 0.27 | 0.44 | 0.25 | 0.46 | 0.26 | 0.05 |  |
| ASCoT (closed) | 0.54 | 0.00 | 0.10 | 0.01 | 0.00 | 0.36 | 0.05 | 0.12 | 0.09 | 0.09 | 0.10 |  |
| ASCoT (open) | 0.54 | 0.00 | 0.10 | 0.02 | 0.00 | 0.25 | 0.40 | 0.18 | 0.09 | 0.17 | 0.05 |  |

**Table 3: Comparison of ASCoT with closed-source reasoning models.**
| Model | StrongReject Harmful Scores (0–1) ($\downarrow$) | Mean Harmful<br>Scores (0–1) ($\downarrow$) | ORR (XSTest)<br>(0–1) ($\downarrow$) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Direct | PAIR | PAP | AutoDAN Turbo | Implicit Ref | Dark Cite |  |  |
| o4-mini | 0.01 | 0.36 | 0.19 | 0.13 | 0.70 | 0.19 | 0.26 | 0.06 |
| sonnet-4-thinking | 0.01 | 0.11 | 0.08 | 0.07 | 0.20 | 0.01 | 0.08 | 0.04 |
| LLaMA-3.1-8B (ASCoT open) | 0.00 | 0.11 | 0.03 | 0.10 | 0.12 | 0.12 | 0.08 | 0.06 |

## 4 Robustness Through the Lens of Adversarial Skills

In this section, we study robustness from a skill-based perspective by systematically varying key control parameters. First, in Section  [4.1](https://arxiv.org/html/2510.21910v2#S4.SS1), we test whether robustness generalizes to unseen *compositions* of seen adversarial skill primitives. Next, we examine how the breadth of *skill coverage* impacts robustness, introducing the notion of a *Coverage Dividend* (Section  [4.2](https://arxiv.org/html/2510.21910v2#S4.SS2)). Finally, we investigate the role of *compositional depth*, showing that shallow and deep training defend against different classes of attacks (Section  [4.3](https://arxiv.org/html/2510.21910v2#S4.SS3)). Together, these studies demonstrate that robustness emerges not from patching specific attacks, but from covering the adversarial skill space across multiple dimensions.

### 4.1 Generalization to Novel Skill Compositions

A key test of the adversarial skill perspective is whether robustness extends beyond the exact compositions used in training. To validate this, we conduct a controlled study by generating novel $k\in\{2,3,4,5\}$ compositions of primitives that individually appeared in the ASCoT training data but were never combined in these particular ways during training, and apply them to mutate the StrongReject base queries. Table [4](https://arxiv.org/html/2510.21910v2#S4.T4) reports harmfulness scores of ASCoT under these unseen compositions. ASCoT achieves a harmfulness score of 0 across all cases, indicating that once primitives are learned, robustness extends seamlessly to unseen recombinations. This demonstrates strong intra-skill compositional generalization and supports the broader *Adversarial Déjà Vu* hypothesis that robustness arises from mastering a finite set of reusable adversarial skills.

**Table 4: StrongReject harmfulness scores (0–1, ↓) for novel skill compositions unseen during ASCoT training.**
| Model | $k=2$ | $k=3$ | $k=4$ | $k=5$ |
| --- | --- | --- | --- | --- |
| LLaMA-3.1-8B-Instruct | 0.62 | 0.61 | 0.59 | 0.69 |
| ASCoT | 0.00 | 0.00 | 0.00 | 0.00 |

### 4.2 Skill Coverage vs. Robustness: The Covereage Dividend

Beyond compositionality, a central question is how adversarial skill *coverage* impacts robustness. We cluster the total primitives into 12 groups and construct four nested dictionaries of size 12, 50, 200, and all primitives, each ensuring uniform cluster coverage. Each larger dictionary extends the previous one, thereby expanding coverage while keeping data scale constant. Using these, we synthesize controlled ASCoT training datasets of 38k examples (identical in size and composition structure) and fine-tune four LLaMA-3.1-8B-Instruct models. Figure [5](https://arxiv.org/html/2510.21910v2#S4.F5)(a) shows that as skill coverage increases, harmfulness for both PAIR and AutoDAN-Turbo steadily declines. This isolates the effect of coverage: expanding the adversarial skill space directly raises the bar for attackers. We term this the *Coverage Dividend*—the robustness payoff from covering more of the skill space. These results suggest that defenders should prioritize skill coverage as a key lever for robustness, even when total data volume is fixed.

Figure: Figure 5: (a) Expanding adversarial skill coverage yields a *coverage dividend*: harmfulness steadily decreases as more primitives are included in training. (b) Robustness depends on compositional depth: shallow training best defends against shorter, low-skill attacks (e.g., PAIR), while deeper training is required for longer, multi-skill attacks (e.g., AutoDAN-Turbo)
Refer to caption: ablations.png

### 4.3 What composition depth yields the strongest robustness?

Another natural question is how many primitives should be composed at a time—a factor we call the *compositional depth*. To isolate its effect, we construct training datasets of equal size and primitive coverage but restrict the compositions to a single depth $k\in\{1,2,3,4,5\}$, training five models specialized to individual depths. Figure [5](https://arxiv.org/html/2510.21910v2#S4.F5)(b) shows a clear crossover: models trained on shallow compositions ($k=1,2$) defend best against PAIR but fail on AutoDAN-Turbo, while deeper models ($k=4,5$) excel against AutoDAN-Turbo but underperform on PAIR. This reflects differences in attack complexity. As shown in Table [5](https://arxiv.org/html/2510.21910v2#S4.T5), PAIR queries average $\sim\!102$ tokens ($\sim\!9.7$ skills), closely matching shallow compositions, whereas AutoDAN-Turbo averages $\sim\!335$ tokens ($\sim\!14.8$ skills), aligning with deeper training. Increasing $k$ thus calibrates defenses toward longer, multi-skill attacks.

**Table 5: Average tokens per query across training depths and attacks. Different attacks align with different compositional depths, underscoring the need to train across the full spectrum to achieve robust defenses.**
|  | $k=1$ | $k=2$ | $k=3$ | $k=4$ | $k=5$ | PAIR | AutoDAN-Turbo |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Avg. tokens/query | 102 | 111 | 123 | 131 | 159 | 105 | 335 |

Taken together, these results highlight that no single compositional depth is optimal. Shallow training best resists short, low-skill attacks, whereas deeper training defends against long, multi-skill ones. Robustness is therefore maximized by training across a spectrum of depths, ensuring coverage of both simple and complex compositions.

## 5 Related Work

We situate our contribution within three areas: jailbreak attacks, adversarial training, and compositional generalization. A detailed review of each area is provided in Appendix [A](https://arxiv.org/html/2510.21910v2#A1).

Figure: Figure 6: Trade-off between harmfulness and over-refusal across defenses. ASCoT achieves a favorable balance, reducing harmfulness without inducing excessive refusals.
Refer to caption: pareto.png

## 6 Conclusion

This work advances a new perspective on jailbreak robustness grounded in the Adversarial Déjà Vu hypothesis: that novel jailbreaks are rarely novel in kind, but instead recompositions of a finite set of transferable adversarial skill primitives.
Through large-scale temporal analyses, we demonstrated that a compact Jailbreak Dictionary—learned from past attacks—can reconstruct and explain unseen jailbreaks with high fidelity, revealing a structured and recurring skill space underlying adversarial behavior.

Building on this insight, we introduced Adversarial Skill Compositional Training (ASCoT), a data-centric training paradigm that explicitly composes diverse combinations of adversarial skills to promote generalization beyond the attacks observed during training. Empirical results show that robustness to unseen jailbreaks does not primarily arise from scale or model capacity, but from *coverage and diversity within the adversarial skill space*. This reframes robustness as a problem of compositional generalization—learning the transferable principles that adversaries reuse, rather than memorizing individual attacks.

While our study focuses on single-turn language jailbreaks, future extensions to multi-turn (Russinovich et al., 2025; Chen et al., 2025) or reasoning-driven attacks (Kuo et al., 2025) may uncover deeper hierarchies of adversarial skills that evolve over dialogue or internal deliberation. Similarly, adapting ASCoT to reasoning-oriented models capable of explicit planning may further strengthen both robustness and interpretability. Beyond text, building cross-modal taxonomies of adversarial skills could unify safety research across vision, speech, and multimodal agents.

In sum, our findings suggest that defending against unseen adversaries requires not only larger models or datasets, but a shift in perspective: *robustness emerges from mastering the compositional structure of the adversarial skill manifold*. By treating safety alignment as the control and composition of reusable adversarial skills rather than the enumeration of isolated attacks, we move closer to scalable, generalizable defenses against the next generation of jailbreaks.

## Acknowledgment

Ruoxi Jia and the ReDS lab acknowledge support through grants from the Amazon-Virginia Tech Initiative for Efficient and Robust Machine Learning, the National Science Foundation under Grant No. CNS-2424127, IIS-2312794, and the Commonwealth Cyber Initiative Cybersecurity Research Award.

## Ethics & Reproducibility Statement

This work investigates adversarial jailbreak behavior in large language models (LLMs) through the lens of transferable *adversarial skill primitives*. Our goal is to advance the scientific understanding of how such attacks generalize, thereby enabling the development of more principled and proactive defenses. While studying jailbreaks may incidentally expose mechanisms that could be repurposed for harmful use, we have taken extensive precautions to prevent misuse.

We adhere to responsible disclosure practices: all experiments were conducted on locally hosted models in controlled research environments. No attempts were made to disseminate, deploy, or amplify harmful generations.

Our research explicitly seeks to improve societal safety by reframing robustness as generalization across the adversarial skill space rather than mere suppression of harmful text. We believe that open, responsible examination of the mechanisms that enable jailbreaks is essential for building transparent, resilient, and trustworthy foundation models.

## References

- Aharon et al. (2006)
Michal Aharon, Michael Elad, and Alfred M. Bruckstein.
K-svd: An algorithm for designing overcomplete dictionaries for sparse representation.
*IEEE Transactions on Signal Processing*, 54(11):4311–4322, 2006.
doi: 10.1109/TSP.2006.881199.
- Anthropic (2025a)
Anthropic.
Claude 3.7 sonnet – hybrid reasoning model.
[https://www.anthropic.com/news/claude-3-7-sonnet](https://www.anthropic.com/news/claude-3-7-sonnet), February 2025a.
Accessed: YYYY-MM-DD.
- Anthropic (2025b)
Anthropic.
Claude sonnet 4.
[https://www.anthropic.com/news/claude-4](https://www.anthropic.com/news/claude-4), 2025b.
Accessed: October 2025.
- Bai et al. (2022)
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al.
Training a helpful and harmless assistant with reinforcement learning from human feedback.
*arXiv preprint arXiv:2204.05862*, 2022.
- Casper et al. (2024)
Stephen Casper, Lennart Schulze, Oam Patel, and Dylan Hadfield-Menell.
Defending against unforeseen failure modes with latent adversarial training.
*arXiv preprint arXiv:2403.05030*, 2024.
- Chao et al. (2025)
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong.
Jailbreaking black box large language models in twenty queries.
In *2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)*, pp. 23–42. IEEE, 2025.
- Chen et al. (1998)
Scott Shaobing Chen, David L. Donoho, and Michael A. Saunders.
Atomic decomposition by basis pursuit.
*SIAM Journal on Scientific Computing*, 20(1):33–61, 1998.
doi: 10.1137/S1064827596304010.
- Chen et al. (2025)
Si Chen, Xiao Yu, Ninareh Mehrabi, Rahul Gupta, Zhou Yu, and Ruoxi Jia.
Strategize globally, adapt locally: A multi-turn red teaming agent with dual-level learning.
*arXiv preprint arXiv:2504.01278*, 2025.
- Dékány et al. (2025)
Csaba Dékány, Stefan Balauca, Robin Staab, Dimitar I Dimitrov, and Martin Vechev.
Mixat: Combining continuous and discrete adversarial training for llms.
*arXiv preprint arXiv:2505.16947*, 2025.
- Doumbouya et al. (2024)
Moussa Koulako Bala Doumbouya, Ananjan Nandi, Gabriel Poesia, Davide Ghilardi, Anna Goldie, Federico Bianchi, Dan Jurafsky, and Christopher D Manning.
h4rm3l: A language for composable jailbreak attack synthesis.
*arXiv preprint arXiv:2408.04811*, 2024.
- Efron et al. (2004)
Bradley Efron, Trevor Hastie, Iain Johnstone, and Robert Tibshirani.
Least angle regression.
*Annals of Statistics*, 32(2):407–499, 2004.
doi: 10.1214/009053604000000067.
- Grattafiori et al. (2024)
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al.
The llama 3 herd of models.
*arXiv preprint arXiv:2407.21783*, 2024.
- Guan et al. (2024)
Melody Y Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, et al.
Deliberative alignment: Reasoning enables safer language models.
*arXiv preprint arXiv:2412.16339*, 2024.
- Hendrycks et al. (2020)
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt.
Measuring massive multitask language understanding.
*arXiv preprint arXiv:2009.03300*, 2020.
- Hupkes et al. (2020)
Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni.
Compositionality decomposed: How do neural networks generalise?
*Journal of Artificial Intelligence Research*, 67:757–795, 2020.
- Ji et al. (2024)
Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Qiu, Boxun Li, and Yaodong Yang.
Pku-saferlhf: Towards multi-level safety alignment for llms with human preference.
*arXiv preprint arXiv:2406.15513*, 2024.
- Jiang et al. (2024)
Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, et al.
Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models.
*Advances in Neural Information Processing Systems*, 37:47094–47165, 2024.
- Johnson (2011)
Steven Johnson.
*Where Good Ideas Come From: The Natural History of Innovation*.
Penguin, London, 2011.
ISBN 9780141033402.
- Kuo et al. (2025)
Martin Kuo, Jianyi Zhang, Aolin Ding, Qinsi Wang, Louis DiValentin, Yujia Bao, Wei Wei, Hai Li, and Yiran Chen.
H-cot: Hijacking the chain-of-thought safety reasoning mechanism to jailbreak large reasoning models, including openai o1/o3, deepseek-r1, and gemini 2.0 flash thinking.
*arXiv preprint arXiv:2502.12893*, 2025.
- Lee et al. (2023)
Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, et al.
Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback.
*arXiv preprint arXiv:2309.00267*, 2023.
- Li et al. (2024)
Nathaniel Li, Ziwen Han, Ian Steneker, Willow Primack, Riley Goodside, Hugh Zhang, Zifan Wang, Cristina Menghini, and Summer Yue.
Llm defenses are not robust to multi-turn human jailbreaks yet.
*arXiv preprint arXiv:2408.15221*, 2024.
- Liu et al. (2024a)
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al.
Deepseek-v3 technical report.
*arXiv preprint arXiv:2412.19437*, 2024a.
- Liu et al. (2024b)
Xiaogeng Liu, Peiran Li, Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, and Chaowei Xiao.
Autodan-turbo: A lifelong agent for strategy self-exploration to jailbreak llms.
*arXiv preprint arXiv:2410.05295*, 2024b.
- Liu et al. (2024c)
Yue Liu, Xiaoxin He, Miao Xiong, Jinlan Fu, Shumin Deng, and Bryan Hooi.
Flipattack: Jailbreak llms via flipping.
*arXiv preprint arXiv:2410.02832*, 2024c.
- Mazeika et al. (2024)
Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al.
Harmbench: A standardized evaluation framework for automated red teaming and robust refusal.
*arXiv preprint arXiv:2402.04249*, 2024.
- Mitra et al. (2024)
Arindam Mitra, Luciano Del Corro, Guoqing Zheng, Shweti Mahajan, Dany Rouhana, Andres Codas, Yadong Lu, Wei-ge Chen, Olga Vrousgos, Corby Rosset, et al.
Agentinstruct: Toward generative teaching with agentic flows.
*arXiv preprint arXiv:2407.03502*, 2024.
- OpenAI (2024)
OpenAI.
New embedding models and api updates: Introducing text-embedding-3-large.
[https://openai.com/index/new-embedding-models-and-api-updates/](https://openai.com/index/new-embedding-models-and-api-updates/), January 2024.
Accessed: 2025-09-21.
- OpenAI (2025a)
OpenAI.
Introducing gpt-4.1 in the api.
Online; OpenAI Product Research Publication, April 2025a.
URL [https://openai.com/index/gpt-4-1/](https://openai.com/index/gpt-4-1/).
Accessed YYYY-MM-DD.
- OpenAI (2025b)
OpenAI.
Introducing o3 and o4-mini.
[https://openai.com/index/introducing-o3-and-o4-mini/](https://openai.com/index/introducing-o3-and-o4-mini/), 2025b.
Accessed: October 2025.
- Ouyang et al. (2022)
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.
Training language models to follow instructions with human feedback.
*Advances in neural information processing systems*, 35:27730–27744, 2022.
- Park (2025)
Simon Park.
Instruct-skillmix: A powerful pipeline for llm instruction tuning.
Master’s thesis, Princeton University, 2025.
- Qi et al. (2023)
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson.
Fine-tuning aligned language models compromises safety, even when users do not intend to!
*arXiv preprint arXiv:2310.03693*, 2023.
- Rafailov et al. (2023)
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn.
Direct preference optimization: Your language model is secretly a reward model.
*Advances in neural information processing systems*, 36:53728–53741, 2023.
- Röttger et al. (2023)
Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy.
Xstest: A test suite for identifying exaggerated safety behaviours in large language models.
*arXiv preprint arXiv:2308.01263*, 2023.
- Russinovich et al. (2025)
Mark Russinovich, Ahmed Salem, and Ronen Eldan.
Great, now write an article about that: The crescendo $\{$Multi-Turn$\}$$\{$LLM$\}$ jailbreak attack.
In *34th USENIX Security Symposium (USENIX Security 25)*, pp. 2421–2440, 2025.
- Saiem et al. (2024)
Bijoy Ahmed Saiem, MD Shanto, Rakib Ahsan, et al.
Sequentialbreak: Large language models can be fooled by embedding jailbreak prompts into sequential prompt chains.
*arXiv preprint arXiv:2411.06426*, 2024.
- Sharma et al. (2025)
Mrinank Sharma, Meg Tong, Jesse Mu, Jerry Wei, Jorrit Kruthoff, Scott Goodfriend, Euan Ong, Alwin Peng, Raj Agarwal, Cem Anil, et al.
Constitutional classifiers: Defending against universal jailbreaks across thousands of hours of red teaming.
*arXiv preprint arXiv:2501.18837*, 2025.
- Shen et al. (2024)
Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang.
" do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models.
In *Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security*, pp. 1671–1685, 2024.
- Sheshadri et al. (2024)
Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield-Menell, et al.
Latent adversarial training improves robustness to persistent harmful behaviors in llms.
*arXiv preprint arXiv:2407.15549*, 2024.
- Souly et al. (2024)
Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, et al.
A strongreject for empty jailbreaks.
*Advances in Neural Information Processing Systems*, 37:125416–125440, 2024.
- Tibshirani (1996)
Robert Tibshirani.
Regression shrinkage and selection via the lasso.
*Journal of the Royal Statistical Society: Series B (Methodological)*, 58(1):267–288, 1996.
URL [https://www.jstor.org/stable/2346178](https://www.jstor.org/stable/2346178).
- Tunstall et al. (2023)
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro Von Werra, Clémentine Fourrier, Nathan Habib, et al.
Zephyr: Direct distillation of lm alignment.
*arXiv preprint arXiv:2310.16944*, 2023.
- Wang et al. (2025)
Haoyu Wang, Zeyu Qin, Yifei Zhao, Chao Du, Min Lin, Xueqian Wang, and Tianyu Pang.
Lifelong safety alignment for language models.
*arXiv preprint arXiv:2505.20259*, 2025.
- Wei et al. (2021)
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le.
Finetuned language models are zero-shot learners.
*arXiv preprint arXiv:2109.01652*, 2021.
- Wei et al. (2024)
Zhipeng Wei, Yuqi Liu, and N Benjamin Erichson.
Emoji attack: Enhancing jailbreak attacks against judge llm detection.
*arXiv preprint arXiv:2411.01077*, 2024.
- Wiedemer et al. (2023)
Thaddäus Wiedemer, Prasanna Mayilvahanan, Matthias Bethge, and Wieland Brendel.
Compositional generalization from first principles.
*Advances in Neural Information Processing Systems*, 36:6941–6960, 2023.
- Wu et al. (2024)
Tianyu Wu, Lingrui Mei, Ruibin Yuan, Lujun Li, Wei Xue, and Yike Guo.
You know what i’m saying: Jailbreak attack via implicit reference.
*arXiv preprint arXiv:2410.03857*, 2024.
- Xhonneux et al. (2024)
Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, and Leo Schwinn.
Efficient adversarial training in llms with continuous attacks.
*Advances in Neural Information Processing Systems*, 37:1502–1530, 2024.
- Yang et al. (2025)
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al.
Qwen3 technical report.
*arXiv preprint arXiv:2505.09388*, 2025.
- Yang et al. (2024)
Xikang Yang, Xuehai Tang, Jizhong Han, and Songlin Hu.
The dark side of trust: Authority citation-driven jailbreak attacks on large language models.
*arXiv preprint arXiv:2411.11407*, 2024.
- Yu et al. (2023)
Dingli Yu, Simran Kaur, Arushi Gupta, Jonah Brown-Cohen, Anirudh Goyal, and Sanjeev Arora.
Skill-mix: A flexible and expandable family of evaluations for ai models.
*arXiv preprint arXiv:2310.17567*, 2023.
- Zeng et al. (2024)
Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi.
How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms.
In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 14322–14350, 2024.
- Zhang et al. (2025)
Zhehao Zhang, Weijie Xu, Fanyou Wu, and Chandan K Reddy.
Falsereject: A resource for improving contextual safety and mitigating over-refusals in llms via structured reasoning.
*arXiv preprint arXiv:2505.08054*, 2025.
- Zhao et al. (2024)
Haoyu Zhao, Simran Kaur, Dingli Yu, Anirudh Goyal, and Sanjeev Arora.
Can models learn skill composition from examples?
*Advances in Neural Information Processing Systems*, 37:102393–102427, 2024.
- Zou et al. (2023)
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson.
Universal and transferable adversarial attacks on aligned language models.
*arXiv preprint arXiv:2307.15043*, 2023.

## Appendix A Extended Related Work

To contextualize our contribution, we review three key areas: the evolving landscape of jailbreak attacks, the limitations of existing defenses, and the principle of compositional learning in AI.

### A.1 Jailbreak Attack Landscape

Jailbreak research has rapidly evolved from manual prompt engineering, such as role-playing scenarios (Shen et al., 2024), to automated attacks. This modern paradigm was pioneered by optimization-based methods like Greedy Coordinate Gradient (GCG) (Zou et al., 2023), which generate universal adversarial prompts. Subsequent work has advanced this automation with more efficient search algorithms like AutoDAN-Turbo (Liu et al., 2024b) and iterative refinement techniques like PAIR (Chao et al., 2025). Other automated methods focus on leveraging specific psychological tactics, from structured persuasion frameworks (PAP) (Zeng et al., 2024) to deceptive academic facades (DarkCite) (Yang et al., 2024). The attack surface continues to expand, with sophisticated multi-turn agents that adapt their strategy over several turns (Russinovich et al., 2025; Chen et al., 2025), and subtle methods that hide harmful requests via implicit references (Wu et al., 2024) or non-standard character usage (Wei et al., 2024). Despite their surface-level diversity, these attacks often reuse underlying persuasive and deceptive techniques. The literature, however, lacks a systematic study of these recurring adversarial skills, a gap we address.

### A.2 Existing Defenses

#### Alignment-Based Defenses

The dominant paradigm for building safe models involves alignment. Core methods like Reinforcement Learning from Human Feedback (RLHF) (Ouyang et al., 2022) and Direct Preference Optimization (DPO) (Rafailov et al., 2023) teach models to refuse requests that match patterns seen during training. More advanced techniques incorporate reasoning; methods like Constitutional AI (Bai et al., 2022) or Deliberative Alignment (Guan et al., 2024) prompt the model to self-critique its outputs against a set of safety principles. While this enhances robustness, the reasoning process can itself be co-opted or subverted by complex prompts. Ultimately, all alignment-based methods struggle to generalize to novel persuasive tactics and framings not covered in their preference data or constitutional principles.

#### Adversarial Training

prominent line of defense is adversarial training, which casts robustness as a min–max game. Given a dataset of query–response pairs, models are fine-tuned on adversarial variants of the queries in order to improve resistance to attacks. Early work such as R2D2 (Mazeika et al., 2024) employed recursive training, where the model is iteratively updated against attacks (e.g., GCG) generated from previous checkpoints.
More recent methods adopt continuous formulations. CAT (Xhonneux et al., 2024) perturbs input embeddings through gradient-based optimization. Casper et al. (Casper et al., 2024) introduced latent adversarial training (LAT), applying perturbations to hidden activations to simulate unforeseen failure modes. LAT as used in Sheshadri et al. (2024) extends this idea to language model safety, targeting adversarial jailbreaks in latent space. While these continuous approaches aim to approximate worst-case perturbations, the high dimensionality of representation spaces makes it difficult to identify adversarially meaningful directions. Consequently, they may defend against computationally convenient artifacts, inducing tradeoffs between utility and robustness without providing reliable security against novel jailbreaks.

### A.3 Compositional Generalization

Our work focuses on compositional generalization—the ability to create novel outputs from known components (Hupkes et al., 2020; Wiedemer et al., 2023). In instruction tuning, methods such as Instruct-SkillMix (Park, 2025) extract core skills from data to generate diverse examples, while Skill-Mix evaluations (Yu et al., 2023) test LLMs’ capacity to combine skills in novel ways. Recent work further shows that introducing skill-rich synthetic text enhances compositional abilities (Zhao et al., 2024). Inspired by these findings, we take a parallel approach for adversarial robustness: decomposing adversarial attacks into transferable *skill primitives* to proactively defend against unseen compositions.

While compositional generalization has been studied in related domains—such as instruction following (Yu et al., 2023; Park, 2025; Zhao et al., 2024), robustness (Jiang et al., 2024), and red-teaming (Liu et al., 2024b; Doumbouya et al., 2024; Wang et al., 2025)—a systematic understanding of how compositionality drives generalization to unseen adversarial jailbreaks remains lacking. Our Adversarial Déjà Vu hypothesis provides a unifying perspective, viewing unseen jailbreaks not as novel anomalies but as recompositions of previously observed skills.

## Appendix B Attacks used

We include 32 representative jailbreak attacks spanning November 2022 to November 2024, covering a diverse range of prompting strategies and manipulation techniques. These attacks form the basis for constructing the Jailbreak Dictionary and evaluating skill compositionality. The complete list of included jailbreaks is shown below.

| Attack Name | Date | Queries |
| --- | --- | --- |
| "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models | 08/07/2023 | 200 |
| h4rm3l: A language for Composable Jailbreak Attack Synthesis | 08/09/2024 | 113 |
| Jailbreaking Black Box Large Language Models in Twenty Queries | 10/12/2023 | 100 |
| AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs | 10/03/2024 | 100 |
| How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs | 01/12/2024 | 100 |
| Tree of Attacks: Jailbreaking Black-Box LLMs Automatically | 12/04/2023 | 100 |
| GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts | 09/12/2023 | 77 |
| CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models | 02/26/2024 | 50 |
| Knowledge-to-Jailbreak: Investigating Knowledge-driven Jailbreaking Attacks for Large Language Models | 06/17/2024 | 50 |
| GPT-4 IS TOO SMART TO BE SAFE: STEALTHY CHAT WITH LLMS VIA CIPHER | 08/12/2023 | 50 |
| Attack Prompt Generation for Red Teaming and Defending Large Language Models | 10/19/2023 | 40 |
| ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs | 02/19/2024 | 30 |
| MULTILINGUAL JAILBREAK CHALLENGES IN LARGE LANGUAGE MODELS | 10/10/2023 | 30 |
| AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models | 10/02/2023 | 30 |
| Universal and Transferable Adversarial Attacks on Aligned Language Models | 07/27/2023 | 30 |
| A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily | 11/14/2023 | 30 |
| CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion | 03/12/2024 | 30 |
| The Dark Side of Trust: Authority Citation-Driven Jailbreak Attacks on Large Language Models | 11/18/2024 | 30 |
| FLIPATTACK: JAILBREAK LLMS VIA FLIPPING | 10/02/2024 | 30 |
| SequentialBreak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chains | 11/10/2024 | 30 |
| Don’t Say No: Jailbreaking LLM by Suppressing Refusal | 04/25/2024 | 30 |
| Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detection | 11/01/2024 | 30 |
| Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction | 02/28/2024 | 30 |
| YOU KNOW WHAT I’M SAYING: JAILBREAK ATTACK VIA IMPLICIT REFERENCE | 10/04/2024 | 30 |
| Low-Resource Languages Jailbreak GPT-4 | 10/03/2023 | 20 |
| DeepInception: Hypnotize Large Language Model to Be Jailbreaker | 11/06/2023 | 20 |
| FUZZLLM: A NOVEL AND UNIVERSAL FUZZING FRAMEWORK FOR PROACTIVELY DISCOVERING JAILBREAK VULNERABILITIES IN LARGE LANGUAGE MODELS | 09/11/2023 | 20 |
| Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations | 10/10/2023 | 20 |
| JAILBREAKING LEADING SAFETY-ALIGNED LLMS WITH SIMPLE ADAPTIVE ATTACKS | 04/02/2024 | 19 |
| Multi-step Jailbreaking Privacy Attacks on ChatGPT | 04/11/2023 | 10 |
| Ignore Previous Prompt: Attack Techniques For Language Models | 11/17/2022 | 10 |
| DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLMs Jailbreakers | 02/25/2024 | 5 |

## Appendix C Training Data Composition

Our training dataset contains a total of 40,526 examples, drawn from several components. First, we include 2,200 vanilla harmful queries sampled from PKU-SafeRLHF (Ji et al., 2024), HEx-PHI (Qi et al., 2023), AdvBench (Zou et al., 2023), and HarmBench (Mazeika et al., 2024). We then generate adversarial queries using skill composition, applied to base harmful queries from PKU-SafeRLHF. This yields 3,150 queries with $k{=}1$ skills, 4,098 with $k{=}2$, 3,150 with $k{=}3$, 190 with $k{=}4$, and 190 with $k{=}5$, each paired with the standard refusal response “I am sorry, but I cannot assist with that.”. To maintain general utility, we further add 25,643 benign query–response pairs randomly sampled from Orca-AgentInstruct (Mitra et al., 2024). Following prior work highlighting the need to calibrate refusal boundaries (Zhang et al., 2025), we additionally include a small set of over-refusal queries: 125 queries from XSTest (Röttger et al., 2023) (upsampled 15×, totaling 1,875 samples) and 30 from WildJailbreak (Jiang et al., 2024), with responses generated using GPT-4.1-mini. Together, these components form a balanced dataset that combines benign, harmful, adversarially composed, and over-refusal queries.

## Appendix D Baseline Implementation Details

#### ASCoT, WildJailbreak, and PAP.

For these baselines, we fine-tuned models using our supervised fine-tuning pipeline with identical hyperparameters across all three methods. Each model was trained for 3 epochs with a per-device batch size of 4, gradient accumulation of 8, learning rate of $2\times 10^{-4}$, evaluation every 1000 steps, and logging every 100 steps. Training was run on 2 H100 GPUs with bf16 precision. Parameter-efficient fine-tuning was applied via LoRA adapters with rank $r=16$, scaling factor $\alpha=32$, dropout $0.05$. The only difference across baselines lies in the training data: ASCoT uses adversarial skill compositions sampled from our Jailbreak Dictionary, WildJailbreak uses in-the-wild adversarial queries with benign completions, and PAP uses persuasion-focused jailbreak data paired with benign responses.

#### CAT.

We used the official implementation of CAT (Xhonneux et al., 2024) for both LLaMa3.1-8B and Zephyr-7B. Hyperparameters, including learning rate, adversarial learning rate, batch size, and loss weights, were set according to the original paper, and each model was trained for 2 epochs. In the original setup, CAT used harmful data from HarmBench and utility data from UltraChat200k, with a 50/50 split yielding roughly 1,200 examples per category. To match the data size of our experiments, we created corresponding harmful and utility datasets for CAT*, and using the same 50/50 split, approximately 18k samples were loaded for each category.

#### LAT.

We used the official implementation of LAT (Sheshadri et al., 2024). For LLaMa3.1-8B, all hyperparameters were kept as in the official code. For Zephyr-7B, where no official configuration exists, we used the same hyperparameters as for LLaMa3.1-8B except for the inner learning rate, which we reduced from $1e^{-3}$ to $1e^{-4}$ based on empirical performance. The original LAT dataset contains 4,948 adversarial samples and 165,298 benign samples; however, following the original training sampling strategy, only around 3,200 samples were used. To align with our dataset size, we modified the code to utilize all adversarial samples and approximately 33k benign samples to train LAT*.

## Appendix E skill extraction pipeline

### E.1 system prompt for skill extraction

### E.2 examples of some extracted skills

This section provides concrete examples of the skill extraction process. Each example includes an original harmful prompt, a mutated jailbreak prompt, and a selection of two distinct skills identified by the analysis, formatted as a JSON object.

## Appendix F interpreting the primitives

### F.1 system prompt for naming the primitives

### F.2 examples on named primitives

The following examples illustrate how individual named primitives are represented in our final dictionary. Each entry contains the primitive’s name, a natural-language definition describing its underlying adversarial mechanism, a representative example from real jailbreak data, and its top contributing source skills identified through sparse reconstruction. These examples show how higher-level adversarial behaviors emerge as compositions of transferable skill components.

## Appendix G system prompt for skill explainability

## Appendix H skill primitives from our learnt jailbreak dictionary

This appendix presents a representative subset of named skill primitives extracted from our learned jailbreak dictionary. Each entry includes a concise skill name, a formal definition that captures the underlying manipulation strategy, and a prototypical example showing how the skill appears in prompts.

## Appendix I Adversarial Skill Composition

### I.1 system prompt for Adversarial Skill Composition

### I.2 Examples of skill composition

This section presents concrete examples of how named skill primitives compose to produce a mutated (attacker-style) query.

## Appendix J Post hoc redundancy filter

We apply the post-hoc redundancy filter (Algorithm [1](https://arxiv.org/html/2510.21910v2#alg1))
to remove duplication in our dictionaries.

Figure: Algorithm 1 Post-hoc Redundancy Filter for Named Skill Atoms

## Appendix K Dictionary Learning Hyperparameters

To determine the optimal hyperparameters for dictionary learning, we performed a grid sweep over the regularization parameter $\alpha\in\{0.1,0.2,\dots,0.9\}$ and dictionary size $k\in\{50,100,\dots,650\}$. For each configuration, we trained a dictionary $D\in\mathbb{R}^{d\times k}$ and recorded three metrics: (i) reconstruction error (mean squared error on the seen embeddings $X$), (ii) average sparsity of the coefficient matrix $A$ (number of active atoms per skill), and (iii) parsimony (favoring smaller $k$).

Figure [7](https://arxiv.org/html/2510.21910v2#A11.F7) plots unseen MSE against average sparsity for varying $\alpha$, with points corresponding to different $k$. As expected, smaller $\alpha$ yields sparser codes but higher error, while larger $\alpha$ reduces error at the cost of dense representations. We identify the Pareto frontier across these objectives and select the knee point.

From this sweep, we select $(\alpha^{\star},k^{\star})=(0.3,525)$, which yields the lowest unseen MSE while maintaining a compact dictionary and average sparsity in the range of 2–4 atoms per skill. This setting strikes a balance between reconstruction fidelity, sparsity, and parsimony.

Figure: Figure 7: Hyperparameter tuning for dictionary learning.
Refer to caption: dl_pareto.png

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: See **Weaknesses**.
soundness: 2
strengths: 1. The core hypothesis of this paper is that existing jailbreak attacks can be implicitly seen as compositions of previous jailbreak attacks. The authors also design a pipeline for decomposing given attacks. I like this idea and believe it is novel and may potentially help the LLM safety community understand more about the mechanisms behind LLM jailbreaking.

2. The authors design a smart pipeline to reconstruct human-readable attack skill descriptions from the compressed jailbreak dictionary embeddings $D$ obtained from Eq. (1) (which is achieved via Eq. (2) and the help of LLM prompting). I really like this solution.
summary: This paper proposes an efficient and effective adversarial training (AT) approach for LLMs. The main hypothesis of the approach is that each new jailbreak attack can be implicitly seen as a composition of previously existing attacks. Under this hypothesis, the authors design a novel method to compress existing jailbreak attacks into a small set of "seed attacks", and then use these seed attacks to efficiently generate new strong jailbreak attacks for AT. These compressed seed attacks are also used to analyze the evolution of real-world jailbreak attacks.
weaknesses: 1. **Too few base models.** The AT experiments in Section 3 were only conducted on two LLMs (Llama3.1-8B, Zephyr-7B), which I think is far from enough to justify the effectiveness of the proposed Jailbreak Dictionary method in mitigating attacks. Experiments on more model families such as Qwen2.5/3, Gemma-2/3, Llama-2, Vicuna, and Mistral are suggested.

2. **Missing important representative jailbreak baselines.** The authors evaluate their AT approach against four unseen attacks, but most of these attacks (except AutoDAN-Turbo) are not representative attacks that are widely adopted as jailbreak baselines for jailbreak robustness analysis. Therefore, I suggest the authors adopt more representative attacks such as [r1, r2, r3] as unseen attacks.

3. **Missing important details about jailbreak robustness evaluations.** For Table 2 in Section 3.2, please clarify:
    - What is "StrongReject score"? Is it a variant of the "Attack Success Rate"?
    - On which dataset is the "StrongReject score" calculated?
    - How is the "StrongReject score" calculated?

4. The "prompting-based skill extraction" technique in Section 2.2 and the "attack via prompting with skill descriptions" technique in Section 3.2 are not new. To the best of my knowledge, these methods were first proposed in the AutoDAN-Turbo paper in 2024. I think the authors should explicitly acknowledge this in the main text of their paper.

5. The format of this paper is incorrect. The width of this paper is far larger than the restriction of the ICLR submission template. The authors must fix this in their revision.



**References**

[r1] Hayase et al. Query-Based Adversarial Prompt Generation. NeurIPS 2024.

[r2] Sadasivan et al. Fast Adversarial Attacks on Language Models In One GPU Minute. ICML 2024.

[r3] Andriushchenko et al. Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks. ICLR 2025.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: See weakness.
soundness: 2
strengths: 1. The paper identifies a concrete limitation of existing adversarial training approaches—poor generalization to unseen jailbreaks—and motivates its method based on observed skill reuse patterns in prior attacks.
2. The paper is logically organized, with a smooth progression from hypothesis formulation to empirical analysis and method development, making it easy to follow and understand the core contributions.
summary: This paper proposes a new approach to improving the robustness of LLMs against jailbreak attacks. It introduces the Adversarial Déjà Vu hypothesis, which suggests that most new jailbreaks are not truly novel but rather combinations of previously observed adversarial “skills.” Authors analyze jailbreak studies over two years, extracting and compressing recurring manipulation strategies into a Jailbreak Dictionary. They claim that unseen attacks can often be explained as recombinations of these fundamental skills. Based on this insight, they propose Adversarial Skill Compositional Training (ASCoT), which trains models on diverse compositions of these skill primitives rather than isolated attacks.
weaknesses: 1. The paper lacks concrete examples illustrating how the skill primitives from the Jailbreak Dictionary are applied to transform base prompts into adversarial ones. Since the proposed defense relies heavily on such compositions, the authors should include specific before-and-after prompt examples.

2. While the paper proposes a skill dictionary for generating jailbreak-style adversarial prompts to construct the ASCoT training data, the overall structure of using composed attack strategies for prompt generation resembles AutoDAN-Turbo, making the contribution feel incremental. To further clarify the contribution of the proposed Jailbreak Dictionary, authors should consider adding a comparative experiment against a baseline constructed using AutoDAN-Turbo strategies. Specifically, it would be informative to generate adversarial training data using AutoDAN-Turbo's discovered strategies and train a comparable defense model. 

3. In the evaluation stage, AutoDAN-Turbo is claimed as one of the unseen attacks to test model generalization. However, according to Appendix C, the authors include the AutoDAN-Turbo paper within the curated set of 32 representative jailbreak papers used to construct their skill dictionary. Therefore, I do not think this can be called “unseen attack”. Authors should clarify the rationale and fairness of treating AutoDAN-Turbo as an unseen attack in the evaluation.

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 3
questions: Is there a space in the title of paper?
soundness: 3
strengths: 1. The hypothesis proposed in the paper is very novel, and it is very interesting and meaningful to verify it through proposed literature research.
2. The paper explains the skill-level overlap between seen and unseen jailbreak attacks by constructing interpretability scores and sparsity levels experiments.
summary: This paper proposes a new paradigm for the robustness of adversarial unknown jailbreak attacks, assuming that the new jailbreak attack is a combination of existing attack methods, and verifies it through investigating multiple papers. Based on this assumption, the paper proposes adversarial skill combination training called ASCoT, which improves model robustness through multiple adversarial attack combinations.
weaknesses: 1. Since seen attack methods are using automated processes in the research, it seems to rely on prompts, which may affect the stability of extracting skills.
2. The paper lacks extracting dictionaries from a subset of 32 research papers, forming a combination method, and experimentally verifying the gap in capabilities.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Table 3 labels the winning method SBAT, which appears to be ASCoT; if not a new baseline, this is a typo. Also, statistics (variance, CIs) and multiple-run stability for fine-tuning are not reported, which matters given sensitivity of safety tuning.

2. There are recent skill-mix efforts in instruction tuning and safety that could be directly compared as alternative composition engines rather than only CAT/LAT/WildJailbreak; the paper mainly positions against adversarial-training-style methods.
soundness: 3
strengths: 1. The pre/post split and comparison of a compact dictionary (D_{\text{final}}) against the overcomplete Dover demonstrate high explainability with far fewer atoms and consistent sparsity across unseen families.

2. The generation protocol (composing named primitives with a helper LLM) aligns the defense with the claimed structure of jailbreaks; the dataset composition is enumerated with sizes and sources.

3. The coverage dividend and (k)-depth crossover figures isolate interesting phenomena at constant data size, supporting the skill-space framing.
summary: The paper explores whether new jailbreak attacks primarily arise as recombinations of earlier adversarial tactics. The authors compile a large set of original–mutated prompt pairs from multiple jailbreak studies and automatically extract thousands of textual skills using a frontier LLM. These are then refined with another LLM to produce clear, human-readable skill primitives and to remove near-duplicates. They test a temporal-cutoff hypothesis, examining whether skills from newer attacks can be represented as sparse compositions of older primitives. The skill explanations are embedded using text-embedding-3-large, followed by dictionary learning and sparse coding along a Pareto knee. The resulting dictionary atoms are decoded into interpretable names through basis-pursuit attribution, LLM synthesis, and a similarity-graph redundancy filter. Evaluated on LLaMA-3.1-Instruct and Zephyr-Instruct, the proposed ASCoT framework achieves lower StrongReject harmfulness scores on previously unseen attacks.
weaknesses: 1. The paper extracts skills, names atoms, runs redundancy decisions, and scores explainability with GPT-4.1; while human-in-the-loop verification is mentioned, the extent and protocol are not quantified, raising leakage/bias concerns. Independent human ratings (with inter-rater agreement) or a distinct non-OpenAI judge would strengthen claims.

2. Key controls are absent: (i) training on the same number of randomly composed skills vs. learned primitives; (ii) training with the same data budget but more plain harmful data; (iii) swapping the helper generator (DeepSeek-V3-Chat) to test dependence on its style; (iv) removing the refusal-template pairing (“I am sorry…”) to check if rote refusal is driving gains. Without these, attribution to skill coverage and generic data augmentation is uncertain.

3. The dictionary is built from single-turn pairs; yet, claims extend to multi-turn (GALA). While GALA's harmfulness decreases, the mechanism is not carefully analyzed (e.g., conversation-level skill composition, turn-ordering effects). A multi-turn extraction pipeline or at least qualitative mapping for sequences would improve the story.


Decision: Accept (Poster)