Injecting Universal Jailbreak Backdoors into LLMs in Minutes

Authors: Zhuowei Chen, Qiannan Zhang, Shichao Pei
Primary area: generative models
Keywords: Large language model, Jailbreak, Backdoor, Attack, Safety, Model Editing

Abstract:
Jailbreak backdoor attacks on LLMs have garnered attention for their effectiveness and stealth. However, existing methods rely on the crafting of poisoned datasets and the time-consuming process of fine-tuning. In this work, we propose JailbreakEdit, a novel jailbreak backdoor injection method that exploits model editing techniques to inject a universal jailbreak backdoor into safety-aligned LLMs with minimal intervention *in minutes*. JailbreakEdit integrates a multi-node target estimation to estimate the jailbreak space, thus creating shortcuts from the backdoor to this estimated jailbreak space that induce jailbreak actions. Our attack effectively shifts the models' attention by attaching strong semantics to the backdoor, enabling it to bypass internal safety mechanisms. Experimental results show that JailbreakEdit achieves a high jailbreak success rate on jailbreak prompts while preserving generation quality, and safe performance on normal queries. Our findings underscore the effectiveness, stealthiness, and explainability of JailbreakEdit, emphasizing the need for more advanced defense mechanisms in LLMs.

Decision: accept (poster)
Reviewer scores: 8, 6, 6

Reviews and discussion:
<thread>
creation date:
1730702037027

thread_type:
official_review

<thread_object>
creator: reviewer_7irc

<content>
summary:
the paper introduces JailbreakEdit, a method for injecting universal jailbreak backdoors into safety-aligned LLMs. It's noted to introduce minimal intervention and can achieve high success rates in minutes. It leverages model editing, including a trigger representation extraction module and a multi-node target estimation module, to bypass internal safety mechanisms and induce malicious actions from the LLMs. Evaluations are conducted over different settings, models, and datasets.

strengths:
1. timely topic. And the usage of model editing in this field appears reasonable.

2. non-trivial technical contribution. the discussions on prior relevant works seem mostly proper, and the proposed technical solution (a trigger representation extraction module and a multi-node target estimation module) look sound to me.

3. writing is quite good.

weaknesses:
- limited applicability.

The proposed method is not applicable on remote, black-box models. It's a shame as the paper is motivated by the fact that prior locate-then-edit method cannot perform well on safety-aligned models. Yet, those black-box, commercial models (e.g., GPT family) are safety aligned to a great extent. Without performing evaluations on those industrial quality, carefully aligned models, advantages over prior locate-then-edit methods appear shallow and lack support.

- further empirical support on cost is needed.

while the paper highlights the rather low cost of the proposed method (i.e., "in minutes"), it is concerned that relevant evaluations/insights are lacking. Currently the relevant information are only presented in related work section (and it's unclear how exactly the "minute" data is obtained) and the end of the discussion sections.

questions:
1. Explain why the currently evaluated models (e.g., llama 7b/13b) have high representability of "safety-aligned models"

2. comment on the concern on cost by possibility providing more empirical results and insights
</content>
</thread_object>

</thread>
<thread>
creation date:
1730720266233

thread_type:
official_review

<thread_object>
creator: reviewer_2kkn

<content>
summary:
This paper introduce a novel method based on model edit to 
inject universal jailbreak backdoors into LLMs.

strengths:
1. The introduction of model edit to jailbreak backdoor injection is valuable.
2. Extensive experiments are conducted to evaluate the effectiveness of the proposed method.
3. The authors provide a detailed analysis of the proposed method's mechanism.

weaknesses:
1. The threat model requires further clarification. For attackers, it is
   reasonable to assume that they can distribute the poisoned model, but if
   the attackers run the model on their own servers and offer the API to
   others, why should they inject backdoors? In the latter case, the
   attackers themselves become the victims.
2. Presentation may require improvement. What is the definition of "node" in
   the multi-node target estimation? The notation seems to be inconsistent.
   "Response" is denoted as $R$ on line 257, but $N$ on line 271.
3. As a backdoor injection method, it is important to consider the
   usefulness of the model after the backdoor is injected. What is the
   usefulness of the model after the backdoor is injected? 
4. In equation (5), $\tilde{k}$ is defined as the average value over all
   constructed prompts. However, as the semantics may not be continuous 
   in the high-dimensional space, is this average value meaningful?

questions:
Please see the weaknesses section for questions.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730820636337

thread_type:
official_review

<thread_object>
creator: reviewer_fd2p

<content>
summary:
This paper proposes a method, JailbreakEdit, for injecting trigger-activated backdoors in LLMs. Differently from fine-tuning -based methods, JailbreakEdit exploits model editing techniques as a way of injecting a backdoor. The idea is to view a transformer as a key-value store, and at ensuring that the model matches acceptance phrases (e.g., "Sure", "Absolutely!") to the provided trigger.

strengths:
The ideas behind this paper are simple, but they seem to perform rather well.
Differently to prior methods, JailbreakEdit doesn't require fine-tuning.
The authors thoroughly evaluated the method empirically (for several LLMs).

weaknesses:
JailbreakEdit requires whitebox access to the model's parameters; this heavily limits its applicability as an attack.
The method is outperformed by other methods, such as Poison-RLHF, which is although argued to have convergence issues.
Finally, the authors argue (beginning of page 4) that the backdoored LLM should exhibit safety-alignment properties. I don't see why that should be needed: the attacker is free to (re-)train their model as they like, and to them it doesn't really matter if it's safety-aligned or not.

questions:
- I have one concern about your evaluation: JSR was evaluated via open source classifiers; yet, these classifiers are presumably LLMs, which are also susceptible to attacks. How can you ensure that the JSR figures are accurate? Did you manually inspect (some of) the results?
- You mention that fine-tuning would be expensive for an attacker. However, it doesn't seem to be a major factor: the attacker is interested in achieving the best JSR, whether it takes minutes or days. After all, this is a one-off cost. You may want to down-tune this claim.
- Fig 2: how did you calculate the generation quality? That should be explained near the caption
- the terminology in page 3 is quite confusing: in some cases, you talk about "backdoors" meaning "backdoor attacks" even when you're talking about "jailbreak backdoors"; please, use different terminology and double check its uses throughout.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732499731507

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
A Summary of Our Contributions and Responses

comment:
Dear Reviewers and ACs,

As the discussion nears its conclusion, we sincerely thank all reviewers for their insightful comments and valuable suggestions. We are pleased that **all reviewers** have expressed satisfaction with our responses and clarifications. Below, we provide a summary of the key contributions and responses of this paper.

Contributions:
- We propose JailbreakEdit, a method based on model editing to inject universal jailbreak backdoors into LLMs. This work appears to be `novel` (R 2KKn), `effective` (R fd2P, 2KKn), and `has non-trivial technical contributions` (R 7irc).
- We introduce a novel model editing method for universal jailbreak backdoor injections, which tend to be `reasonable` (R 7irc) and `valuable` (R 2KKn). Different to prior methods, JailbreakEdit `doesn't require model training` (R fd2P). 
- Our paper is `well-written`, the discussion on relevant works provides `valuable technical insights` (R 7irc), and the analysis of the proposed method’s mechanism is `sufficient` (R 2KKn). Our experiments on multiple models and datasets are `comprehensive` and `persuasive` (R fd2P, 2KKn, 7irc).

Responses and Revisions:
- For Reviewer fd2P’s concerns:
  - We performed **additional human evaluations on the evaluation classifier**, providing more evidence that our metric is trustworthy.
  - We further highlighted the applicability of such an attack under the white-box setting.
  - We included more discussion of relevant finetuning-based backdoor injection methods.
  - We revised some terminology and expressions on Page 3 and Figure 2 for more clarity.
- For Reviewer 2KKn’s concerns:
  - We performed **additional evaluations on backdoored LLMs using the MMLU dataset**, further demonstrating their capabilities and effectiveness while reinforcing the validation of the proposed JailbreakEdit.
  - We clarified the threat model setting, providing more detailed scenarios involving API offerings and open-source models.
  - We corrected the typo in the definition of Node to provide a clearer description of our method.
  - We provided an explanation of the rationality of the operation to average sentence representations in retrieving $\tilde{k}$.
- For Reviewer 7irc’s concerns:
  - We conducted **additional time-cost experiments** to provide deeper insights, with results showing that attacks on all models can be executed within minutes.
  - We restate the common limitation of white-box backdooring attacks, and highlight the applicability of the proposed JailbreakEdit.
  - We explained the representability of Llama models, highlighting their status as top safety-aligned models, and noted that our experiments included not only Llama but also other LLMs.

Thank you once again for your efforts in reviewing our work. We hope our responses have adequately addressed any concerns regarding this study.

Warm Regards,
The Authors of Submission 10721
</content>
</thread_object>

</thread>


Metareview:
metareview:
All reviewers are generally positive that this work presents an interesting and effective jailbreak backdoor.

The reviewers noted that the threat model assumes quite a strong white-box attacker, but the rebuttal clarified that this is in line with prior works on LLM backdoors. Since the attack is quite simple, it is plausible that future work could improve it by relaxing the threat model.

There were also concerns about the utility of the backdoored models, which were addressed by showing evaluation results on MMLU.

Overall, I think this is a nice continuation of this line of work and I thus recommend acceptance.