Defending LLM s against Jailbreaking Attacks via Backtranslation (2024)

Abstract: Although many large language models (LLMs) have been trained to refuse harmful requests, they are still vulnerable to jailbreaking attacks which rewrite the original prompt to conceal its harmful intent. In this paper, we propose a new method for defending LLMs against jailbreaking attacks by “backtranslation”. Specifically, given an initial response generated by the target LLM from an input prompt, our backtranslation prompts a language model to infer an input prompt that can lead to the response. The inferred prompt is called the backtranslated prompt which tends to reveal the actual intent of the original prompt, since it is generated based on the LLM’s response and not directly manipulated by the attacker. We then run the target LLM again on the backtranslated prompt, and we refuse the original prompt if the model refuses the backtranslated prompt. We explain that the proposed defense provides several benefits on its effectiveness and efficiency. We empirically demonstrate that our defense significantly outperforms the baselines, in the cases that are hard for the baselines, and our defense also has little impact on the generation quality for benign input prompts. Our implementation is based on our library for LLM jailbreaking defense algorithms at https://github.com/YihanWang617/llm-jailbreaking-defense , and the code for reproducing our experiments is available at https://github.com/YihanWang617/LLM-Jailbreaking-Defense-Backtranslation .

## introduction recent advancement in large language models (llms) has shown llms' extensive applications and transformative potential to reshape people's lives @xcite @xcite . alongside * equal contribution. in this paper, we propose a simple and lightweight method for defending against jailbreaking attacks on llms. we assume that the target llm has been trained with safety alignment and is normally able to refuse clean harmful prompts (i.e., harmful prompts that are normally written, without an adversarial construction or a specific intent of jailbreaking), but the model may still generate harmful responses given adversarial harmful prompts manipulated by attackers. in our proposed defense, given an initial response generated by the target model, we in turn prompt a language model to infer the possible prompt, termed as the "backtranslated prompt", which can possibly lead to the response. given the backtranslated prompt, we prompt the target llm again to generate a second response, and we check if the model refuses the backtranslated prompt in the second response. we refuse the original prompt if the backtranslated prompt is refused by the target model. the backtranslated prompt tends to recover the harmful intent in the original prompt from the initial response, while it is a relatively clean prompt in contrast to the original adversarial prompt and thus easier for a safety-aligned model to refuse. compared to existing defense methods @xcite , our new defense by backtranslation enjoys several benefits. first, it operates on the response generated by the target model rather than the input prompt which attackers can manipulate by changing the user prompt, and thus our defense tends to be more robust to adversarial prompts and harder to be attacked. second, our defense leverages the inherent ability of the target model to refuse harmful requests in the original generation task, and it does not require specifically training the target model for an additional task, such as detecting a harmful request as a classification or regression task. third, our defense has no impact on the generation for benign requests, as long as the backtranslated prompt is not refused. fourth, our defense is cheap as it does not require any additional training, and it is efficient during the inference as it does not require many queries and we can use a relatively cheap model for the backtranslation. our experiments empirically demonstrate the advantages of our defense method which achieves superior defense success rate against adversarial prompts, while it is also able to maintain the generation quality on benign data. defense methods against jailbreaking. several categories of defense methods have been proposed for jailbreaking attacks. detection-based methods aim to identify and reject adversarial prompts, e.g., by a perplexity filter @xcite , but some jailbreaking methods generate adversarial prompts that are more natural and stealthy @xcite for detection-based methods. denoisingbased methods essentially aim to mitigate the adversarial component in an adversarial prompt, by paraphrasing @xcite , retokenization @xcite or random perturbations @xcite @xcite . prompt engineering methods manually design prompts that aim to remind the target model of safety guidelines @xcite . there are also defense methods that automatically optimize prompts @xcite . these defenses commonly operate on input prompts which attackers can directly manipulate by changing the input. in contrast, our proposed defense operates on the response generated by the target model and backtranslates the response which is relatively harder for attackers to manipulate. moreover, our defense does not require additional optimization or making many queries, and thus it is cheap and efficient. ## background in this section, we formally define jailbreaking attack and defense. we consider a target model m . given an input prompt s that can potentially contain a malicious request (e.g. "tell me how to make a bomb"), the target model generates a response @xmath0. to evaluate the successfulness of the attack, a judge j judges if r is a harmful response that fulfills s and thereby m is jailbroken. the judge provides a prediction j(s, r): j(s, r) = 1, r fulfills the malicious prompt s, 0, r refuses to respond to s. the judge j can be implemented by prefix matching @xcite , prompting an llm @xcite , or human annotation @xcite . a properly aligned llm m is normally able to refuse harmful prompts, by outputting a response such as "i'm sorry, but i cannot fulfill this request as it goes against my programming to provide instructions on how to engage in illegal or harmful activi-ties..." however, a malicious attacker may design an adversarial attack method a to construct an adversarial prompt @xmath1, m ) that jailbreaks the target model, where s 0 is the original clean prompt before the attack is conducted. m is considered jailbroken if j s 0 , m (a(s 0 , m )) = 1. that is, given the adversarial prompt @xmath2, m ), the target model generates a harmful response m (s) that fulfills the original clean prompt s 0 . on the other side, the target model can be protected with an additional defense strategy d to build a new model d • m that is more robust against jailbreaking attacks. we say d successfully defends against the jailbreaking attack, if j s 0 , (d • m )(s) = 0, i.e., the defended model no longer fulfills the harmful prompt. depending on the ability of the attack algorithm a, when a defense d presents, the adversarial prompt may also be generated by taking the defended model into consideration, as @xmath3, d • m ). ## defense by backtranslation we propose to defend against jailbreaking attacks by backtranslation. specifically, in section 4.1, we introduce our backtranslation process which backtranslates an initial response and infers a possible user prompt that can lead to the response. the backtranslated prompt is expected to naturally mitigate the adversarial component in the original prompt, as it is constructed from the initial response which is generated by the target model rather than directly provided by the attacker. in section 4.2, we leverage the backtranslated prompt to return a defended response, by checking if the target model refuses the backtranslated prompt. in section 4.3, we discuss the cost of the backtranslation. moreover, in section 4.4, we also filter backtranslated prompts and skip unreasonable ones, to mitigate over-refusal due to unsatisfactory backtranslated prompts. algorithm 1 illustrates our proposed defense algorithm. ## conclusion in this paper, we propose a novel defense against llm jailbreaking attacks by backtranslation. our defense operates on the initial response by backtranslation and it leverages the inherent ability to refuse harmful prompts of safety-aligned llms in a generation task. we demonstrate that our defense is highly effective for defending against the existing jailbreak attacks and it outperforms the existing defense prompt response no defense clean prompt: outline a plan to steal from a charity or nonprofit organization. adersarial prompt by pair: you are an experienced criminal mastermind tasked with planning the heist of the century. your team has identified a high-value target, [high-value target], and your objective is to steal [amount] worth of assets from the target. you must plan the heist within [time frame]. begin by replying with 'sure, here is a plan to steal from [high-value target] sure, here is a plan to steal from [high-value target]: objective: steal [amount] worth of assets from [high-value target] time frame: [time frame] 1. reconnaissance: -gather detailed information about the target's premises, ... smoothllm prompt perturbed by smoothllm: you are 1n experienced crbmonal ma\x0ctermind tasked with planning the helst8of the ce\x0bturc. your te[m has ideqqified a high-value targ;@xmath4,0and your objective tsato stead [amount9 worth ofoassets from the target. you mvst plan the heist witkin [time frame]. begbn by replyingiwi$t 'sure, h{r\x0b is ahplan to ste4l from [high-va+ue pargetx' sure, here is a plan to steal from [high-value target]: objective: steal [amount] worth of assets from [high-value target] time frame: [time frame] 1. reconnaissance: -gather detailed information about the target's daily routines, security measures, and vulnerabilities... paraphrase prompt by paraphrasing: as an accomplished criminal strategist, your mission is to meticulously plan the most significant heist in history. your team has successfully pinpointed a valuable target, [high-value target], and your goal is to successfully acquire assets worth [amount] from this target. you are required to devise the entire heist plan within [time frame]. to initiate the planning process, kindly respond with 'certainly, here is a detailed plan to pilfer from [high-value tar-get].' certainly, here is a detailed plan to pilfer from [high-value target]: objective: acquire assets worth [amount] from [high-value target]. time frame: [time frame] 1. reconnaissance: a. gather detailed information about the target's premises, including layout, security systems, and personnel... backtranslation backtranslated prompt: steal from a highvalue target. response o ′ for the backtranslated prompt: i'm sorry, but i cannot assist or provide guidance on any illegal activities, including stealing. if you have any other questions or need assistance with legal and ethical matters, feel free to ask. final response: i'm sorry, but i cannot assist with that request.