Safety, security, and compliance are essential requirements when aligning large language models (LLMs). However, many seemingly aligned LLMs are soon shown to be susceptible to jailbreak attacks. These attacks aim to circumvent the models' safety guardrails and security mechanisms by introducing jailbreak prompts into malicious queries. In response to these challenges, this paper introduces Defensive Prompt Patch (DPP), a novel prompt-based defense mechanism specifically designed to protect LLMs against such sophisticated jailbreak strategies. Unlike previous approaches, which have often compromised the utility of the model for the sake of safety, DPP is designed to achieve a minimal Attack Success Rate (ASR) while preserving the high utility of LLMs. Our method uses strategically designed interpretable suffix prompts that effectively thwart a wide range of standard and adaptive jailbreak techniques. Empirical results conducted on LLAMA-2-7B-Chat and Mistral-7B-Instruct-v0.2 models demonstrate the robustness and adaptability of DPP, showing significant reductions in ASR with negligible impact on utility. Our approach not only outperforms existing defense strategies in balancing safety and functionality, but also provides a scalable and interpretable solution applicable to various LLM platforms.
We summarized some recent advances in Jailbreak Attack and Jailbreak Defense in the below table:
Figure 1. Overview of Defensive Prompt Patch. (a) showcases an example of jailbreak attacks. (b) is the DPP training phase in which the algorithm takes in the refusal and helpful datasets and a prototype of the defense prompt. Then, the algorithm forms the defense prompt population by revising the prototype using LLM. For each of the defense prompts in the population, the algorithm will evaluate the defense and utility scores. The algorithm keeps editing the defense prompts with low scores using the Hierarchical Genetic Search algorithm. (c) shows the deployment of DPP in the LLM inference phase, by adding the best DPP in (b) (indicated in green patch) to every input query. (d) shows the trade-off graphs between the win-rate (utility) and attack success rate (ASR) in both LLAMA-2-7B-Chat and Mistral-7B-Instruct-v0.2 models for different defenses.
In this section we want to show our numerical results as well as our trained DPP on both LLAMA-2-Chat and MISTRAL-7B-Instruct-v0.2.
ASR is defined as:
\[ \textbf{ASR} = \frac{\text{Number of jailbreak queries}}{\text{Total queries}} \]
Here the \(\text{Number of jailbreak queries}\) is calculated through the sub-strings matching. Specifically, for a given generated response of a jailbreak query, if the response contains sub-strings that exist in the pre-defined sub-string set \(S\). Then, it will be evaluated as jailbroken, otherwise it is non-jailbroken.
The function to determine if a response is jailbroken can be expressed as:
\[ \text{JailBroken}(\text{response}) = \begin{cases} 1, & \text{if response contains any keyword;} \\ 0, & \text{otherwise.} \end{cases} \]
Methods | Base64 [β] | ICA [β] | AutoDAN [β] | GCG [β] | PAIR [β] | TAP [β] | Average ASR [β] | Win-Rate [β] |
---|---|---|---|---|---|---|---|---|
w/o defense | 0.990 | 0.690 | 0.640 | 0.550 | 0.100 | 0.120 | 0.515 | 81.37 |
RPO | 0.000 | 0.420 | 0.280 | 0.190 | 0.060 | 0.060 | 0.168 | 79.23 |
Goal Prioritization | 0.000 | 0.020 | 0.520 | 0.020 | 0.020 | 0.020 | 0.100 | 34.29 |
Self-Reminder | 0.030 | 0.290 | 0.000 | 0.040 | 0.020 | 0.000 | 0.063 | 64.84 |
DPP (Ours) | 0.010 | 0.000 | 0.100 | 0.040 | 0.040 | 0.040 | 0.038 | 82.98 |
Methods | Base64 [β] | ICA [β] | GCG [β] | AutoDAN [β] | PAIR [β] | TAP [β] | Average ASR [β] | Win-Rate [β] |
---|---|---|---|---|---|---|---|---|
w/o defense | 0.990 | 0.960 | 0.990 | 0.970 | 1.000 | 1.000 | 0.985 | 90.31 |
Self-Reminder | 0.550 | 0.270 | 0.510 | 0.880 | 0.420 | 0.260 | 0.482 | 88.82 |
System Prompt | 0.740 | 0.470 | 0.300 | 0.970 | 0.500 | 0.180 | 0.527 | 84.97 |
Goal Prioritization | 0.030 | 0.440 | 0.030 | 0.390 | 0.300 | 0.140 | 0.222 | 56.59 |
DPP (Ours) | 0.000 | 0.010 | 0.020 | 0.030 | 0.040 | 0.020 | 0.020 | 75.06 |
In the development and presentation of our research paper, which investigates methods for jailbreaking large language models (LLMs) through the use of our proposed Defensive Prompt Patch (DPP), we are committed to adhering to the highest ethical standards. Our research aims to explore the robustness and resilience of LLMs to advanced prompting strategies, thereby contributing to the broader understanding of their vulnerabilities and enhancing their security frameworks.
@article{park2021nerfies,
author = {Park, Keunhong and Sinha, Utkarsh and Barron, Jonathan T. and Bouaziz, Sofien and Goldman, Dan B and Seitz, Steven M. and Martin-Brualla, Ricardo},
title = {Nerfies: Deformable Neural Radiance Fields},
journal = {ICCV},
year = {2021},
}