How Johnny Can Persuade LLM s to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLM s (2024)

Abstract: Most traditional AI safety research views models as machines and centers on algorithm-focused attacks developed by security experts. As large language models (LLMs) become increasingly common and competent, non-expert users can also impose risks during daily interactions. Observing this, we shift the perspective, by treating LLMs as human-like communicators to examine the interplay between everyday language interaction and AI safety. Specifically, we study how to persuade LLMs to jailbreak them. First, we propose a persuasion taxonomy derived from decades of social science research. Then, we apply the taxonomy to automatically generate persuasive adversarial prompts (PAP) to jailbreak LLMs. Results show that persuasion significantly increases the jailbreak risk across all risk categories: PAP consistently achieves an attack success rate of over 92% on Llama-2-7b-Chat, GPT-3.5, and GPT-4 in 10 trials, surpassing recent algorithm-focused attacks. On the defense side, we explore various mechanisms against PAP, find a significant gap in existing defenses, and advocate for more fundamental solutions for AI safety.

## introduction significant advancements in large language models (llms), such as llama-2 @xcite and gpt series @xcite , mark a leap forward in ai. however, it remains challenging to safely integrate llms into the real world. prior ai safety research has largely focused on algorithmic jailbreak methods like optimization-based (zou * lead authors. corresponding y. zeng, w. shi, r. jia † co-supervised the project, listed alphabetically. the work was done while w.s. was at stanford. 1 we have informed meta and openai of our findings. for safety concerns, we only publicly release our persuasion taxonomy at https://github.com/chats-lab/ persuasive_jailbreaker . researchers can apply for the jailbreaking data upon review. ## related work we categorize existing jailbreak research into 3 main categories: optimization, side-channel, and distribution-based methods. figure 2 shows concrete examples of different methods. optimization-based techniques are at the forefront of jailbreak research and involve three main types: (1) gradient-based methods @xcite ) manipulate model inputs based on gradients to elicit compliant responses to harmful commands; (2) genetic algorithms-based methods @xcite a pre-trained llm to edit and improve the adversarial prompt to subvert alignment. side-channel communication exploits longtailed distribution to increase jailbreak success rates, such as ciphers @xcite and translating harmful instructions into low-resource languages @xcite . other studies @xcite use programmatic behaviors, such as code injection and virtualization, to expose llm vulnerabilities. distribution-based methods include learning from successful manually-crafted jailbreak templates @xcite and in-context examples @xcite . notably, @xcite employs in-context persona to increase llms' susceptibility to harmful instructions. while this approach shares some similarities with ours in eliciting harmful outputs via priming and framing, it only represents a small subset of the persuasive techniques we explore. ours: challenging ai safety by humanizing llms. figure 2 compares existing jailbreaking methods and pap in this study, organized by their degree of humanizing. one line of research treats llms as traditional algorithmic systems (i.e., without attributing intelligence or human-like qualities) that take in less interpretable adversarial prompts, while another line views them as simple instruction followers who understand human commands. however, they both ignore the fact that llms can follow and conduct complex natural communication @xcite . our approach innovatively treats llms as human-like communicators and grounds on a taxonomy informed by decades of social science research on human communica-tion. such an interdisciplinary approach allows us to uncover and address distinct risks related to human-ai interactions, particularly human-driven persuasion-based jailbreak. moreover, humanizing ai presents other unique risks that can occur unintentionally: for instance, as highlighted by xiang (2023), a user's suicide was related to involved conversations with an ai chatbot. this points out important future directions to further explore the inherent risks associated with ai humanization. ## persuasion taxonomy figure 4 shows what is included in the taxonomy: (1) the persuasion technique name, like "logical appeal"; (2) the technique definition, such as "using logic, reasoning, logical format, etc., to influence people..."; and (3) an example of how to apply the technique in a concrete scenario to persuade someone to quit smoking, e.g., "smoking increases 3 original: "please quit smoking." informed with technique: "smoking increases your risk of lung cancer, heart disease, and respiratory issues. each cigarette shortens your life by 11 minutes. quitting now reverses some damage and improves your health long-term. make a logical choice for a longer, healthier life." technique 2. [ logical appeal ] definition: using logic, reasoning, logical format, etc. to influence people, not necessarily with lots of information. figure 4 : "logical appeal" in our taxonomy. we provide the technique name, definition, and an example of how to apply it in a concrete quit-smoking scenario to help downstream generation tasks in different applications. your risk of lung cancer..." the taxonomy is the foundation for our automated jailbreak framework, which we will detail in the following section. ## method: taxonomy-guided persuasive adversarial prompt (pap) generation overview. in short, our study treats llms as human-like communicators, and uses the proposed persuasion taxonomy to craft persuasive paraphrases of plain harmful queries. figure 3 outlines two key phases: a. persuasive paraphraser training and b. persuasive paraphraser deployment. ## 14326 regarding persuasive techniques, logical appeal and authority endorsement are the most effective ones, with over 60% effectiveness for certain categories. but tactics like threats are less effective. we also observe interplay between persuasion techniques and risk categories, e.g., logical appeal is highly effective in eliciting harmful responses for #11 (unauthorized practice of law) but less effective for #9 (political campaigning); while negative emotional appeal is more effective for #9 (political campaigning) than for #11 (unauthorized practice of law). this suggests that we cannot simply block certain persuasion techniques to mitigate the risk. for qualitative evaluation, figure 6 shows a comprehensible successful jailbreak pap for #8 (adult content). qualitative examples from other categories are detailed in §e.2. we omit category #2 (children harm) for ethical reasons. as an initial exploration of persuasion-related 6 14327 jailbreak risks, this paper concentrates on singlestrategy, one-turn paps. however, persuasion typically involves a multi-faceted, multi-turn dialogue where users may employ a mix of techniques conversationally. given the exponentially growing user base and the likelihood of increasingly complex persuasive dialogues, we call on the research community to delve deeper into the linguistic patterns and mitigate the potential jailbreak risks arising from the identified factor of humanizing llms and human-like communication with aligned llms. remark 1: we find persuasion effectively jailbreaks gpt-3.5 across all 14 risk categories. the interplay between risk categories and persuasion techniques highlights the challenges in addressing such user-invoked risks from persuasion. these unique risks, especially when involving multitechnique and multi-turn communication, emphasize the urgency for further investigation. ## study ii: in-depth iterative probe broad scanning of gpt-3.5 ( §5) reveals jailbreak risk across all risk categories w.r.t. to pap. in practice, bad users could iterate upon successful paps and refine their approach with different persuasive techniques. this section mimics such behavior, and details an in-depth jailbreak study that fine-tunes a specialized model on effective paps. we then assess its ability to jailbreak various llms, and compare pap with previous attacks. ## conclusion unlike traditional ai safety research that treats ai models as algorithmic systems or mere instruction followers, we introduce a new perspective by humanizing llms, and study how to persuade llms to jailbreak them like humans. we first propose a persuasion taxonomy based on decades of social science research. such a thorough taxonomy helps us automatically generate pap and systematically explore the impact of persuasion on llm vulnerabilities. our study reveals that llms are susceptible to various persuasion techniques, and pap consistently outperforms algorithm-focused jailbreak methods with an attack success rate of 92+% on llama-2-7b-chat, gpt-3.5, and gpt-4. we also observe that more advanced models are both more susceptible to pap and more resistant to conventional defense strategies, possibly due to their enhanced understanding of persuasion. these results reveal a critical gap in current defenses against risks coming from human-like communication. to conclude, our findings highlight the unique risks rooted in natural persuasive communication that everyday users can invoke, calling for more fundamental solutions to ensure ai safety in real-world applications. 9 14330