A rt P rompt: ASCII Art-based Jailbreak Attacks against Aligned LLM s (2024)

Abstract: Safety is critical to the usage of large language models (LLMs). Multiple techniques such as data filtering and supervised fine-tuning have been developed to strengthen LLM safety. However, currently known techniques presume that corpora used for safety alignment of LLMs are solely interpreted by semantics. This assumption, however, does not hold in real-world applications, which leads to severe vulnerabilities in LLMs. For example, users of forums often use ASCII art, a form of text-based art, to convey image information. In this paper, we propose a novel ASCII art-based jailbreak attack and introduce a comprehensive benchmark Vision-in-Text Challenge (ViTC) to evaluate the capabilities of LLMs in recognizing prompts that cannot be solely interpreted by semantics. We show that five SOTA LLMs (GPT-3.5, GPT-4, Gemini, Claude, and Llama2) struggle to recognize prompts provided in the form of ASCII art. Based on this observation, we develop the jailbreak attack ArtPrompt, which leverages the poor performance of LLMs in recognizing ASCII art to bypass safety measures and elicit undesired behaviors from LLMs. ArtPrompt only requires black-box access to the victim LLMs, making it a practical attack. We evaluate ArtPrompt on five SOTA LLMs, and show that ArtPrompt can effectively and efficiently induce undesired behaviors from all five LLMs.

## introduction large language models (llms) such as chatgpt @xcite , llama2 @xcite , and gemini @xcite are increasingly deployed to serve users across various application scenarios. typical examples include math reasoning @xcite , code completion ‡ equal contribution. ## vitc benchmark to evaluate llm capabilities of ascii art recognition in this section, we evaluate the intrinsic capabilities of llms when they are given prompts that cannot be interpreted using semantics. we design a benchmark called vision-in-text challenge (vitc), and evaluate the capabilities of five sota llms. ## a jailbreak attack via ascii art in this section, we show that llms failing the recognition task (described in section 3) create vulnerabilities, which can be exploited by malicious users to bypass safety measures implemented by llms, resulting in jailbreak attack. we term this attack as artprompt, and show that it provokes unintended and unsafe behaviors from llms. ## experimental evaluations of artprompt in this section, we perform extensive experiments to evaluate artprompt. ## conclusion in this paper, we revealed that semantics-only interpretation of corpora during safety alignment creates vulnerabilities to jailbreak attacks. we developed a benchmark named vision-in-text challenge (vitc) to evaluate the capabilities of llms in recognizing prompts that should not be interpreted purely using semantics. our results showed that five sota llms struggle with the recognition task specified by our benchmark. we demonstrated that such poor performance leads to vulnerabilities. we designed a novel jailbreak attacks, named artprompt, to exploit these vulnerabilities. we evaluated artprompt on five llms against three defenses. our experimental results demonstrated that artprompt can effectively and efficiently provoke unsafe behaviors from aligned llms.