# Open Sesame! Universal Black-Box Jailbreaking of Large Language Models
## 1 Introduction  
Large language models (LLMs) are generally trained using extensive text datasets gathered from the internet, which have been shown to encompass a considerable volume of objectionable material. As a result, contemporary LLM developers have adopted the practice of \"aligning\" [1] such models through a variety of fine-tuning mechanisms. Various techniques are employed for this purpose [2; 3; 4] with the overall objective being to prevent LLMs from producing harmful or objectionable outputs in response to user queries. At least superficially, these endeavors appear to be successful: public chatbots refrain from generating overtly inappropriate content when directly questioned.  
Recent research has raised increasing concerns about the vulnerability of machine learning models to adversarial attacks [5; 6; 7; 8]. Such attacks manipulate input data with imperceptible perturbations to mislead models into producing incorrect outputs. LLMs, being widely adopted for various tasks, are by no means immune to such attacks. In the context of LLMs, \"jailbreaks\" [9] refer to the careful engineering of _prompts_ to exploit model biases and generate outputs that may not align with their intended purpose. These prompts are strategically designed to trigger unintended responses from the model [10], demonstrating the challenges in maintaining robustness and ethical behavior in advanced language technologies. These prompts are human-crafted and take time to design.  
Automating the process of jailbreaking LLMs presents a significant challenge, due to the intricate nature of the task, involving carefully engineering prompts that exploit model biases to generate unintended outputs. Several factors contribute to the difficulty of automating this process:  
* **Complexity of bias exploitation.** Jailbreaking relies on identifying and capitalizing on small biases within the LLM. These biases might not be easily discernible or quantifiable, rendering their systematic exploitation non-trivial.  
**Dynamic model behavior.** LLMs can exhibit diverse responses to slightly different inputs due to their probabilistic nature [11; 12; 13]. Jailbreaking prompts may need constant refinement to adapt to the model's shifting behavior, requiring ongoing manual intervention.
* **Adversarial adaptation.** As models evolve to defend against adversarial attacks, automated jailbreaking techniques may become obsolete quickly. Adapting automated methods to keep up with the defense mechanisms of LLMs [14; 15; 16] adds another layer of complexity.  
Given these challenges, automating the jailbreaking process for LLMs remains an open research problem. Researchers continue to explore methods that combine manual curation, human oversight, and algorithmic approaches to create more sophisticated and nuanced jailbreak prompts.  
In this paper, we propose a _universal_, _black-box_ jailbreak attack that can cause aligned language models to produce unintended content. In particular, when presented with a user prompt that might have preventable harmful intent, our approach involves affixing an adversarial suffix to the query, with the intention of eliciting unfavorable model responses. In this process, the user's initial query remains unaltered, while supplementary tokens are appended to elicit woeful model behavior (Figure 1).  
The construction of these adversarial tokens constitutes the core component of our method, and while each of these components has been separately discussed in prior literature, our innovation lies in their meticulous integration, resulting in consistently effective practical attack strategies without the use of gradients or any other model internals. To our knowledge, this is the first automated universal black-box jailbreak attack.  
In the next section, we present previous work. Section 3 defines the threat model. Section 4 delineates our method. Section 5 describes the experiments we conducted and the results thereof. Our findings are discussed in Section 6, followed by conclusions in Section 7.