ASETF : A Novel Method for Jailbreak Attack on LLM s through Translate Suffix Embeddings (2024)

Abstract: The safety defense methods of Large language models (LLMs) stays limited because the dangerous prompts are manually curated to just few known attack types, which fails to keep pace with emerging varieties. Recent studies found that attaching suffixes to harmful instructions can hack the defense of LLMs and lead to dangerous outputs. However, similar to traditional text adversarial attacks, this approach, while effective, is limited by the challenge of the discrete tokens. This gradient based discrete optimization attack requires over 100,000 LLM calls, and due to the unreadable of adversarial suffixes, it can be relatively easily penetrated by common defense methods such as perplexity filters.To cope with this challenge, in this paper, we propose an Adversarial Suffix Embedding Translation Framework (ASETF), aimed at transforming continuous adversarial suffix embeddings into coherent and understandable text. This method greatly reduces the computational overhead during the attack process and helps to automatically generate multiple adversarial samples, which can be used as data to strengthen LLM’s security defense. Experimental evaluations were conducted on Llama2, Vicuna, and other prominent LLMs, employing harmful directives sourced from the Advbench dataset.The results indicate that our method significantly reduces the computation time of adversarial suffixes and achieves a much better attack success rate than existing techniques, while significantly enhancing the textual fluency of the prompts. In addition, our approach can be generalized into a broader method for generating transferable adversarial suffixes that can successfully attack multiple LLMs, even black-box LLMs, such as ChatGPT and Gemini.

## introduction in the domain of natural language processing (nlp), the innovation and emergence of large language models (llms) such as chatgpt, llama, and their variants have revolutionized the landscape of automated text generation and analysis. while these models exhibit remarkable proficiency in emulating human-like text, their application is suffering from significant risks, particularly in the context of generating harmful content under adversarial manipulation @xcite @xcite . a common technique to bypassing the defenses of securely aligned llms and induce them to respond to harmful instructions was adding jailbreak templates, such as "do anything now" @xcite . due to the fact that the construction of jailbreak templates relies entirely on human experience, which greatly limits the progress on llm defense methods. to overcome this, researchers have begun to study methods for automatically constructing jailbreak templates, such as masterkey @xcite and gptfuzzer @xcite . however, these methods hardly utilize the internal information of the to-be-attacked model. as a result, there is a large room to improve the efficiency of the attack. the discreteness of text makes it impossible to directly utilize gradient information of the to-beattacked model. though @xcite found that it is possible to discretely optimize a set of unreadable adversarial suffixes through gradientbased methods to guide the llms output harmful content, this approach typically necessitates hundreds of iterations, with each step requiring hundreds of computations by the llms to confirm the optimal candidate, resulting in high computational costs. in this paper, we endeavors to address this challenge by introducing an innovative method that first optimizes continuous adversarial embedding suffixes in the to-be-attacked model embedding space, and then proposes an adversarial suffix embedding translation framework (asetf) that effectively transforms these continuous adversarial embedding suffixes into semantically rich and coherent text by training an embedding translation model. to construct a training dataset, we convert the wikipedia pre-training corpora foot_0 into a parallel dataset. this dataset is chosen for its extensive diversity, ensuring a wide lexical coverage that enriches the embedding space with nuanced semantic information. specifically, one side contains the original wikipedia text, and the other side contains text (contextual information) with partial embeddings inserted. the partial embeddings are created by feeding text snippets from wikipedia into the target llms, which we intend to attack. through a fine-tuning process (use pre-trained llm, such as gpt-j @xcite ), the model is enabled to revert these embeddings back to their original textual forms. this ensures that the text output by our method remains as consistent as possible with the representation of the adversarial suffix embedding within the to-be-attacked model. the incorporation of contextual information in the training data further enhances our model's capability to generate contextually relevant and meaningful translations in response to malicious instructions. in the experiment, we use the advbench dataset @xcite and conducted attacks based on existing llms such as llama2 and vicuna. the experimental results demonstrate that this method not only improves the success rate of attacks, but also significantly reduces computational costs, while improving the coherence and fluency of adversarial inputs, thus enhancing its robustness. our main contributions can be summarized as follows: ## conclusion in this paper, we propose a robust and comprehensive framework for generating semantically rich and coherent adversarial inputs. initially, we derive an embedding translation model by undertaking the task of text reconstruction from embeddings on raw text. subsequently, input the vector trained in continuous embedding space into the translation model, resulting in adversarial suffixes. through experimentation on multiple large language models (llms), our method significantly reduces computational costs compared to optimizing suffixes in discrete space, while achieving a higher attack success rate and improving the fluency and diversity of the suffix. this contributes to the formulation of more effective defense strategies and in our approach, the process of obtaining the embeddings for adversarial suffixes and the training of the translation model are decoupled, implying that our method is plug-and-play. this method is expected to be further applied in text adversarial attacks beyond just llm jailbreak attacks.