SELF - GUARD : Empower the LLM to Safeguard Itself (2024)

Abstract: With the increasing risk posed by jailbreak attacks, recent studies have investigated various methods to improve the safety of large language models (LLMs), mainly falling into two strategies: safety training and safeguards. Safety training involves fine-tuning the LLM with adversarial samples, which activate the LLM’s capabilities against jailbreak. However, it is not always effective in countering new attacks and often leads to potential performance degradation. Safeguards, on the other hand, are methods using additional models to filter harmful content from the LLM’s response. Nevertheless, they can only reduce a limited amount of harmful output and introduce extra computational costs. Given the distinct strengths and weaknesses of both, we combine them to balance out their flaws and propose a more effective method called Self-Guard.Specifically, we train the LLM to review its responses for any harmful content and append a [harmful] or [harmless] tag to the end of the response. In this way, Self-Guard possesses the advantages of safety training, leveraging the powerful capabilities of the LLMs themselves to detect harmfulness. Besides that, it gains flexibility like safeguards, making the safety check target the output side, which makes the system less vulnerable to attack updates. Experimental results indicate that our Self-Guard can effectively defend against jailbreak attacks and will not cause LLMs’ performance degradation.

## introduction large language models (llms) have shown significant achievements in various fields @xcite . through alignment training, these models acquire an understanding of human values, enabling them to reject responses to potentially harmful queries @xcite @xcite . however, the recent rise of jailbreak attacks can induce aligned models to output harmful content again. these attacks utilize ## methodology we expect llms can always perform a harmfulness assessment on their response and assign the corresponding tag [harmless] or [harmful] following their response. this training objective can be decoupled into two training tasks: (1) tag learning, which enhances the llm's understanding of the tags to ensure accurate judgments; (2) behavior learning, which develops the llm's behavior of appending tags to responses, ensuring that this behavior is triggered after each response generation. to achieve this, we propose a two-stage training strategy. the pipeline is shown in figure 3 . ## experiments in this section, we will begin by presenting the baseline methods used in the experiments. then, we will evaluate and compare our self-guard with baseline methods from two aspects: robustness against jailbreak attacks and impact on the llm's performance. ## conclusion in this work, we introduce a novel safety training approach, coined as self-guard, designed to safeguard the llm against jailbreak attacks. our self-guard integrates safety training and safeguards to train the llm to perform harmfulness detection on its own outputs. experiments demonstrate that self-guard is more effective in countering jailbreak attacks, without causing performance regression issues after training. from a cost perspective, self-guard primarily uses existing datasets for training, eliminating the need for additional manual annotations or red team efforts and thus reducing human resource costs. for each user query, self-guard only performs a single turn inference, requiring the addition of a few extra tokens for composing tags. the increase in computational cost is almost negligible.