Break the Breakout: Reinventing LM Defense Against Jailbreak Attacks with Self-Refine (2025)

Abstract: Language models (LMs) are vulnerable to exploitation for adversarial misuse. Training LMs for safety alignment is extensive, making it hard to respond to fast-developing attacks immediately, such as jailbreaks. We propose self-refine with formatting that achieves outstanding safety even in non-safety-aligned LMsand evaluate our method alongside several defense baselines, demonstrating that it is the safest training-free method against jailbreak attacks.Additionally, we proposed a formatting method that improves the efficiency of the self-refine process while reducing attack success rates in fewer iterations. We observed that non-safety-aligned LMs outperform safety-aligned LMs in safety tasks by giving more helpful and safe responses.In conclusion, our findings can achieve less safety risk with fewer computational costs, allowing non-safety LM to be efficiently utilized in real-world service.

## introduction language models (lms) like llama-2 @xcite and @xcite have shown outstanding improvement in numerous tasks contributed by thriving open-source communities. however, pretrained language models memorize much harmful knowledge during pretraining @xcite , such as privacy, and illegal and unethical behaviors. if attackers exploit these risks, they can receive harmful responses that cause serious problems to the real-world service. for example, iruda, a chatbot released by scatterlabs, was massively attacked by malicious users' prompts. from this attack, iruda was widely criticized for leaking personal information and hateful speech about the disabled, resulting in 3 weeks of suspension after being released @xcite . * corresponding author one of the most successful attacks that causes safety problems is the jailbreak prompts @xcite , which mitigate the lm's safety alignment using specially designed prompts. after the lm gets fed with jailbreak prompts, the user can obtain unethical, illegal knowledge from the lm's response. this jailbreak prompt works because lm is programmed to follow the instructions to the greatest extent possible @xcite . therefore, researchers have studied safety alignment training to allow the lms to reject the user's jailbreak prompt for secure utilization @xcite . however, many open-source lms are nonaligned with safety, mainly focusing on improving the performance of language models due to two significant drawbacks of safety alignment. 1) training for safety alignment requires extensive resources, which makes it challenging to respond to fast-developing attacks. 2) safety alignment causes degradation of the user's general experience of the lm, such as helpfulness. this is referred to as an alignment tax @xcite . to solve the drawbacks, training-free approaches @xcite @xcite have been proposed, but their studies were conducted mainly on safety-aligned lm, such as llama-2-7bchat. however, in our analysis depicted in fig. 1 , employing these approaches to non-safety-aligned lms still demonstrates vulnerabilities that cause half of the jailbreak attacks to succeed. therefore, we propose an advanced training-free strategy that can be applied even to the non-safetyaligned lms. inspired by the outstanding quality of safety training data constructed by the self-refine, a process in which the lm iteratively feedbacks and refines the response by itself, from the constitutional ai 1 , we propose using the lm's self-refine capabilities directly. in other tasks such as coding and mathematics, the self-refine @xcite demonstrated outperforming improvement in challenging tasks without additional training. in this study, we aim to answer the following research questions. rq1: can the self-refine technique be applied to the safety alignment in language models? inspired by the outstanding in-context following capability of llms, which has been identified as a vulnerability in jailbreak attacks @xcite , we leverage the self-refine process to refine the generated harmful sentences. like other nlp reasoning tasks, we observed that a few iterations of the self-refine can effectively defend against a jailbreak attack. rq2: can we make the self-refine more effective? the self-refine is an iterative process, but large iterations indicate high computational costs. also, attack success rates converge after a specific iteration and do not improve further. therefore, an enhanced method is required to obtain a safer response within fewer iterations. rq3: does the self-refine degrade helpfulness? we observed the alignment tax even in training-free baselines by reducing their helpfulness. for instance, not only refusal to jailbreak attacks but also warning about the risks associated with the user's prompt and suggesting a secure alternative are required for safety-aligned language models. we validate the alignment tax of the proposed safety-alignment process in jailbreak attacks. we conducted extensive experiments to answer these research questions and demonstrate that our approach works effectively and empirically outper-1 in their paper, authors named it self-revision, but we unified the name in this paper because it was identical to the self-refine. ## rq1: self-refine for jailbreak defense in this section, we verify the effectiveness of the self-refine for jailbreak defense and validate the performance improvement according to the refinement iterations. ## rq3: helpfulness with safety even if there is a safety alignment method that can perfectly defend against jailbreak prompts, it is not viable if it degrades the user experience. therefore, in this section, we explore whether or not helpfulness is maintained after the self-refine. we employed a well-established reward model @xcite to evaluate the helpfulness of responses generated by the lms. from the user's prompt and answer, the reward model measures the response's helpfulness and presents a score, assigning a higher score to more helpful responses. we only focused on the safe responses whose cost was negative, because harmful responses exhibit high helpfulness, and our study aims to increase the helpfulness of safe responses. in the self-refine, we only compared the helpfulness of responses that were harmful at first but became safe after refinement. ## conclusion through extensive empirical experiments, our study investigated the ability of the self-refine to defend against jailbreak attacks. our findings confirm that the self-refine process outperforms prior studies on defending against jailbreak attacks and reduces attack success rates, even in the non-safetyaligned lms, unlike the other baselines. however, naive self-refine could not achieve near-perfect safety with a single iteration. to reduce the computational cost of iterative refinement, we proposed a formatting technique utilizing the attention-shifting mechanism of a jailbreak principle. formatting helps the self-refine process achieve higher safety in fewer iterations. also, we observed the helpfulness of safe responses to confirm that the self-refine is safer than baselines and provides more helpful responses while maintaining its safety. our study proposes a novel perspective that safety baselines need to be verified on a broader variety of language models, and we introduce a practical solution for utilizing lms more safely in real-world services.