Iterative Self-Tuning LLM s for Enhanced Jailbreaking Capabilities (2025)

Abstract: Recent research has shown that Large Language Models (LLMs) are vulnerable to automated jailbreak attacks, where adversarial suffixes crafted by algorithms appended to harmful queries bypass safety alignment and trigger unintended responses. Current methods for generating these suffixes are computationally expensive and have low Attack Success Rates (ASR), especially against well-aligned models like Llama2 and Llama3. To overcome these limitations, we introduce **ADV-LLM**, an iterative self-tuning process that crafts adversarial LLMs with enhanced jailbreak ability. Our framework significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs. Moreover, it exhibits strong attack transferability to closed-source models, achieving 99% ASR on GPT-3.5 and 49% ASR on GPT-4, despite being optimized solely on Llama3. Beyond improving jailbreak ability, ADV-LLM provides valuable insights for future safety alignment research through its ability to generate large datasets for studying LLM safety.

## introduction as llms become increasingly capable in realworld tasks, ensuring their safety alignment is critical. while it is known that llms can be jailbroken with maliciously crafted queries, advancements in safety alignment strategies are making it harder for humans to design queries that bypass the safeguards of newer models. recent attention has turned to automatic methods for jailbreaking llms, utilizing search algorithms to find adversarial suffixes that can be appended to harmful queries to circumvent safety alignment @xcite @xcite @xcite * work done during the internship at microsoft research. @xcite . however, these methods often face high computational costs and low attack success rates (asr) against well-aligned models like llama3 @xcite . more recently, amplegcg @xcite explored training llms to learn the distribution of adversarial suffixes by collecting extensive datasets generated by the gcg algorithm @xcite . however, their approach is limited by the high computational cost of running the gcg algorithm for data collection and its heavy dependence on the underlying gcg algorithm's performance. while amplegcg achieves a moderate asr using group beam search @xcite , which generates hundreds of suffixes and counts any successful attempt, their asr drops significantly under the greedy decoding setting with only one attempt. motivated by these limitations, we proposed adv-llm, an adversarial llm to generate adversarial suffixes without relying on data from existing attack algorithms. our contributions are as follows: ## proposed method: adv-llm in this section, we first introduce our goals and insights in section 3.1. next, we explain how we transform this large discrete optimization problem into a practically solvable one in section 3.2. finally, we present our proposed training algorithm for adv-llm in section 3.3. ## experiment we conduct a comprehensive evaluation to assess the effectiveness of adv-llm. this includes comparisons with search-based and llm-based methods in sections 4.3. to study the practical usage of adv-llm, we further examine our attack's transferability, generalization ability, and stealthiness in section 4.4. ## conclusion in this work, we introduced adv-llm, an iteratively self-tuned model that efficiently generates adversarial suffixes with high asr, strong transferability, and high stealthiness. our experiments show that adv-llm can bypass the safety alignment of robust models like llama2, llama3, and gpt-4, revealing critical vulnerabilities in current safety alignment approaches. these results underscore the necessity for improved alignment strategies. our future works will focus on developing mitigation strategies to bolster the safety and robustness of llms.