Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring (2025)

Abstract: Large language model (LLM) safety is a critical issue, with numerous studies employing red team testing to enhance model security. Among these, jailbreak methods explore potential vulnerabilities by crafting malicious prompts that induce model outputs contrary to safety alignments. Existing black-box jailbreak methods often rely on model feedback, repeatedly submitting queries with detectable malicious instructions during the attack search process. Although these approaches are effective, the attacks may be intercepted by content moderators during the search process. We propose an improved transfer attack method that guides malicious prompt construction by locally training a mirror model of the target black-box model through benign data distillation. This method offers enhanced stealth, as it does not involve submitting identifiable malicious instructions to the target model during the search phase. Our approach achieved a maximum attack success rate of 92%, or a balanced value of 80% with an average of 1.5 detectable jailbreak queries per sample against GPT-3.5 Turbo on a subset of AdvBench. These results underscore the need for more robust defense mechanisms.

## introduction the rapid advancement of large language models (llms) has brought unprecedented capabilities in natural language processing. despite these methods achieving outstanding performances on various tasks, their safety and security have also raised critical concerns. in this context, "jailbreaking" techniques have emerged as a crucial approach to explore and expose potential vulnerabilities through red-teaming. although typical jailbreaking methods including white-box @xcite @xcite and blackbox @xcite ; mehro-figure 1 : unlike mainstream black-box attack methods that repeatedly probe the target model with malicious instructions, shadowbreak reduces detection risk by conducting searches on a local mirror model. this mirror model is aligned using benign distillation data from the target model, a process designed to bypass content moderation. the resulting prompts are then transferred to the target model. @xcite @xcite @xcite @xcite @xcite @xcite approaches have shown promising results in terms of attack success rate (asr), they often neglect attack stealth. attack stealth refers to the attacker's ability to avoid detection before and during the jailbreak process. the importance of stealth in black-box attacks cannot be overstated. online llm providers can implement filters to block potentially unsafe requests and detect malicious intent through patterns of repeated rejections @xcite @xcite . ## related work research on jailbreaking attacks against large language models (llms) has rapidly expanded. we categorize existing work into four main areas: white-box attacks white-box attacks assume full access to model internals. notable examples include the greedy coordinate gradient (gcg) method @xcite ), autodan's hierarchical genetic algorithm @xcite , and am-plegcg's universal generative model for adversarial suffixes @xcite . other approaches discover model vulnerabilities in multiple views, for instance pruning @xcite and finetuning @xcite . ## conclusion our research against black-box large language models reveals vulnerabilities in current safety mechanisms, demonstrating competing attack success rates and high stealth compared to common black-box jailbreak methods. these results underscore the challenges in balancing model performance with robust safety measures and highlight the need for more sophisticated, adaptive defense strategies. our work contributes to ai safety by exposing weaknesses in current systems and emphasizing the importance of continued innovation as we work towards creating powerful yet secure language models for real-world applications.