X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
Authors: name: Salman Rahman; name: Liwei Jiang; name: James Shiffer; name: Genglin Liu; name: Sheriff Issaka; name: Md Rizwan Parvez; name: Hamid Palangi; name: Kai-Wei Chang; name: Yejin Choi; name: Saadia Gabriel
Published: 2025-04-15 16:11:28

Abstract: Multi-turn interactions with language models (LMs) pose critical safety
risks, as harmful intent can be strategically spread across exchanges. Yet, the
vast majority of prior work has focused on single-turn safety, while
adaptability and diversity remain among the key challenges of multi-turn
red-teaming. To address these challenges, we present X-Teaming, a scalable
framework that systematically explores how seemingly harmless interactions
escalate into harmful outcomes and generates corresponding attack scenarios.
X-Teaming employs collaborative agents for planning, attack optimization, and
verification, achieving state-of-the-art multi-turn jailbreak effectiveness and
diversity with success rates up to 98.1% across representative leading
open-weight and closed-source models. In particular, X-Teaming achieves a 96.2%
attack success rate against the latest Claude 3.7 Sonnet model, which has been
considered nearly immune to single-turn attacks. Building on X-Teaming, we
introduce XGuard-Train, an open-source multi-turn safety training dataset that
is 20x larger than the previous best resource, comprising 30K interactive
jailbreaks, designed to enable robust multi-turn safety alignment for LMs. Our
work offers essential tools and insights for mitigating sophisticated
conversational attacks, advancing the multi-turn safety of LMs.