Constitutional Classifiers: Defending against Universal Jailbreaks
  across Thousands of Hours of Red Teaming
Authors: name: Mrinank Sharma; name: Meg Tong; name: Jesse Mu; name: Jerry Wei; name: Jorrit Kruthoff; name: Scott Goodfriend; name: Euan Ong; name: Alwin Peng; name: Raj Agarwal; name: Cem Anil; name: Amanda Askell; name: Nathan Bailey; name: Joe Benton; name: Emma Bluemke; name: Samuel R. Bowman; name: Eric Christiansen; name: Hoagy Cunningham; name: Andy Dau; name: Anjali Gopal; name: Rob Gilson; name: Logan Graham; name: Logan Howard; name: Nimit Kalra; name: Taesung Lee; name: Kevin Lin; name: Peter Lofgren; name: Francesco Mosconi; name: Clare O'Hara; name: Catherine Olsson; name: Linda Petrini; name: Samir Rajani; name: Nikhil Saxena; name: Alex Silverstein; name: Tanya Singh; name: Theodore Sumers; name: Leonard Tang; name: Kevin K. Troy; name: Constantin Weisser; name: Ruiqi Zhong; name: Giulio Zhou; name: Jan Leike; name: Jared Kaplan; name: Ethan Perez
Published: 2025-01-31 01:09:32

Abstract: Large language models (LLMs) are vulnerable to universal jailbreaks-prompting
strategies that systematically bypass model safeguards and enable users to
carry out harmful processes that require many model interactions, like
manufacturing illegal substances at scale. To defend against these attacks, we
introduce Constitutional Classifiers: safeguards trained on synthetic data,
generated by prompting LLMs with natural language rules (i.e., a constitution)
specifying permitted and restricted content. In over 3,000 estimated hours of
red teaming, no red teamer found a universal jailbreak that could extract
information from an early classifier-guarded LLM at a similar level of detail
to an unguarded model across most target queries. On automated evaluations,
enhanced classifiers demonstrated robust defense against held-out
domain-specific jailbreaks. These classifiers also maintain deployment
viability, with an absolute 0.38% increase in production-traffic refusals and a
23.7% inference overhead. Our work demonstrates that defending against
universal jailbreaks while maintaining practical deployment viability is
tractable.