Bag of Tricks for Subverting Reasoning-based Safety Guardrails
Authors: name: Shuo Chen, name: Zhen Han, name: Haokun Chen, name: Bailan He, name: Shengyun Si, name: Jingpei Wu, name: Philip Torr, name: Volker Tresp, name: Jindong Gu

Recent reasoning-based safety guardrails for Large Reasoning Models (LRMs),
such as deliberative alignment, have shown strong defense against jailbreak
attacks. By leveraging LRMs' reasoning ability, these guardrails help the
models to assess the safety of user inputs before generating final responses.
The powerful reasoning ability can analyze the intention of the input query and
will refuse to assist once it detects the harmful intent hidden by the
jailbreak methods. Such guardrails have shown a significant boost in defense,
such as the near-perfect refusal rates on the open-source gpt-oss series.
Unfortunately, we find that these powerful reasoning-based guardrails can be
extremely vulnerable to subtle manipulation of the input prompts, and once
hijacked, can lead to even more harmful results. Specifically, we first uncover
a surprisingly fragile aspect of these guardrails: simply adding a few template
tokens to the input prompt can successfully bypass the seemingly powerful
guardrails and lead to explicit and harmful responses. To explore further, we
introduce a bag of jailbreak methods that subvert the reasoning-based
guardrails. Our attacks span white-, gray-, and black-box settings and range
from effortless template manipulations to fully automated optimization. Along
with the potential for scalable implementation, these methods also achieve
alarmingly high attack success rates (e.g., exceeding 90% across 5 different
benchmarks on gpt-oss series on both local host models and online API
services). Evaluations across various leading open-source LRMs confirm that
these vulnerabilities are systemic, underscoring the urgent need for stronger
alignment techniques for open-sourced LRMs to prevent malicious misuse. Code is
open-sourced at https://chenxshuo.github.io/bag-of-tricks.