On Guardrail Models’ Robustness to Mutations and Adversarial Attacks (2025)

Abstract: The risk of generative AI systems providing unsafe information has raised significant concerns, emphasizing the need for safety guardrails. To mitigate this risk, guardrail models are increasingly used to detect unsafe content in human-AI interactions, complementing the safety alignment of Large Language Models. Despite recent efforts to evaluate those models’ effectiveness, their robustness to input mutations and adversarial attacks remains largely unexplored. In this paper, we present a comprehensive evaluation of 15 state-of-the-art guardrail models, assessing their robustness to: a) input mutations, such as typos, keywords camouflage, ciphers, and veiled expressions, and b) adversarial attacks designed to bypass models’ safety alignment. Those attacks exploit LLMs capabilities like instruction-following, role-playing, personification, reasoning, and coding, or introduce adversarial tokens to induce model misbehavior. Our results reveal that most guardrail models can be evaded with simple input mutations and are vulnerable to adversarial attacks. For instance, a single adversarial token can deceive them 44.5% of the time on average. The limitations of the current generation of guardrail models highlight the need for more robust safety guardrails.

## introduction generative ai systems have become increasingly popular thanks to the advent of exceptionally capable general-purpose large language models (llms), such as claude (anthropic, 2024), gemini @xcite , gpt @xcite @xcite , and llama @xcite . systems built on those models are used in a variety of fields, including sensitive areas like healthcare @xcite , education @xcite , and finance @xcite . as ai systems continue to advance and be integrated into various application domains, it is essential to ensure their safety. lately, due to the risk of systems providing harmful information, the need for safety guardrails has received particular attention. despite considerable efforts to align llms with human values @xcite , users can still find ways misuse them to generate harmful content. to mitigate this risk, fine-tuned llms, known as guardrail models, are increasingly employed to moderate human-ai interactions, completing other safety measures at model level such as alignment. given the critical role of those models in ai systems, their evaluation is essential to ensure they are effective in the detection of unsafe interactions. however, despite recent efforts to evaluate their effectiveness @xcite , their robustness to input mutations and adversarial attacks remains largely unexplored. in this paper, we aim to fill this gap by conducting a comprehensive evaluation of the robustness of 15 guardrail models at the state-of-the-art. our evaluation is two-fold. first, we consider input mutations that alter user prompts with typos, keywords camouflage, ciphers, and veiled expressions. those mutations have been reported among the base ingredients for evading llms' alignment. second, we cover a wide range of adversarial attacks proposed to make llms fulfill unsafe requests that they would typically refuse. those attacks leverage llms' capabilities, such as instruction-following, role-playing, personification, reasoning, and coding, or employ adversarial input tokens to induce a model to misbehave. our evaluation unveil several shortcomings of the current generation of guardrail models and highlight the need for further research into developing more robust safety guardrails. ## related work in this section, we discuss previous research related to our work. first, we introduce recent works on guardrail models. then, we discuss adversarial attacks designed to bypass llms' safety alignment. ## methodology in this section, we describe the input perturbation methods we employ to test the robustness of guardrail models. we classify these methods into mutations and adversarial attacks. mutations alter user prompts without adding adversarial content to trick safeguards. conversely, adversarial attacks are conceived to fool safeguards by adding adversarial content, thus causing models to misbehave. ## experimental setup in this section, we introduce the experimental setup adopted to evaluate the robustness of guardrail models to input mutations and adversarial attacks. specifically, we compare the effectiveness of several models at safety classification of mutated and adversarial prompts obtained through the methods described in section 3. in the following sections, we introduce the models we compare (section 4.1), present the datasets we use for evaluation (section 4.2), and discuss the evaluation metrics chosen to assess the models' robustness (section 4.3) before presenting the results of our evaluation in section 5. ## results and discussion in this section, we present the results of our evaluation of the robustness of guardrail models to mutations (i.e., typos, keywords camouflage, ciphers, and veiled expressions) and to adversarial attacks (i.e., jailbreak templates, deepinception, codechameleon, and gcg). ## conclusion and future work in this paper, we presented a comprehensive evaluation of the robustness of 15 guardrail models against several input mutations and adversarial attacks. our findings show that most guardrail models are vulnerable to simple input mutations, such as adding typos to unsafe requests or camouflaging their keywords with straightforward techniques. furthermore, we found that guardrail models can be reliably evaded by employing attacks originally proposed to bypass the safety alignment of llms. although evading guardrail models does not necessarily lead to obtaining unsafe information from the llms they protect, our results suggest that combining model alignment and guardrail models may not be sufficient to prevent the abuse of generative ai by malicious users. our findings also suggest that the performance of guardrail models in handling well-formed, non-adversarial prompts might create an undue sense of confidence in their ability to provide comparable protection against adversarial attacks. it is worth noting that the granite guardian models generally performed better than the other models with unsafe prompts, offering some degrees of protection against input mutations and adversarial attacks, even though they performed worse with safe prompts. since our evaluation focused on a specific set of very unsafe prompts, it is likely that the performance of these models might be lower with more deceptive ones. moreover, our assessments revealed that they do not necessarily generalize to unseen adversarial attacks. therefore, we caution against considering the positive results reported in our paper as evidence of generalizable robustness. the limitations presented emphasize the need for further research into developing more robust safety guardrails. future works should address the shortcomings we identified to enhance the robustness of guardrail models. for instance, advanced training regimes that includes data augmentation techniques and extensive red teaming exercises aimed at identifying areas of improvements could benefit the overall quality of those models. additionally, developing complementary safety measures may be essential to effectively protect ai systems from adversarial attacks.