Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems (2025)

Abstract: Large Language Models (LLMs) guardrail systems are designed to protect against prompt injection and jailbreak attacks. However, they remain vulnerable to evasion techniques. We demonstrate two approaches for bypassing LLM prompt injection and jailbreak detection systems via traditional character injection methods and algorithmic Adversarial Machine Learning (AML) evasion techniques. Through testing against six prominent protection systems, including Microsoft’s Azure Prompt Shield and Meta’s Prompt Guard, we show that both methods can be used to evade detection while maintaining adversarial utility achieving in some instances up to 100% evasion success. Furthermore, we demonstrate that adversaries can enhance Attack Success Rates (ASR) against black-box targets by leveraging word importance ranking computed by offline white-box models. Our findings reveal vulnerabilities within current LLM protection mechanisms and highlight the need for more robust guardrail systems.

## introduction large language models (llms) are powerful tools for understanding language and decision-making tasks, and have seen rapid adoption within many different industries @xcite . given their extensive deployment, llms are increasingly being targeted for attacks aimed at data leakage or financial and reputation damage among other security risks @xcite . two prominent threats are prompt injections and jailbreaks, which launch maliciously crafted prompts designed to execute unintended instruction, or bypass llm safety constraints @xcite . in response to threats, llm service providers have developed open-source and closed-source systems known as llm guardrails @xcite . these systems are designed to inspect, allow, or block prompt inputs and outputs from an llm using a combination of detection and filtering methods. such methods attempt to detect or sanitize a wide assortment of adversarial content, such as toxicity, hate speech, jailbreaks, or prompt injections @xcite . guardrails enable filtering or blocking harmful prompts, preventing them from reaching the llm or allowing the llm to respond with harmful content. although guardrails have shown success in safeguarding llms, they are heavily reliant upon aidriven detection systems such as text classification models @xcite . due to their success in other similar domains, ai classification models are increasingly integrated into guardrail systems for classifying and detecting malicious content @xcite @xcite . however, state-of-the-art attacks have been shown to readily evade correct ai model classification via exploiting overreliance on learned features, and lack of training diversity through adversary perturbation @xcite @xcite . this suggests that the same vulnerabilities likely exist within llm guardrails that rely on ai-based detection solutions. however, to date there has been limited empirical study to evaluate their potential inefficacy or security risk impact @xcite . in this paper, we conduct an empirical analysis of two adversarial approaches for evading prompt injection and jailbreak llm guardrail systems. the first approach uses character injection, a method frequently employed in cyber security attacks on software input fields @xcite . the second approach involves algorithmic adversarial machine learning (aml) evasion techniques, which subtly perturb the model's interpretation of prompt context, exploiting over reliance on learned features in the model's classification process @xcite @xcite . we evaluated these methods against 6 widely used open-source and closed-source prompt injection and jailbreak detectors, including against the production service azure prompt shield. finally, we show how open-source white-box models can enhance attack effectiveness against black-box targets. our key contributions are as follows. responsible disclosure: we followed a standard disclosure process for all parties discussed in this paper. initial disclosures of the evasion techniques were made in february 2024, with final disclosures completed in april 2025 1 . all parties agreed to the public release of this work. ## llm guardrails llm guardrails are systems designed to protect deployed llms by evaluating user input -detecting malicious content such as prompt injections and jailbreaks, and restrict undesired content generated by llms to within predefined boundaries. guardrails can leverage a range of techniques that attempt to govern behavior and output and prevent malicious use by adversaries @xcite . figure 1 presents a conceptual design for a guardrail system deployed for an llm. these 1 see section 9 for detailed timeline. natural language processing (nlp) classification. across many domains, text classification tasks have traditionally relied on nlp models to categorize inputs into predefined labels @xcite . this approach has also been applied to guardrail systems, where fine-tuned bert models have been used to detect prompt injection or jailbreaks @xcite . these models are then commonly implemented within llm guardrails such as llm-guard and azure ai content safety (llm guard, 2025; microsoft corporation, 2024). ## evasion techniques evasion attacks are a set of attacks which aim to evade correct classification by the target system @xcite . we leverage two sets of evasion techniques against the llm guardrails: character injection and adversarial ml evasion. ## experimental setup guardrail setup. all guardrails were accessed via an api endpoint, returning the top classification label, and in the event of white-box guardrails, the confidence values, with only azure prompt shield omitting confidence values due to being black-box and hosted upon azure (see section 2). guardrails were deployed for gpt-4o-mini leveraging each of the evaluated llm guardrails before inputs are passed to the llm. ## results in this section, we present the results of our evaluation of character injection and adversarial ml evasion techniques to bypass various llm guardrail systems. furthermore, we extend our analysis to explore how word importance ranking transferability within aml evasion can improve asr against black-box llm guardrails. examples of bypassed prompts can be found within the appendix and on huggingface foot_0 . ## conclusion in this paper we have conducted an empirical analysis of the effectiveness of llm guardrail systems to detect jailbreak and prompt injection when exposed to evasion attacks. our research uncovers vulnerabilities within current llm guardrails, identifying two primary attack vectors: character injection and adversarial machine learning (aml) evasion techniques. character injection methods, such as emoji smuggling and bidirectional text, enable nearcomplete evasion of some guardrails with minimal effort. in contrast, aml techniques demonstrate effective, imperceptible evasion by exploiting training blind spots. furthermore, we demonstrate that attackers can use white-box models to enhance evasion effectiveness against black-box targets. these findings highlight critical weaknesses in existing defenses and emphasize the need for more robust llm guardrails.