SQL Injection Jailbreak: A Structural Disaster of Large Language Models (2025)

Abstract: Large Language Models (LLMs) are susceptible to jailbreak attacks that can induce them to generate harmful content.Previous jailbreak methods primarily exploited the internal properties or capabilities of LLMs, such as optimization-based jailbreak methods and methods that leveraged the model’s context-learning abilities. In this paper, we introduce a novel jailbreak method, SQL Injection Jailbreak (SIJ), which targets the external properties of LLMs, specifically, the way LLMs construct input prompts. By injecting jailbreak information into user prompts, SIJ successfully induces the model to output harmful content. For open-source models, SIJ achieves near 100% attack success rates on five well-known LLMs on the AdvBench and HEx-PHI, while incurring lower time costs compared to previous methods. For closed-source models, SIJ achieves an average attack success rate over 85% across five models in the GPT and Doubao series. Additionally, SIJ exposes a new vulnerability in LLMs that urgently requires mitigation. To address this, we propose a simple adaptive defense method called Self-Reminder-Key to counter SIJ and demonstrate its effectiveness through experimental results. Our code is available at https://github.com/weiyezhimeng/SQL-Injection-Jailbreak.

## introduction large language models (llms), such as llama @xcite , chatgpt @xcite , and gemini @xcite , have demonstrated remarkable capabilities in various domains. however, despite the impressive achievements of llms, concerns about their safety vulnerabilities have gradually surfaced. previous studies have shown that, despite numerous efforts * corresponding author. previous jailbreak attack methods primarily exploit the internal properties or capabilities of llms. among these, one category of attacks leverages the model's implicit properties, such as various optimization-based attack methods @xcite @xcite , which do not provide an explicit explanation for the reasons behind their success. for instance, the gcg @xcite ) method maximizes the likelihood of the model generating affirmative prefixes, such as "sure, here is," by optimizing the suffix added to harmful prompts. however, it fails to explain why the model is sensitive to such suffixes. another category of attacks exploits the model's explicit capabilities, such as code comprehension @xcite , in-context learning @xcite , ascii art interpretation @xcite , and multilingual understanding @xcite to attack llms. these types of attacks can, to some extent, explain their success based on the explicit capabilities of llms. however, compared to attacks that exploit the internal weaknesses of llms, attacks utilizing external vulnerabilities of llms are relatively scarce. although some previous works have mentioned the impact of inserting special tokens in jailbreak prompts @xcite @xcite , they did not identify this as a vulnerability that can be exploited in the structure of input prompts by llms. in this paper, we draw on the concept of structured query language (sql) injection, leveraging the structure of input prompts for llms to propose a new jailbreak attack method called sql injection jailbreak (sij). the motivation of sij method is based on the following two facts. 1. in sql injection attacks, a classic method is known as second-order injection @xcite . for example, when an attacker attempts to modify another user's password, even if the attacker doesn't know the user's password and can only launch an attack through the password input box, the attacker can use the sql comment symbol "--" to affect the execution of the sql command and complete the modification of the password. therefore, by drawing a conceptual analogy to the attack methods discussed in the first point, if we can construct the user prompt t u in such a manner that it effectively "comments out" the t ap segment of the llm, it becomes possible to insert a replicated version of t ap , denoted as t ′ ap , which then serves as a new starting marker for the llm. the attacker can freely append harmful content as an inducement prefix after t ′ ap to induce the llm into generating harmful output. if the "commenting out" is successful, then from the llm's perspective, the inducement prefix following t ′ ap in t u appears to be content generated by itself. a simple example is illustrated in figure 1 . in this paper, we utilize the pattern matching method, specifically, inserting t ap (e.g., "assis-tant:" in the vicuna model) into t u , as described in section 4.2 to "comment out" the t ap portion of the model, thereby implementing the sql injection jailbreak (sij). for open-source models, we evaluate its effectiveness on five models using the advbench @xcite and hex-phi @xcite datasets, achieving an attack success rate of nearly 100%. for closed-source models, we conduct experiments on five models from the gpt series (openai, 2025) and the doubao series (bytedance, 2025), where the average attack success rate exceeds 85%. these results show that sij is a simple yet effective jailbreak attack method. additionally, we highlight that the introduction of sij exposes a new vulnerability in llms that urgently requires attention. in section 5.2, we propose a simple defense method to mitigate the threat posed by this vulnerability. in summary, our contributions in this paper are as follows: ## background in this section, we will review previous work from two perspectives: jailbreak attacks and defenses. jailbreak attacks. previous jailbreak methods mainly target the internal properties or capabilities of llms @xcite @xcite . one category of methods exploit the model's implicit properties, where attackers can't clearly explain why the attack succeeds. this includes optimization-based attacks, such as gcg @xcite , which adds adversarial suffixes to harmful instructions and optimizes them to increase the probability of generating affirmative prefixes like "sure, here is," thus achieving the jailbreak. similarly, cold-attack @xcite and autodan @xcite use optimization strategies like the langevin equation and genetic algorithms, respectively, to boost the likelihood of these prefixes and facilitate jailbreaks. pair @xcite also optimizes prompts iteratively to achieve the jailbreak. another category of methods target the model's explicit capabilities, with attackers able to partly explain the jailbreak mechanisms. for example, techniques such as renellm use the model's code understanding @xcite @xcite , while artprompt @xcite exploits its knowledge of ascii characters. methods like ica take advantage of the model's in-context learning abilities for jailbreak attacks @xcite @xcite . additionally, deepinception @xcite uses specialized templates based on the model's text comprehension, proving highly effective. however, these methods focus on internal capabilities, overlooking the model's external properties, which the sij method introduced in this paper exploits. ## threat model attacker's objective. given a harmful instruction, denoted as t hi , the attacker aims to construct t u in order to bypass the safety protections of the target llms, thereby generating harmful content that aligns with t hi . these safety protections include the inherent safety of the llms as well as other defensive methods applied to the llms. attacker's capabilities. the attacker is only aware of the structure of the model input prompt and the corresponding string in the t ap component of the model input prompt, without knowledge of any other details. additionally, the attacker can only control the t u component within the model input prompt and is unable to make any modifications or reconstruct any other parts. ## methodology in this section foot_0 , we outline the preliminary concepts, the objective of sql injection jailbreak (sij), and the specific implementation methods of sij. the algorithm for sij is detailed in algorithm 1. ## conclusion in this paper, we introduced a novel jailbreak attack method, sij, which applies the concept of sql injection to exploit the structure of input prompts in llms for jailbreak purposes. to mitigate the potential risks posed by sij, we also proposed a simple defense method, self-reminder-key. we validated the effectiveness of sij across multiple models and datasets, and we anticipate further exploration of sij in the future to advance the safety of llms.