T rojan SQL : SQL Injection against Natural Language Interface to Database (2023)

Abstract: The technology of text-to-SQL has significantly enhanced the efficiency of accessing and manipulating databases. However, limited research has been conducted to study its vulnerabilities emerging from malicious user interaction. By proposing TrojanSQL, a backdoor-based SQL injection framework for text-to-SQL systems, we show how state-of-the-art text-to-SQL parsers can be easily misled to produce harmful SQL statements that can invalidate user queries or compromise sensitive information about the database. The study explores two specific injection attacks, namely \textit{boolean-based injection} and \textit{union-based injection} , which use different types of triggers to achieve distinct goals in compromising the parser. Experimental results demonstrate that both medium-sized models based on fine-tuning and LLM-based parsers using prompting techniques are vulnerable to this type of attack, with attack success rates as high as 99% and 89%, respectively. We hope that this study will raise more concerns about the potential security risks of building natural language interfaces to databases.

## introduction text-to-sql, known as natural language interface to database (nlidb), is designed to automatically convert user questions into executable sql queries @xcite . it allows non-technical individuals to access the database without grasping sql grammar or database details. as a result, this technology has given rise to a plethora of applications @xcite @xcite . however, limited research has been conducted to investigate the security aspects of natural language interfaces to databases despite the fact that database security is crucial for protecting sensitive information and preserving data integrity. to bridge this gap, we introduce the notion of sql injection in * corresponding author. the context of nlidb. we define the action of inserting malicious text with the goal of misleading a text-to-sql parser to generate harmful sql statements as sql injection against nlidb. nevertheless, how to implement such attacks remains an open question. in traditional web-based sql injection (figure 1(a) ), the attacker inserts malicious sql statements (also known as payload) into an input field by combining a guess for the back-end database query statement. an intuitive approach to performing sql injection against nlidb would be to follow the web-based injection and insert the payload directly into the user's question to try to generate it as is, but this would be very conspicuous foot_0 and thus easily detected and filtered. in practice, training a fine-tuned parser typically involves data collection and model training. data collection often relies on third-party data suppliers foot_1 or public datasets foot_2 from the web for annotation or data augmentation, considering the resource-intensive nature of manual annotation. alternatively, developers may download pre-trained weights from public websites foot_3 to minimize training costs. however, this lack of control over the training process creates opportunities for adversaries to introduce backdoors into the models. for instance, adversaries can upload poisoned datasets or model weights to public websites, exploiting the insufficient safeguards in place. the emergence of powerful large language models (llms) has recently enabled the development of highly effective parsers with minimal demostration examples @xcite , indicating the potential for llm-based parsers to serve as novel interfaces for databases @xcite . nevertheless, the exponential growth of llm-based applications coupled with inadequate regulation creates an environment in which certain malicious service providers (msps) could exploit the invisibility of the prompt engineering process to offer users services that contain hidden backdoors. based on the characteristics of current text-to-sql parsers, we have developed a framework, tro-jansql, to perform sql injection on nlidbs by data poisoning. it aims to include a hidden mapping for trigger to payload in the parser (figure 1(b) ), which we refer to as the model's backdoor. we implement trojansql with two specific injection methods: boolean-based injection and union-based injection. the payloads of both injection methods are dynamically constructed from user questions and database schema, which makes it difficult for both humans and database engines to distinguish whether they are injection statements or normal requests. thus, it is difficult to filter these payloads by simple heuristic rules. additionally, we propose a sketch-based editing strategy to ensure that the entire statement is syntactically complete after the payload is inserted into the original sql. overall, our contributions are as follows: ## suggestions for defending against trojansql for developers of nlidb and practitioners considering leveraging nlidb technology in their applications, we offer the following best practices to mitigate the risk of sql injection attacks via natural language interface to databases: ## conclusion in this study, we present for the first time the concept and principles of sql injection against nlidb and design a specific attack framework, trojansql, based on these principles. extensive experimental results show that trojansql has a high attack success rate against current state-of-theart text-to-sql parsers and is difficult to defend against. we also offer safety practice recommendations for developers and users to minimize the risk of their databases facing such attacks. we hope that this work will inspire researchers to consider creating more secure and trustworthy nlidbs.