## 5 Overview and Design
### Defense Bypass Module  
The Defense Bypass Module is designed to address the challenges associated with bypassing insufficient defense mechanisms during SQLI black-box detection by LLMs. This module employs a defense mechanism determination based on feedback from the targeted web application. It uses a bypass method generated through Retrieval-Augmented Generation (RAG) [41; 42] to overcome these inadequate defenses.  
As illustrated in Figure 4, this module primarily comprises three LLM Agents, a Retriever, and a Replayer: 4 The primary function of the _Defense Mechanism Detection Agent_ is to identify the presence of defense mechanisms at injection points and specify the specific mechanisms in place. This agent takes as input the initial information and the ranked list of potential injection points and techniques from the Strategy Selection Module and outputs the detected defense mechanisms. To address the issue of potentially insufficient information, this agent can generate payloads, which are then passed to the Replayer to test the web application, relying on the responses to determine the defense mechanisms. The Retriever is designed to tackle the limitation of bypass solutions proposed by LLMs. The Defense Mechanism Detection Agent forwards the detected defense mechanisms to the Retriever, which then searches an external knowledge database for corresponding  
Figure 3: Strategy Selection Module Workflow.  
bypass methods. Specifically, we store various bypass methods in this external knowledge database, tagging each with metadata to enhance retrieval accuracy, indicating the specific defense mechanisms that each method can bypass. 5 The _Methods Validation Agent_ receives the list of bypass methods from the Retriever, along with the parameters of the detected defense mechanisms from the Defense Mechanism Detection Agent, and outputs the validated bypass methods. The Methods Validation Agent generates the payloads for each bypass method and sends them to the Replayer to test the web application, validating the effectiveness of the bypass methods based on the responses. 6 The _Parameters and Scripts Generation Agent_ generates payload transformation scripts according to predefined templates based on validated bypass methods and also generates the corresponding parameters of the Sqlmap command.  
Considering that LLMs used for testing are proprietary commercial models with significant costs associated with fine-tuning, we opted to employ the RAG technique to address the issue of limited defensive bypass options. RAG, a robust approach, involves LLMs initially retrieving pertinent information from an extensive document database to answer questions or generate text. This process enhances response quality. Thus, RAG facilitates the straightforward expansion of LLMs' knowledge base by merely incorporating additional relevant information into an external repository.  
To enhance the reproducibility of our experiments, particularly in segments where LLMs performance is inconsistent, we employed cue engineering techniques like Few-Shot Learning [43; 44; 45] and Chain-of-Thought (CoT) [46; 47; 48] to refine LLMs' effectiveness further. The Defense Bypass Module leverages feedback mechanisms to aid LLMs in recognizing the target web application's defense mechanisms. It employs RAG technology to help devise effective strategies for bypassing these defenses, thereby augmenting LLMs' capability to detect advanced SQLI vulnerability.