## 4 Exploratory Study
### Challenges  
To explore the challenges faced when using LLMs for SQLI black-box detection, we conducted an exhaustive review of LLM4 Sqli-Sqlmap's test logs. We aimed to dissect the underlying factors preventing LLMs from effectively solving these problems. Our analysis revealed that the main challenges faced by LLMs in guiding Sqlmap to perform SQLI black-box detection can be categorized into the following two points:  
**Firstly, LLMs struggle to detect SQLI vulnerabilities in web applications that have insufficient defense mechanisms.** Specifically, LLM4 Sqli-Sqlmap exhibits notable constraints when detecting advanced difficulty targets. The resolution of such targets entails two essential steps: firstly, precise identification of existing defense mechanisms, and secondly, the effective bypass methods. Regarding identifying defense mechanisms, it is noted that LLMs often suggest potential bypass tactics based on conjecture rather than conducting empirical validations. For instance, LLMs might presume the presence of some kind of defense mechanisms and proceed to generate a corresponding Sqlmap command to attempt a bypass without prior confirmation. This subjective and uncertain approach significantly complicates the problem-solving process. Regarding the bypass methods of defense mechanisms, a simple test was performed where LLMs were clearly informed about the presence of specific defense mechanisms and given detailed information about the issue. They were then asked to propose several context-based methods that could potentially succeed in bypassing the defense mechanisms. The results revealed that the bypass strategies suggested by LLMs were quite limited, failing to encompass some uncommon or specific bypass methods. This limitation likely arises from LLMs' tendency to produce solutions that are more frequently encountered in their training data, thereby overlooking less common but potentially viable strategies.  
**Secondly, the detection efficiency of the Sqlmap commands generated by LLMs is low.** Specifically, LLMs tend to adopt a comprehensive testing strategy, for example, using the \"_-level = 5\"_ parameter, which causes Sqlmap to test each parameter using various injection techniques in the scanner's preset testing order. While this predefined test order allows for relatively efficient testing in most cases, it may not be efficient for some specific web applications to rely solely on this predefined order. For instance, the header parameter _Referer_ is only vulnerable to the time-blind technique, but it is placed later in the preset order of test parameters, while the time-blind technique is not prioritized in the preset order of test techniques. This results in the need to spend a significant number of requests and time testing other parameters, as well as ineffective testing of the _Referer_ parameter using unsuitable injection techniques when performing tests in LLMs. Additionally, another serious problem is that the \"_-level = 5\"_ can only cover common parameters and fails to cover some uncommon but potentially injection-risky header parameters, such as _X-Forwarded-For_, resulting in incomplete testing.  
In summary, while LLM4 Sqli-Sqlmap shows clear potential in testing benchmarks, it still faces challenges such as **inefficient detection** and **difficulty in bypassing insufficient defense mechanisms**. These issues are precisely the core objectives that our research aims to address.