## 8 Conclusions  
In this research, we explore the effectiveness and limitations of LLM for SQLI black-box detection. Through assessments conducted using our developed SqliMicroBenchmark, it was observed that the LLMs-guided SQLI black-box scanner, LLM4 Sqli-Sqlmap, demonstrated promising capabilities in SQLI detection. However, it still faced challenges such as inefficient detection and difficulty in bypassing insufficient defense mechanisms. Consequently, we developed SqliGPT, an innovative LLM-based SQLI black-box detection scanner. SqliGPT enhances LLM4 Sqli-Sqlmap by integrating the Strategy Selection Module and the Defense Bypass Module to boost detection efficiency and overcome defense-related challenges. We conducted a comparative analysis of SqliGPT against six other state-of-the-art SQLI scanners using the SqliMicroBenchmark. The results indicated that SqliGPT successfully identified all 45 targets, surpassing the performance of the other scanners. Additionally, SqliGPT demonstrated superior efficiency in detection tasks. Notably, among the 45 SQL injection targets successfully identified by SqliGPT, 5 included real-world CVEs, further underscoring the practical effectiveness of SqliGPT. While it was slightly less effective than Arachni and SQIRL on 27 Class-1 targets, it achieved the best results on 18 targets across Class-2 and Class-3. Overall, this study reveals the potential of LLMs in SQLI black-box detection and provides a wide scope for future research and improvement. Future work will focus on reducing hallucinations to improve SqliGPT's stability and strengthening the Defense Bypass Module to handle more complex insufficient defenses. We will explore integrating other scanners to enhance robustness and consider using open-source LLMs to reduce costs. Additionally, we aim to improve the efficiency of LLMs in SQL injection black-box detection for better performance. Conceptualization, Z.G. and W.X. ; methodology, Z.G. ; software, Z.G., B.D., M.Z., Y.C. and S.W. ; validation, Z.G., B.D., M.Z., Y.C. and S.W. ; investigation, Z.G., B.W., W.X. and B.W. ; resources, Z.G. ; data curation, B.D., M.Z., Y.C. and S.W. ; writing--original draft preparation, Z.G. ; writing--review and editing, E.W., W.X. and B.W. ; supervision, Z.G. ; project administration, Z.G. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable. We have made SqliMicroBenchmark and SqliGPT available as open-source resources at [[https://github.com/guizhiwen/SqliGPT](https://github.com/guizhiwen/SqliGPT)]([https://github.com/guizhiwen/SqliGPT](https://github.com/guizhiwen/SqliGPT)), accessed on 11 July 2024. We sincerely thank the reviewers for their perceptive comments, which helped us to improve this work. The authors declare no conflict of interest. The following abbreviations are used in this manuscript:  
SqliStructured Query Language Injection LLMs Large Language Models AI Artificial Intelligence CoT Chain-of-Thought CVE Common Vulnerabilities and Exposures WAF Web Application Firewalls API Application Programming Interface  
This appendix describes in detail the experimental testing strategies used to evaluate the effectiveness of different LLMs in detecting SQLI vulnerability. **Repeated Testing Criteria**: Given the inherent variability of LLMs' outputs, we use a repeated testing approach to verify the effectiveness of the program. Specifically, we test each target five times with each LLM. In this process, as long as one test succeeds, LLMs are considered to have successfully solved that SQLI problem. **Failure Determination Criteria**: _LLM4 Sqli-Sqlmap and PentestGPT_: If LLMs fail to detect a usable injection point or consider the target immune to SQLI attacks in five rounds of interactions, it is judged as a test failure._LLM4 Sqli-Manual_: If LLMs fail to provide a payload that helps to advance the direction of the SQLI problem solution in three consecutive interaction cycles, it is judged as a test failure.