## 6 Evaluation
### Efficiency Evaluation (RQ2)  
Table 2 also shows the average number of HTTP requests sent by each scanner to successfully detect different classes of targets. Since there may be significant quantitative differences in the number of HTTP requests required to detect different targets, we categorized the 45 targets into three classes for independent comparison based on the scanners' target completion, ensuring fairness in the evaluation. The specific target categorization is shown in Table 2, where Class-1 totals 22 targets, including 23 basic and 4 advanced difficulty targets; Class-2 consists of 5 basic difficulty targets; and Class-3 covers 13 advanced difficulty targets. On Class-1 targets, Arachni performs best in terms of the number of HTTP requests, followed by SQIRL. SqliGPT performs slightly worse than Arachni and SQIRL on Class-1 but still outperforms the other scanners. SqliGPT performs best on both \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Scanner**} & \multicolumn{3}{c}{**Avg Requests per Successful Target**} & \multicolumn{2}{c}{**Basic**} & \multirow{2}{*}{**Advanced**} & \multirow{2}{*}{**Total (45)**} \\  & & **Class-1** & & **Class-2** & & **Class-3** \\ \hline ZAP & 388 & – & – & 20 (71\%) & 4 (24\%) & 24 (53\%) \\ BurpSuite & 283 & – & – & 21 (75\%) & 4 (24\%) & 25 (56\%) \\ Arachni & 77 & – & – & 23 (82\%) & 4 (24\%) & 27 (60\%) \\ Sqlmap & 717 & 9488 & – & 27 (96\%) & 4 (24\%) & 31 (69\%) \\ SQIRL & 83 & – & – & 20 (71\%) & 2 (12\%) & 22 (49\%) \\  & GPT-3.5 & 357 & – & – & 18 (64\%) & 4 (24\%) & 22 (49\%) \\ PentestGPT & GPT-4 & 351 & – & – & 18 (64\%) & 4 (24\%) & 22 (49\%) \\  & Claude-3 & 338 & – & – & 18 (64\%) & 4 (24\%) & 22 (49\%) \\  & GPT-3.5 & 609 & 6712 & – & 26 (93\%) & 4 (24\%) & 30 (67\%) \\ LLMdSqlmap & GPT-4 & 517 & 8451 & – & 27 (96\%) & 4 (24\%) & 31 (69\%) \\  & Claude-3 & 504 & 11,436 & – & 26 (93\%) & 4 (24\%) & 30 (67\%) \\  & GPT-3.5 & 127 & 761 & 1684 & **28 (100\%)** & **17 (100\%)** & **45 (100\%)** \\ SqliGPT & GPT-4 & 119 & 822 & 1681 & **28 (100\%)** & **17 (100\%)** & **45 (100\%)** \\  & Claude-3 & 113 & **749** & **1669** & **28 (100\%)** & **17 (100\%)** & **45 (100\%)** \\ \hline \hline \end{tabular}  
* The best performance for each metric is highlighted in bold font.  
\end{table} 
Table 2: Performance comparison of each scanner on SqliMicroBenchmark. Since different numbers of HTTP requests need to be sent to detect different targets, to ensure fairness of the evaluation, we categorized the 45 targets into three classes based on the target completion of the scanners and compared the average number of HTTP requests that each scanner needs to send to detect different classes of targets respectively. Class-2 and Class-3 targets. **Overall, SqliGPT performs well in terms of efficiency in performing detection tasks. **  
SqliGPT failed to outperform Arachni in the detection performance of Class-1 targets, a situation that can be attributed to the differences in detection strategy and evaluation criteria between SqliGPT and Arachni. SqliGPT guided Sqlmap to perform exhaustive SQLI detection of the targets, ensuring comprehensive coverage of potential injection points. In contrast, Arachni employs a narrower detection strategy, which allows it to complete tests more efficiently in some cases. However, the limitation of this strategy is that it may need to adequately cover all potential injection points, resulting in a less comprehensive test. Additionally, Arachni may use looser criteria when determining injection points, which, while helpful in quickly identifying potential vulnerabilities, may also increase false positives. Compared to SqliGPT, the advantages demonstrated by SQIRL on Class-1 targets stem from its use of a reinforcement learning-based approach to payload mutation, which enables more rapid and efficient SQLI black-box detection.