## 5 Evaluation and Results
### Results  
Table 3 summarizes the results of SQL injection (SQLi) generation using the GPT-4o and Gemini LLM models. Both models were provided with a prompt (detailed in Appendix A) along with 11 manually crafted obfuscated SQLi samples as in-context examples. The generated samples were validated for correctness using a MySQL database. \begin{table}
\begin{tabular}{c c c c} \hline
**LLM Model** & **Num SQLi Attacks** & **Num Valid** & **Num Invalid** \\ \hline GPT-4o & 514 & 475 & 39 \\ \hline Gemini-Pro & 393 & 209 & 184 \\ \hline \end{tabular}  
* GPT-4o: Generated 514 samples, of which 475 were valid SQLi payloads, achieving a validity rate of 92.5%.
* Gemini: Generated 393 samples, of which 209 were valid, resulting in a validity rate of 53.1%.  
\end{table} 
Table 3: SQLi attack generation results for GPT-4o and Gemini-Pro. The attacks are validated with MySQL to ensure that they are functional SQL queries. These results indicate that GPT-4o is more consistent in generating valid SQLi attacks. Notably, Gemini Pro exhibits a higher tendency to generate hallucinated outputs compared to GPT-4o. Tables 4 and 5 provide a breakdown of the validated SQLi samples generated by GPT-4o and Gemini-Pro, respectively. The SQLi attacks are listed by type of attack. yielded a minimal yet meaningful set of clusters. We used silhouette scores to evaluate cluster cohesion and separation, which range from \(-1\) to \(+1\), with \(+1\) indicating strong clustering and \(-1\) indicating poor clustering. The silhouette scores for TF-IDF + HAC and SequenceMatcher + DBSCAN were 0.37 and 0.48, respectively, reflecting moderate clustering performance. The clusters generated were input to the LLM for rule generation. We note that SeqMatcher+DBSCAN clustering-driven rule generation by GPT-4o shows the best performance. A total of 23 SecRules, generated using the SeqMatcher algorithm combined with the DBSCAN classifier, successfully blocked 99% of the 475 SQLi attacks that had previously bypassed ModSecurity. Samples of these rules are provided in Appendix B. In Table 8, we present False Positive, Precision, Recall, and F1 Score metrics to provide a comprehensive evaluation of the performance of the security rules, balancing its accuracy in detecting true positives while minimizing errors and assessing its overall effectiveness. Real-world web traffic often exhibits a higher proportion of normal samples compared to attacks. A commonly used ratio is 4:1 (80% normal, 20% attack). For this study, we created 1596 normal samples spanning four operations (Create, Read, Update, Delete) and 475 attack samples. The definitions of the metrics and their calculated values are provided in Table 8. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Attack Types** & **Num SQLi Attacks** & **Num Blocked** & **Num Bypass WAF** \\ \hline Conditional & 128 & 56 & 72 \\ \hline Error-Based & 35 & 35 & 0 \\ \hline Tautology & 220 & 99 & 121 \\ \hline Tautology and Conditional & 47 & 47 & 0 \\ \hline Exist-Based & 45 & 45 & 0 \\ \hline Total & 475 & 282 & 193 \\ \hline Percentage & 100\% & 59.3\% & 40.7\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Validated SQLi attacks by type generated by GPT-4o that bypassed AWS WAF. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Attack Types** & **Num Samples** & **Clustering Algorithms** & **Num Rules** & **Num Blocked** \\ \hline Conditional & 128 & TF-IDF + HAC & 6 & 127 \\ \cline{2-4}  & SeqMatcher+DBSCAN & 6 & 125 \\ \hline Tautology & 220 & TF-IDF + HAC & 8 & 189 \\ \cline{2-4}  & SeqMatcher+DBSCAN & 3 & 217 \\ \hline Tautology \& Conditional & 47 & TF-IDF + HAC & 5 & 47 \\ \cline{2-4}  & SeqMatcher+DBSCAN & 5 & 47 \\ \hline Error-Based & 35 & TF-IDF + HAC & 4 & 35 \\ \cline{2-4}  & SeqMatcher+DBSCAN & 5 & 35 \\ \hline Exist-Based & 45 & TF-IDF + HAC & 5 & 45 \\ \cline{2-4}  & SeqMatcher+DBSCAN & 4 & 45 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: ModSecurity rules generated by GPT-4o based on different algorithms used to cluster attacks that initially bypassed ModSecurity.