# SQL Injection Detection Based on Lightweight Multi-Head Self-Attention
## 4 Experimental Results
### Numerical Evaluations  
Table 2 shows the weight size of the proposed model. We broke down the total weight size of the model into the ones for embedding section and detection section, respectively. Furthermore, the weight size for detection section was divided into the ones for the CNN, self-attention, and output stages, respectively. It can be observed from Table 2 that the weight size of the proposed model came out to only 69,269. Therefore, the proposed model can be easily deployed in the devices with only limited storage size. We can also see from Table 2 that there are only 1404 weights in the embedding section in the proposed model. This is because the proposed tokenizer is SQL-specific. That is, only pre-defined keywords and symbols in SQL codes are considered as vocabularies. In fact, there are only 158 vocabularies in the tokenizer. Simple networks may then suffice for the embedding operations. Although generic tokenizers can be adopted in the proposed model, the resulting network size may be significantly increased. To show this fact, we included the weight size of the proposed model combined with generic SentencePiece tokenizer in Table 2. To facilitate the deployment of SentencePiece, the tokenizer used by ALBERT was adopted, which contains 30,000 vocabularies. The weight size of the tokenizer is 60,000. To accommodate the SentencePiece tokenizer [31], the weight size of the CNN and self attention stages were also enlarged. The total weight size came out to 749,537. The proposed model with SQL-specific tokenizer requires only 9.24% of the weight size of the proposed model with SentencePiece tokenizer (i.e., 69,269 vs. 749,537). \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** &
\begin{tabular}{c} **Embedding** \\ **Section** \\ \end{tabular} & \multicolumn{2}{c}{**Detection Section**} & \multicolumn{2}{c}{**Total**} \\  & & **CNN** & **Self Attention** & **Output** & \\ \hline Proposed Model (SQL-Specific Tokenizer+ & & & & \\ Proposed Detection Model) & 1404 & 7680 & 51,992 & 8193 & 69,269 \\ \hline SentencePiece Tokenizer [31]+ & & & & & \\ Proposed Detection Model & 600,000 & 8544 & 132,800 & 8193 & 749,537 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The weight sizes of the proposed model with different tokenizers. In addition to weight sizes, the precision rate, recall rate, F1 score, and accuracy of the proposed model were also considered, as shown in Table 3. To achieve meaningful comparisons, all the models in Table 3 were trained by the same training set \(\mathcal{T}\) and were evaluated by the same test set \(\mathcal{S}\). With a significantly lower weight size, we can see from Table 3 that the proposed model with SQL-specific tokenizer yielded accuracy, precision rate, recall rate, and F1 score values comparable to that of the proposed model with the SentencePiece tokenizer. These results justify the employment of the SQL-specific tokenizer for the malicious detection. To further demonstrate the effectiveness of the proposed model, we compared the precision rate, recall rate, accuracy, F1 score, and weight size of the proposed model with existing studies, as shown in Table 4. In the experiment, the proposed model and the model in [22] were trained by the same training set \(\mathcal{T}\). The models ALBERT [24], DistilBERT [25], and ELECTRA [27] in Table 4 are the lightweight BERT models. These models have been pre-trained and can be directly used for SQL injection detection. However, in this study, they were subsequently fine-tuned by the training set \(\mathcal{T}\) to further optimize their detection performance. The evaluations of all the models in Table 4 are based on the same test set \(\mathcal{S}\) with 24,707 queries. It can be observed from Table 4 that the proposed model has lowest weight size compared to the existing models. In particular, its weight size is only 0.1% of that of the DistilBERT [25] (i.e., 69,269 versus 66,966,010).