# SQL Injection Detection Based on Lightweight Multi-Head Self-Attention
## 3 The Proposed Approach
### Detection  
The detection operations can be separated into three stages: the CNN, self-attention, and output stages. As shown in Figure 5, the goal of the CNN stage is to extract local features from the input matrix \(\mathbf{U}\). There are two CNN layers with maximum pooling operations in the stage. Let \(\mathbf{X}\) be the output of the CNN stage. Based on \(\mathbf{X}\), the global features are highlighted in the self-attention stage. In this study, the scaled dot-product attention algorithm [17] has been adopted. Its simplest form can be characterized by three weight matrices \(\mathbf{W}_{Q}\), \(\mathbf{W}_{K}\), and \(\mathbf{W}_{V}\) for the computation of query \(\mathbf{Q}\), key \(\mathbf{K}\), and value \(\mathbf{V}\), which are given by  
\[\mathbf{Q}=\mathbf{X}\mathbf{W}_{Q},\ \ \mathbf{K}=\mathbf{X}\mathbf{W}_{K},\ \ \mathbf{V}= \mathbf{X}\mathbf{W}_{V}. \tag{3}\]  
The corresponding self-attention operations, denoted by \(S\), for a set of fixed-weight matrices \(\mathbf{W}_{Q}\), \(\mathbf{W}_{K}\), and \(\mathbf{W}_{V}\) are then given by  
\[S(\mathbf{X})=\text{softmax}(\frac{\mathbf{Q}\mathbf{K}^{T}}{\sqrt{d}}) \mathbf{V}, \tag{4}\]  
where \(d\) is a scaling factor. This attention scheme is termed single-head attention.  
To further learn more diversity for detection, we applied a multi-head self-attention in the proposed model. Let \(h\) be the number of heads in the model. Furthermore, let \(\mathbf{Q}_{i}\), \(\mathbf{K}_{i}\), and \(\mathbf{V}_{i}\) be the query, key, and value associated with the \(i\)-th head, \(i=1,...,h\). They are  
Figure 5: Proposed model for detection operations, which consists of three stages: CNN, self-attention, and output stages. The detection section takes the encoded representation matrix \(\mathbf{U}\) as the input. It then produces \(z\), the estimated probability of malicious attack by the input query. When \(z\geq\eta\), the query is malicious.  
associated with weight matrices \(\mathbf{W}_{Q_{i}}\), \(\mathbf{W}_{K_{i}}\), and \(\mathbf{W}_{V_{i}}\). Analogous to the computation of \(\mathbf{Q}\), \(\mathbf{K}\), and \(\mathbf{V}\) by (3), \(\mathbf{Q}_{i}\), \(\mathbf{K}_{i}\), and \(\mathbf{V}_{i}\) are acquired by  
\[\mathbf{Q}_{i}=\mathbf{X}\mathbf{W}_{Q_{i}},\ \ \mathbf{K}_{i}=\mathbf{X} \mathbf{W}_{K_{i}},\ \ \mathbf{V}_{i}=\mathbf{X}\mathbf{W}_{V_{i}}. \tag{5}\]  
Let \(S_{i}(\mathbf{X})\) be the self-attention operation \(S(\mathbf{X})\) in (3) for \(\mathbf{Q}_{i}\), \(\mathbf{K}_{i}\) and \(\mathbf{V}_{i}\). The multi-head self-attention, denoted by \(H(\mathbf{X})\), is computed by  
\[H(\mathbf{X})=[S_{1}(\mathbf{X})\otimes\cdots\otimes S_{h}(\mathbf{X})] \mathbf{W}_{M}, \tag{6}\]  
where \(\otimes\) is a concatenation operation, and \(\mathbf{W}_{M}\) is the weight matrix for combining the results of different single-head self-attentions. Note that each head is a unique linear transformation of \(\mathbf{X}\) as a query, a key, and a value. Therefore, the concatenation of individual heads allows for a combination of different linear transformation results. This could provide a diverse representation of global information for \(\mathbf{X}\).  
The third stage at the detection section is the output stage. It produces the final detection result \(z\) from the self-attention outcome, where \(0\leq z\leq 1\). The output stage is a fully connected layer with sigmoid activation. The detection result \(z\) indicates the possibility that the input query is malicious. The result \(z\) is then compared with a pre-specified threshold value \(\eta\). A malicious query is detected when \(z>\eta\).