# SQL Injection Detection Based on Lightweight Multi-Head Self-Attention
## 3 The Proposed Approach
### Input Embedding  
To perform the input embedding operations, a SQL-specific tokenizer is adopted. The tokenizer converts words and symbols in the SQL codes into numeric tokens. The user-defined identifiers in the codes are not considered for the conversion. It has been found [12; 29] that the removal of user-defined identifiers or attributes is beneficial for the avoidance of noisy information in the SQL queries. The removal operations can be easily implemented by first defining a set of vocabularies \(\mathcal{V}\), which consists of words and symbols from SQL languages. In the tokenizer, words or symbols from the input queries not belonging to \(\mathcal{V}\) are removed. Otherwise, they are converted into numerical tokens. To facilitate the training operations for neural network models, a semantic label could also be assigned to each numerical token [12] in the queries. In this study, there are three different classes of semantic labels: command, symbol, and expression. An example of the results produced by the SQL-specific tokenizer is shown in Figure 4, where the SQL query after removing user-defined identifiers or attributes is termed the skeleton query. Each element in the skeleton query is a token. As shown in the figure, each token is associated with a token index and a semantic label. Therefore, a token list and a semantic list can be formed after the skeleton query is acquired from the input query. Let \(N\) be the number of tokens accepted by the proposed algorithm from an input SQL query. It is fixed so that zero padding is necessary when the actual number of tokens is less than \(N\). Based on the example of the lists produced by the SQL-specific tokenizer in Figure 4a, the results of the proposed embedding scheme are revealed in Figure 4b. In the example, it can be observed from Figure 4b that each token and its associated semantic label from the lists are mapped to two numerical vectors by functions \(f_{1}\) and \(f_{2}\), respectively. Both \(f_{1}\) and \(f_{2}\) are fully connected neural networks with weights determined by training. Let \(M_{1}\) and \(M_{2}\) be the dimensions of vectors produced by \(f_{1}\) and \(f_{2}\), respectively. The numerical vectors for all the tokens and semantic labels are then concatenated to form the embedding matrix for tokens and semantic labels. The resulting matrix, denoted by \(\mathbf{F}\), is termed combined representation matrix in this study. As shown in Figure 4, it has dimension \(N\times M\), where \(M=M_{1}+M_{2}\). Figure 3: The overview of the proposed model. It contains two sections: input embedded section and detection section. For an input SQL query, the input embedding section provides a feature matrix, termed encoded representation matrix. An estimated probability for malicious attack is then produced by the detection section. A malicious query is detected when the estimated probability is above a pre-specified threshold. To facilitate the subsequent self-attention operations, the information about position of the tokens in the input SQL query is also provided. In this study, the position information is included in a position encoding matrix **C** with dimension \(N\times M\). The \((i,j)\)-th element, \(i=1,...,N,j=1,...,M\), of the matrix **C**, denoted by \(C(i,j)\), contains the position information for the \((i,j)\)-th element of the matrix **F**. A sinusoidal encoding approach [17] is adopted for computing \(C(i,j)\), as shown below. \[C(i,j)=\left\{ \begin{array}{ll}\sin(i/10000^{j/N})&\text{when $j$ is even,}\\ \cos(i/10000^{(j-1)/N})&\text{otherwise.}\end{array} \right. \tag{1}\]  
Figure 4: An example of the results produced by the proposed embedding scheme. (**a**) Based on an input query, a skeleton query is first formed by removing user-defined identifiers and attributes. Lists of \(N\) tokens and their corresponding semantic labels are then created from the skeleton query.