# Intrusion Detection Model Based on Improved Transformer
## 4 Results
### Experimental Results and Discussion
#### 4.3.4 Performance Analysis and Comparison Experiments  
To validate the detection capability of the model, this study first selected recently proposed intrusion detection models, including CNN [32], CNN-LSTM [32], CBA-CLSVE [33], SSC-OCSVM [22], CNN-GRU [34], and FCNN-SE [35], which have shown good performance on the NSL-KDD dataset. Additionally, several intrusion detection models using a Transformer as the base model were selected for comparison, namely RTIDS [26], VIT [27], and CNN-Transformer [28]. To ensure the validity of the experiments, the NSL-KDD dataset was not re-partitioned into training and testing sets under the same experimental conditions, and the official training and testing sets specified by NSL-KDD were used for binary and multiclass comparison experiments. The binary classification results, compared with other models, are presented in Table 4. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Hybrid Method**} & \multicolumn{3}{c}{**Binary Classification**} & \multicolumn{3}{c}{**Multi-Classification**} \\ \hline \multicolumn{2}{c}{**Under-Sampling**} & \multicolumn{3}{c}{**Over-Sampling**} & \multicolumn{3}{c}{**Accuracy**} & \multicolumn{1}{c}{**F1-Score**} & \multicolumn{1}{c}{**Accuracy**} & \multicolumn{1}{c}{**F1-Score**} \\ \hline
**Random** & **KNN-Based** & **Random** & **SMOTE** & **Borderline** & **SMOTE** & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** \\ \hline \(\surd\) & & & \(\surd\) & & & 0.833 & 0.828 & 0.809 & 0.805 \\ \(\surd\) & & & \(\surd\) & & 0.847 & 0.843 & 0.816 & 0.813 \\ \(\surd\) & & & & \(\surd\) & & 0.858 & 0.852 & 0.825 & 0.820 \\  & \(\surd\) & \(\surd\) & & & 0.849 & 0.844 & 0.827 & 0.824 \\  & \(\surd\) & & \(\surd\) & & 0.869 & 0.865 & 0.833 & 0.829 \\  & \(\surd\) & & & \(\surd\) & 0.887 & 0.882 & 0.841 & 0.838 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison experiments of different hybrid sampling methods. In Table 4, the results of our proposed model demonstrate its superiority over other state-of-the-art models in terms of both accuracy and _F1-score_, achieving an impressive accuracy rate of 88.7% and an _F1-score_ of 88.2%. These results indicate the effectiveness of our proposed model in accurately predicting the target variable and suggest its potential for practical applications. Furthermore, the proposed model also exhibited faster training speed than the other models, indicating its efficiency and scalability. Table 5 presents the results of the multi-classification experiment and compares them with other models. In Table 5, our experimental results indicate that our proposed model outperforms other models in the multi-classification task, achieving an accuracy of 84.1% and an _F1-score_ of 83.8%. This performance is superior to other models by 5.3%, 4.4%, 0.9%, 2.6%, 4.1%, 1.1%, 0.6%, 1.3%, and 1% in accuracy, respectively. Furthermore, our proposed model exhibits faster training speed than other models. These results suggest that our proposed model offers a significant improvement over existing models for the multi-classification task. To further demonstrate the robustness of our model, we conducted additional tests on the UNSW-NB15 dataset and compared it with the models mentioned earlier under the same experimental conditions (by splitting the UNSW-NB15 dataset into training and testing sets with a ratio of 7:3). The results of our multi-class classification experiment on the testing set are shown in Table 6.  
The experimental results show that our model has a better detection capability on the UNSW-NB15 dataset compared to other models, indicating the robustness of our proposed model. In the multi-classification task, our model achieves an accuracy of 87.5% and an _F1-score_ of 87.3%, outperforming other models by 4.6%, 4.9%, 0.8%, 2.6%, 3.2%, 1.1%, 0.2%, 1.9%, and 0.5% in accuracy, respectively. Additionally, the training speed of our model is faster.