# Intrusion Detection Model Based on Improved Transformer
## 4 Results
### Experimental Results and Discussion
#### 4.3.5 Ablation Experiment  
In order to verify the effect of each module in the proposed model on the overall performance, we conducted three ablation experiments on the model:  
1. We conducted ablation experiments to assess the impact of the proposed improved position encoding on the classification ability of the Transformer model. The results are presented in Table 7.
2. We conducted ablation studies on the binary classification model proposed in this paper, including the data processing strategy and the first stage of learning, in order to verify the effectiveness of each module in the model. The results of the experimental analysis are listed in Table 8.
3. We conducted ablation experiments on the multi-classification model. There are three parts of the model that have an impact on the classification effect, namely improved positional encoding, hybrid sampling, and a two-stage learning strategy. The effect of each module was verified by comparing the accuracy and _F1-score_ before and after the addition of the module. The results of the experimental analysis are shown in Table 9. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Accuracy** & _F1-Score_ & **Training Time (s)** \\ \hline CNN & 0.829 & 0.827 & 26 \\ CNN-LSTM & 0.826 & 0.806 & 59 \\ CBA-CLSVE & 0.867 & 0.862 & 33 \\ SSC-OCSVM & 0.849 & 0.846 & 23 \\ CNN-GRU & 0.843 & 0.840 & 45 \\ FCNN-SE & 0.864 & 0.861 & 29 \\ RTIDS & 0.873 & 0.869 & 38 \\ VIT & 0.856 & 0.854 & 24 \\ CNN-Transformer & 0.870 & 0.865 & 32 \\ Ours & 0.875 & 0.873 & 19 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison experiment of multi-classification ability on UNSW-15 dataset. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Binary Classification**} & \multicolumn{2}{c}{**Multi-Classification**} \\ \cline{2-5}  & **Accuracy** & _F1-Score_ & **Accuracy** & _F1-Score_ \\ \hline Transformer before improvement & 0.793 & 0.788 & 0.787 & 0.782 \\ Transformer after improvement & 0.804 & 0.799 & 0.801 & 0.794 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Ablation experiment of the Transformer model before and after the addition of the improved position encoding. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Binary Classification**} & \multicolumn{2}{c}{**Multi-Classification**} \\ \cline{2-5}  & **Accuracy** & _F1-Score_ & **Accuracy** & _F1-Score_ \\ \hline Transformer before improvement & 0.793 & 0.788 & 0.787 & 0.782 \\ Transformer after improvement & 0.804 & 0.799 & 0.801 & 0.794 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Ablation experiment of the binary classification model. The results presented in Table 7 demonstrate a significant improvement in the accuracy and _F1-score_ of the Transformer model after the proposed positional encoding method was implemented, as compared to its performance before the improvement. This improvement was observed in both binary and multi-classification tasks, indicating the effectiveness of the proposed method across different types of classification problems. These results highlight the importance of properly encoding positional information in the Transformer model and suggest that the proposed method can be a valuable addition to the existing approaches for enhancing the performance of Transformer-based models. Table 8 presents the results of our experiments examining the impact of each module on the performance of the binary classification model. As can be seen from the results, the removal of any module leads to a significant decrease in the accuracy and _F1-score_ of the model, highlighting the critical role that each module plays in achieving high performance. Moreover, the synergy between all modules is observed when all modules are present, resulting in the highest accuracy and _F1-score_.