## 4 Experiment
### Related Experiments  
This article uses the application of common machine learning algorithms on industrial control network traffic as comparative experiments, such as support vector machine (SVM), extra tree (ET), random forest (RF) and extreme gradient boosting (XGBoost). The evaluation indicators selected in this paper are accuracy, recall, precision, F1-score, FNR and FPR mentioned in Section 4.3. This article first compares with the traditional machine learning methods, and also compares the stacked tree of four machine learning trees (stacking). It can be seen that the model in this paper has certain optimization compared with the traditional machine learning method. The strength of this paper's model on a range of metrics is shown more specifically in Table 3. Among them, FNR and FPR refer to the probability of false negatives and false positives in the training process. The lower the index, the better it reflects the model classification. From the metrics in Table 3, it can be seen that the Acc, Pre, Rec and F1-score scores of the stacking model are 0.9507, 0.9446, 0.9472 and 0.9459, respectively. These metrics are all improved by 3.1-4.5% compared to the single machine learning model. Meanwhile, the stacking model has lower FNR and FPR scores of 0.0408 and 0.0102, respectively. The stacking model also performs a bit better in terms of false negatives and false positives. Thus, we can conclude that a single common machine learning algorithm tends to be less effective than multiple machine learning stacking integration models in handling intrusion detection tasks. It can be proved that the stacked ensemble model (ETM) is reasonable as the first layer of the model in this paper. The last column in Table 3 shows the performance results of the ETM-TBD model in this paper. From the results of the six metrics, the metrics results of the ETM-TBD model are improved by about 2.1% compared to the stacking model. The F1-score of the ETM-TBD model can reach 0.9665. Thus, it can be proved that the ETM-TBD model proposed in this paper is successful in the idea of hybrid machine learning and neural network models. As we all know, the radar chart can display the information of multiple variables at the same time, and it is convenient for comparison. Therefore, in order to show the effect of the algorithms more intuitively, we decided to use the radar chart to display the data, as shown in Figure 5.  
At the same time, this paper also conducts comparative experiments with the following neural network models or machine learning integration models. (1) ETM model: The ETM model is the first layer of the ETM-TBD model, which retains only the integrated machine learning module. We consider it as one of the comparison experiments for demonstrating the advantages of hybrid machine learning and neural network models. (2) ETM-MBD model: Compared with the model in this paper, the ETM-MBD model replaces the Transformer module with the multi-head-attention mechanism to prove whether the skip connect in the Transformer module is more effective for classification. (3) ETM-TD model: Compared with the model in this paper, the ETM-TD model lacks the BiGRU module. It can be used to explore the advantages of the BiGRU module in the case of retaining timing features and long-distance dependencies. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & **Accuracy** & **Precision** & **Recall** & **F1-score** & **FNR** & **FPR** \\ \hline XGBoost & 0.9089 & 0.9156 & 0.9054 & 0.9104 & 0.0871 & 0.0217 \\ RF & 0.9173 & 0.9207 & 0.9103 & 0.9154 & 0.1053 & 0.0263 \\ SVM & 0.9099 & 0.9104 & 0.9015 & 0.9059 & 0.0907 & 0.0206 \\ ET & 0.9124 & 0.9083 & 0.9079 & 0.9081 & 0.0713 & 0.0178 \\ Stacking & 0.9507 & 0.9446 & 0.9472 & 0.9459 & 0.0408 & 0.0102 \\ ETM-TBD & 0.9724 & 0.9658 & 0.9672 & 0.9665 & 0.0263 & 0.0081 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of the results of the model in this paper and traditional machine learning methods.