## 4 Experimental Simulations
### Experimental Results and Analysis
#### 4.3.8 Statistical Test  
In order to accurately evaluate the proposed intrusion detection method, significance tests were conducted on three datasets, UNSW_NB15, NSL-KDD, and CIC-IDS2017, with reference to the methods used in the literature [62; 63]. We used a two-tailed \(t\)-test to analyze the significance of the indicators obtained by the proposed method and thus verify that the obtained results were not obtained by chance. The calculation formula is as follows:  
\[\mu=\frac{1}{n}{\sum\
olimits_{i=1}^{n}}{\hat{y}_{i}} \tag{11}\]  
\[\sigma^{2}=\frac{1}{n-1}{\sum\
olimits_{i=1}^{n}}{(\hat{y}_{i}-\mu)}^{2} \tag{12}\]  
\[t=\sqrt{n}\frac{|\mu-y_{0}|}{\sigma} \tag{13}\]  
We use the UNSW_NB15 dataset as an example for detailed discussion. First, the average accuracy value of the proposed method is calculated as 0.8625 according to Equation (10) (where \(\hat{y}_{i}\) represents the i-th test accuracy value and \(n=20\)). After that, the standard deviation of the accuracy value is calculated as 0.01040 by Equation (11). Finally, the critical value is X calculated by Equation (12) (where \(y_{0}\) is the assumed minimum accuracy value and \(y_{0}=0.8438\)). The critical value 5.268 is greater than the 2.845  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**Feature Selection Method**} & \multicolumn{4}{c}{**Evaluation Indicators (\%)**} \\ \cline{3-5}  & & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline \multirow{9}{*}{NSL-KDD} & Random Forest & 75.41 & 84.00 & 75.41 & 77.53 \\  & K-means & 79.34 & 78.01 & 79.34 & 76.28 \\  & Decision Tree & 76.92 & 71.98 & 54.52 & 55.97 \\  & CNN-LSTM [21] & 98.64 & 98.61 & 98.64 & 98.56 \\  & S-ResNet [56] & 98.33 & 98.39 & 98.33 & 98.34 \\  & CNN [57] & 97.78 & 97.74 & 97.78 & 97.75 \\  & CNN-GRU [58] & 99.15 & 99.15 & 99.15 & 99.15 \\  & CNN-BiLSTM [59] & 99.22 & 99.18 & 99.14 & 99.15 \\  & CNN-GRU [60] & 98.97 & 99.11 & 98.97 & 99.04 \\  & CNN-GRU-attention [61] & 99.26 & 99.23 & 99.26 & 99.24 \\  & Proposed Model & **99.69** & **99.65** & **99.69** & **99.70** \\  & Random Forest & 75.41 & 84.00 & 75.41 & 77.53 \\  & K-means & 70.93 & 82.42 & 70.91 & 76.23 \\  & Decision Tree & 73.37 & 80.94 & 73.36 & 76.30 \\  & CNN-LSTM [21] & 82.6 & 81.9 & 82.6 & 80.6 \\  & S-ResNet [56] & 83.8 & 85.0 & 83.8 & 84.4 \\  & CNN [57] & 82.9 & 82.6 & 82.9 & 82.7 \\  & CNN-GRU [58] & 84.3 & 83.7 & 84.3 & 84.0 \\  & CNN-BRU-attention [61] & 82.08 & 82.68 & 80.00 & 81.32 \\  & CNN-GRU [60] & 84.25 & 84.31 & 84.25 & 84.28 \\  & CNN-GRU-attention [61] & 84.36 & 83.78 & 84.36 & 84.07 \\  & Proposed Model & **86.25** & **86.92** & **86.25** & **86.59** \\  & Random Forest & 98.21 & 98.58 & 93.40 & 95.92 \\  & K-means & 95.03 & 96.40 & 95.21 & 95.80 \\  & Decision Tree & 96.60 & 97.62 & 96.66 & 97.14 \\  & CNN-LSTM [21] & 96.64 & 96.87 & 96.64 & 96.45 \\  & S-ResNet [56] & 95.94 & 96.10 & 95.94 & 95.41 \\ CIC-IDS2017 & CNN [57] & 89.14 & 84.18 & 89.14 & 85.56 \\  & CNN-GRU [58] & 99.42 & 99.34 & 99.42 & 99.38 \\  & CNN-BiLSTM [59] & 99.43 & 99.39 & 99.42 & 99.40 \\  & CNN-GRU [60] & 99.38 & 99.41 & 99.38 & 99.39 \\  & CNN-GRU-attention [61] & 99.45 & 99.39 & 99.45 & 99.42 \\  & Proposed Model & **99.65** & **99.63** & **99.65** & **99.64** \\ \hline \hline \end{tabular}
\end{table}
Table 13: Comparison of different models.  
obtained in the two-tailed \(t\)-test table. This result indicates that the tested accuracy value of our proposed model is greater than the assumed minimum value of 0.8438, which has a confidence degree of (\(1-\alpha=0.99\)). The experimental results for the three data sets are shown in Table 14.  
From Table 14, the minimum values of our assumed accuracies are all higher than the maximum values of the different models mentioned in the paper. Therefore, the model proposed in the paper has a statistically significant difference compared to the single use of the CNN model, GRU model and other existing models, and has a significant advantage over other methods.