## 4 Experiment Results and Discussion
### Experiment Results
#### 4.2.2 Experiment 2  
This experiment presents the results of variant RNN models are learned on the selected feature subset of experiment 1. These models are Simple RNN, LSTM, and GRU. The output class of ISCX dataset is ['Normal', 'Attack']. Hence, it is considered in binary classification in three learning models. ['Normal'] value is denoted by 0 and ['Attack'] value is denoted by 1. In NSL-KDD dataset, the output class includes non-attack and four types of attack, denoted by ['Normal', 'R2L', 'DoS', 'U2R', 'Probe']. In experiment, visualization of ROC results of ['Normal', 'R2L', 'DoS', 'U2R', 'Probe'] are denoted by ['class 0', 'class 1', 'class 2', 'class 3', 'class 4'], respectively. The number of output class is more than two, thus it is a multi-classification problem for three models. Therefore, confusion matrix and ROC measurement are used to visualize the attack type and non-attack detection results. In the NSD-KDD dataset, the attack classification results are illustrated in Figure 12. The confusion matrix results display the number of correct and incorrect prediction compare to actual output class \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **No.FF** & **No.SF** & **Selected Feature Subset** & **Accuracy** & **Error** \\ \hline \multirow{4}{*}{NSL-KDD} & \multirow{4}{*}{41} & \multirow{4}{*}{12} & protocol\_type, service, flag, src\_bytes, & & \\  & & logged\_in,num\_file\_creations, & & \\  & & & is\_guest\_login, count, serv\_count, & 0.969017 & 0.00336 \\  & & & dst\_host\_srv\_diff\_host\_rate, & & \\  & & & dst\_host\_rerrer\_rate, & & \\  & & & dst\_host\_srv\_error\_rate & & \\ \hline Saturday data & 15 & 3 & appName, totalDestinationBytes, source & 0.999991 & 0.00001 \\ \hline Sunday data & 15 & 7 & appName, direction, protocolName, sourcePort, & 0.999920 & 0.00002 \\  & & & destination, destinationPort, startDateTime & & \\ \hline Monday data & 15 & 7 & appName, totalDestinationBytes, direction, source, & 0.999920 & 0.00002 \\  & & & destination, destinationPort, startDateTime & & \\ \hline Tuesday data & 15 & 6 & totalSourceBytes, source, destination, & 0.999719 & 0.00006 \\  & & & destinationPort, startDateTime, stopDateTime & & \\ \hline Wednesday data & 15 & 3 & appName, totalSourceBytes, totalDestinationBytes & 1 & 0 \\ \hline Thursday data & 15 & 3 & appName, totalSourceBytes, source & 1 & 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The selected feature subset by SFSDT algorithm on two IDS datasets. Figure 11: Visualization accuracy and error scores of SFSDT model on each day of ISCX dataset. (**a**) & (**b**) Tuesday data; (**c**) & (**d**) Wednesday data; (**e**) & (**f**) Thursday data. for each output class. The number of corrected predictions are displayed on the main diagonal of the confusion matrix. Besides, the ROC curve is used to measure the performance of different attacks detection on NSL-KDD dataset (see Figure 13). Most attack type detection achieved better results on LSTM and GRU models. Figure 12: Confusion matrix of three IDS models on NSL-KDD dataset. (a) Simple RNN model (b) LSTM model (c) GRU model. Similar to the ISCX dataset, the confusion matrix results of three models are displayed in Figure 14 (on Saturday, Sunday, and Monday data) and Figure 15 (on Tuesday, Wednesday, and Thursday data). Figure 14: _Cont._  
Figure 13: ROC for multi-class of three IDS models on NSL-KDD dataset. (**a**) RNN model (**b**) LSTM model (**c**) GRU model. Figure 14: Confusion matrix results of three approach models RNN, LSTM, and GRU on ISCX selected feature subset. (**a-c**) on Saturday, (**d–f**) on Sunday, (**g-i**) on Monday. Figure 15: _Cont._  
Based on the confusion matrix results, the accuracies of attack types detection in each dataset are calculated and summarized in Table 3. The average accuracies of RNN, LSTM, and GRU model on NSL-KDD subset feature data are 89.6%, 92%, and 91.8%. Similar to on ISCX subset feature data, 94.75%, 97.5%, and 97.08% are average obtained accuracies of RNN, LSTM, and GRU model, respectively.