# Reducing the False Negative Rate in Deep Learning Based Network Intrusion Detection Systems
## 6 Results
### Results of the UNSW-NB15 Experiments  
The results achieved in the three binary classification experiments with the UNSW-NB15 dataset are in Table 23. We can observe that the prediction accuracy is very similar in all three experiments. However, there is a considerable variation between the precision and recall, especially between the \"UNSW-NB15 binary--full features\" and the \"UNSW-NB15 binary--class weights\" experiments. The observed reduction is likely because the \"UNSW-NB15 binary--class weights\" experiment produced more false positives and fewer false negatives. After all, there is a trade-off between those two when using class weights. The goal was to lower the FNR as much as possible, and the model used in the \"UNSW-NB15 binary--class weights\" experiment was the most successful.  
Three multiclass classification experiment results for the UNSW-NB15 dataset are in Table 24. We can observe that the lowest FNR was achieved in the \"UNSW-NB15 multiclass--class weights\" experiment, jointly with the best performance for minority classes. The same experiment also reached the highest precision. However, it is noticeable as there was not a significant variation between the other metrics among all the performed experiments. Since the dataset has a minimal number of records representing attacks  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Experiment** & **Training Accuracy** & **Prediction Accuracy** & **Precision** & **Recall** & **F1 Score** & **ROC AUC Score** & **FPR** & **FNR** \\ \hline Full features & 98.87\% & 98.88\% & 86.03\% & 91.65\% & 88.75\% & 95.45\% & 0.76\% & 8.34\% \\ \hline Reduced features & 98.86\% & 98.86\% & 85.02\% & 92.81\% & 88.75\% & 96\% & 0.83\% & 7.18\% \\ \hline Class weights & 98.63\% & 98.64\% & 78.2\% & 99.72\% & 87.66\% & 99.15\% & 1.41\% & 0.28\% \\ \hline \hline \end{tabular}
\end{table}
Table 23: Comparison of the results achieved in the UNSW-NB15 binary classification experiments.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Experiment** & **Class** & **Precision** & **Recall** & **F1 Score** & **No. of Records** \\ \hline \multirow{2}{*}{Full features} & R2L & 60\% & 5\% & 9\% & 2885 \\ \cline{2-6}  & U2R & 80\% & 12\% & 21\% & 67 \\ \hline \multirow{2}{*}{Modified distribution} & R2L & 96\% & 98\% & 97\% & 719 \\ \cline{2-6}  & U2R & 71\% & 60\% & 65\% & 20 \\ \hline \multirow{2}{*}{Reduced features} & R2L & 90\% & 98\% & 94\% & 719 \\ \cline{2-6}  & U2R & 75\% & 45\% & 56\% & 20 \\ \hline \multirow{2}{*}{Class weights} & R2L & 84\% & 98\% & 91\% & 719 \\ \cline{2-6}  & U2R & 52\% & 70\% & 60\% & 20 \\ \hline \hline \end{tabular}
\end{table}
Table 22: Performance metrics for U2R and R2L classes in NSL-KDD multiclass experiments.  
compared to records representing regular traffic, even introducing the class weights, it is hard for the network to learn how to distinguish between the different classes since there are too few samples.  
The dataset's lowest number of records classes are Worms, Shellcode, Backdoor, and Analysis. The goal is to try to increase the prediction capability for these classes, so we report in Table 25 the class-specific precision, recall, and F1 score for these four classes. As for the other dataset, the column \"No. of records\" refers to the number of records belonging to those classes in the testing dataset.  
When compared to the first two experiments, the \"UNSW-NB15 multiclass--class weights\" experiment has seen an improvement in the performance metrics for all these classes. Since Early Stopping was used in order to prevent the model from overfitting, the average number of epochs needed to reach the optimal result while training the model on the UNSW-NB15 dataset was 20. The average time needed to train the network for this dataset was approximately 30 min for each UNSW-NB15 experiment.