## 8 Performance Evaluation and Discussion  
In this section, we focus on evaluating the performances of the AMiDS models when applying imbalanced learning to address the rare event case using two baseline models and a worst-case AMiDS model. Given the rare event case, the worst-case AMiDS model refers to an AMiDS model that did not perform well under regular learning. In doing so, we consider the performance of the AMiDS models when applying regular learning, as presented by the comparative study in [70], to quantify the improvement gained from applying imbalanced learning. Further, we used the two-class neural network IDS (tNN IDS) model used by the authors of [70] as a baseline to evaluate the performance of the AMiDS models under regular learning conditions.  
In this study, we used the tNN IDS model as a baseline to evaluate the performance of the AMiDS models under imbalanced learning conditions. We called this class of baseline models ADiDS, i.e., anomaly-based deep learning IDS. In addition, we designed a two-class SVM (tSVM) AMiDS model to create an AMiDS baseline model to be used in addition to the ADiDS baseline model to evaluate the performances of the models presented in this paper. We refer to these baseline models as the best-case AMiDS and ADiDS models. The classification results of the worst-case AMiDS models and the best-case baseline models are shown in Table 34, and the ROC curve of the tSVM AMiDS model is shown in Figure 14.  
Based on the analysis results across the categories of the imbalanced learning methods presented in Section 6, we considered the performance of the PCA-based AMiDS model when applying KMS, ROS, SMT, SMTEN, SMTEK, EENSMB, and BALCAS, as well as the performance of the oSVM-based AMiDS when applying the ROS, SMTEK, and BALCAS when assessing the improvement gain. We selected these AMiDS models because they performed the best out of all of the trained AMiDS. This means that, among all the trained PCA- and oSVM-based AMiDS, the listed AMiDS performed better than the rest. Accordingto Table 34, the two AMiDS models' performances as a result of regular learning were poorer compared to their performances when applying imbalanced learning. In contrast, the better-case AMiDS models exhibited better performances, with the PCA-based AMiDS model exhibiting a performance near to that of the best-case model.  
Invariably, we view the generative methods in light of two training objectives: imbalanced learning and passive adversarial learning. Consequently, we considered the ADiDS baseline model when evaluating the impact of generative methods on the performance of the low-complexity AMiDS models. In addition, we considered the better-case performance, i.e., the PCA-based AMiDS model's performance when using the AE and VAEOS methods and the oSVM-based AMiDS model's performance when using the AE method. Although the best-case models' performances were superlative, both AMiDS models exhibited comparative performances to the best-case models and superlative performances when compared to their performances using regular learning (see Tables 34 and 35). Figure 15 shows the performance of the better-case scored AMiDS model based on the weighted average.  
Figure 14: tSVM AMiDS baseline model performance.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & **A** & **P** & **R** & **F** & **AUC** & \
\begin{tabular}{c} **Training** \\ **Time (s)** \\ \end{tabular} &
\begin{tabular}{c} **Inference** \\ **Time (s)** \\ \end{tabular} \\ \hline Worst-case PCA-based AMiDS & 0.158 & 0.716 & 0.168 & 0.272 & 0.002 & 0.03 & 0.03 \\ Worst-Case oSVM-based AMiDS & 0.357 & 0.869 & 0.371 & 0.520 & 0.155 & 0.03 & 0.03 \\ tNN ADiDS Baseline & 1.000 & 1.000 & 1.000 & 1.000 & 1.000 & 0.04 & 0.03 \\ tSVM AMiDS Baseline & 1.000 & 1.000 & 1.000 & 1.000 & 1.000 & 0.07 & 0.07 \\ \hline \hline \end{tabular}
\end{table}
Table 34: Performance of the worst-case AMiDS and the ADiDS and tSVM AMiDS baseline models under regular learning conditions.