## 4 Results
### Evaluation Results  
**Overall Performance of Black-box AI Models:** The tables that follow detail the overall performance of our array of black-box AI models applied to intrusion detection systems (IDSs). Utilizing the top 15 features, as recommended by the datasets' authors, Tables 6-8 document the performance metrics for each model across the RoEduNet-SIMARGL2021, CICIDS-2017, and NSL-KDD datasets. These tables highlight the performance metrics for each AI model, demonstrating our achievement of cutting-edge results in IDSs. In analyzing the performance of black-box AI models across Tables 6-8, distinct patterns emerge regarding the best and worst models for different datasets. In Table 6, which pertains to the RoEduNet-SIMARGL2021 dataset, RF, MLP, and DNN demonstrate exemplary performance with nearly perfect scores across all metrics, including accuracy, precision, recall, F1 score, MCC, and AUC-ROC. Conversely, LightGBM exhibits the weakest performance, with significant improvements when all features are utilized compared to the case using only the top 15 features. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c} \hline \hline \multicolumn{11}{c}{**Top 15 Features**} & \multicolumn{11}{c}{**All Features**} \\ \hline
**AI Model** & **Acc** & **Prec** & **Rec** & **F1** & **Bacc** & **Mcc** & **AucRoc** & **Acc** & **Prec** & **Rec** & **F1** & **Bacc** & **Mcc** & **AucRoc** \\ \hline RF & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** \\ ADA & **0.92** & **0.89** & **0.89** & **0.89** & **0.92** & **0.84** & 0.91 & 0.78 & 0.56 & 0.56 & 0.56 & 0.71 & 0.42 & **0.93** \\ DNN & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** \\ SVM & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & 0.75 & 0.63 & 0.63 & 0.63 & 0.72 & 0.45 & 0.39 \\ KNN & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & 0.75 & 0.62 & 0.62 & 0.62 & 0.71 & 0.43 & 0.59 \\ MLP & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** \\ LightGBM & 0.69 & 0.55 & 0.55 & 0.55 & 0.66 & 0.32 & 0.51 & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** & **0.99** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Overall performance of AI models with top 15 features versus all features (no feature selection) for the RoEduNet-SIMARGL2021 dataset. The CICIDS-2017 dataset detailed in Table 7 reveals ADA and KNN as top performers with 0.99 accuracy for all features, while the worst performer is DNN in this case. When using the top 15 features, SVM and KNN perform poorly relative to the other models, particularly in their AUC-ROC scores. MLP is the top performer with the best overall results, followed by RF. Table 8's analysis of the NSL-KDD dataset highlights MLP as the best performer when all features are considered, particularly excelling in accuracy and F1 score, followed by KNN as a close second. ADA appears as the lesser model for all features, followed by RF and LightGBM in second place with similar performance. When considering the top 15 features, RF is the best performer, followed by DNN, while the weakest performer is KNN, with MLP in second, when considering the respective accuracies. These findings suggest that RF, MLP, and DNN are robust across various datasets, indicating broad applicability regardless of dataset specifics. LightGBM's performance variability suggests a potential dependence on having a richer feature set for optimal results. On the other hand, SVM, KNN, and ADA seem to benefit from feature selection. This feature sensitivity suggests that tailored feature engineering may be necessary to enhance model effectiveness.