## 4 Results
### Evaluation Results
Holistically, comparing Tables 6-8 indicates that using feature selection is beneficial, since it is often close to or surpasses the \"All Features\" setup regarding the results. **Impact of Feature Selection on Black-box AI Models:** In this section, we analyze the overall performance of our various black-box AI models for intrusion detection systems (IDSs) with no feature selection. For the RoEduNet-SIMARGL2021 dataset, Table 6 presents the performance metrics of our different AI models. Notably, three models (ADA, SVM, and KNN) exhibit worse performance when trained with all features, whereas only LightGBM demonstrates improved performance without feature selection. Conversely, the other models (DNN, RF, and MLP) show no significant difference in their performance. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline \hline  & \multicolumn{10}{c}{**Top 15 Features**} & \multicolumn{10}{c}{**All Features**} \\ \hline
**AI Model** & **Acc** & **Prec** & **Rec** & **F1** & **Bacc** & **Mcc** & **AucRoc** & **Acc** & **Prec** & **Rec** & **F1** & **Bacc** & **Mcc** & **AucRoc** \\ \hline RF & 0.98 & 0.93 & 0.93 & 0.93 & 0.96 & 0.92 & **0.99** & **0.99** & **0.96** & **0.98** & **0.98** & **0.94** & **0.99** \\ ADA & **0.99** & **0.99** & **0.99** & **0.99** & **0.98** & 0.95 & 0.97 & 0.89 & 0.89 & 0.89 & 0.94 & 0.88 & **0.96** \\ DNN & 0.95 & 0.83 & 0.83 & 0.83 & 0.90 & 0.80 & 0.96 & **0.98** & **0.94** & **0.94** & **0.97** & **0.94** & **0.99** \\ SVM & **0.97** & **0.91** & **0.91** & **0.91** & **0.94** & **0.89** & **0.75** & 0.96 & 0.87 & 0.87 & 0.87 & 0.92 & 0.85 & 0.93 \\ KNN & **0.99** & **0.97** & **0.97** & **0.97** & **0.98** & **0.96** & **0.89** & **0.99** & **0.97** & **0.97** & **0.97** & **0.98** & **0.96** & **0.89** \\ MLP & 0.97 & 0.91 & 0.91 & 0.91 & 0.94 & 0.89 & **0.98** & **0.99** & **0.98** & **0.98** & **0.98** & **0.98** & **0.99** & **0.98** & **0.98** \\ LightGBM & **0.98** & 0.94 & 0.94 & 0.94 & 0.96 & 0.93 & **0.99** & **0.98** & **0.95** & **0.95** & **0.95** & **0.97** & **0.95** & **0.99** \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Overall performance of AI models with top 15 features versus all features (no feature selection) for the CICIDS-2017 dataset. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline \hline  & \multicolumn{10}{c}{**Top 15 Features**} & \multicolumn{10}{c}{**All Features**} \\ \hline
**AI Model** & **Acc** & **Prec** & **Rec** & **F1** & **Bacc** & **Mcc** & **AucRoc** & **Acc** & **Prec** & **Rec** & **F1** & **Bacc** & **Mcc** & **AucRoc** \\ \hline RF & **0.88** & **0.54** & **0.50** & **0.43** & **0.75** & **0.61** & 0.60 & 0.84 & 0.78 & 0.42 & 0.36 & 0.68 & 0.32 & **0.62** \\ ADA & **0.85** & 0.46 & 0.40 & **0.31** & **0.64** & 0.25 & 0.57 & 0.81 & **0.47** & **0.47** &  
For the CICIDS-2017 dataset, as shown in Table 7, four AI models display enhanced performance when trained with all features (RF, DNN, MLP, and KNN), while the remaining models exhibit decreased performance. This contrasts with the observations from the RoEduNet-SIMARGL2021 dataset. The performance metrics of the AI models on the NSL-KDD dataset, depicted in Table 8, reveal notable differences compared to the results obtained on the RoEduNet-SIMARGL2021 and CICIDS-2017 datasets. In particular, for the NSL-KDD dataset, using the top 15 features instead of the full set generally results in improved performance across most models, indicating a higher sensitivity to feature selection. This trend is especially prominent in the ADA, SVM, KNN, and MLP models. Conversely, the RF and LightGBM models exhibit slightly different trends across the three datasets, with RF performing better with the full feature set and LightGBM maintaining comparable performance regardless of the feature set size. This behavior underscores the significance of feature selection and the varied responses different AI models might exhibit concerning feature complexity.