# Applications of Machine Learning in Cyber Security: A Review
## 5 Intrusion Detection System Evaluation
Future research should primarily aim to address these gaps. Robustness relates to diverse and evolving security threats. An IDS should be able to recognize many known threats reliably and be resilient to new threats [52]. Finally, efficiency was incorporated as a criterion due to the real-time demands of many IDS scenarios. Rapid threat detection and response can be pivotal, and the literature review in IDS with ML highlights its need and the current lack of measurement [66]. The analysis of recent literature against our specified criteria is summarized in Table 6. As demonstrated, no published approach addresses all of the criteria, with a significant gap in bias, robustness, and a noticeable lack of efficiency measurements, which hinders the application of these techniques in real-world scenarios. It should be noted that these papers did not set out to achieve those goals. Primarily, they focus on improved performance as the majority of advanced AI or ML publications. The authors in [52] studied how IDS with ML responds to unknown attacks. While IDS with ML is highly accurate for known attacks, its performance significantly decreases when faced with new attack types. The study looks into strategies to minimize this drop using various models and data splits in training. Their unique analysis indicates that some models are better at predicting unknown attacks, and the effectiveness varies based on the specific unknown attack type. However, according to the authors, understanding why some models may perform better on certain splits or unknown attacks is limited due to limitations in model explainability. The research does not include statistical tests for dataset validity, specifically in terms of feature importance, imbalanced classes, and multicollinearity. This omission can add noise to the study's findings. Moreover, using a dataset not frequently cited in the literature amplifies this concern. Its approach to dealing with unknown attacks is novel and can contribute to robustness. Their evaluation with regards to false positives is to an extent moving towards bias reduction. The study, however, does not report processing time results, so it did not meet the efficiency criterion. The research in [68] focuses on effective feature extraction. The proposed approach does not consider explainability but uses easier-to-explain models such as Jrip or KNN, trying to address bias by using SMOTE to deal with class imbalance, but not in terms of multicollinearity. It does not attempt to evaluate robustness. While it uses image representation of network traffic, a laborious task presented as efficient, the paper does not report on execution times. Thus, efficiency conclusions cannot be drawn. In [55], the authors look at making an explainable AI on deep learning models using a range of local and global explanations, with LIME, SHAP, and rulefit. However, its explainability is limited by treating all attacks as the same, and deep learning may still be too much of a \"black box\" compared to other approaches. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Reference** & **Explainability** & **Bias** & **Robustness** & **Efficiency** \\ \hline
[52] & & & ✓ & \\
[67] & ✓ & & & \\
[68] & ✓ & & & \\
[69] & & & & \\
[55] & ✓ & & & \\
[70] & & & & ✓ \\
[71] & & ✓ & ✓ & ✓ \\
[56] & & & & ✓ & ✓ \\
[72] & & & & \\
[73] & ✓ & & & \\
[74] & ✓ & ✓ & & \\
[75] & ✓ & ✓ & & \\
[58] & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Recent research publications evaluated against our criteria where a checkmark denotes meeting the criterion. The authors in [71] created a novel IDS framework which resulted in a high accuracy. The proposed approach focuses on effective feature selection. They used boruta feature selection, which permutates many copies of the data to improve the robustness of the feature selected. However, the approach would not meet our definition of robustness against unknown attacks.