# Applications of Machine Learning in Cyber Security: A Review
## 5 Intrusion Detection System Evaluation  
Based on the earlier analysis, ML or AI are mostly used in the process of identifying attacks as they happen in an attempt to prevent them from completing their task or spreading further. Such systems are usually referred to as IDS. Recent systematic reviews on ML in IDS are analyzed in this section [59, 60, 61, 62], followed by research papers published at a later date. The literature reviews largely agree that the most effective way of improving the performance of ML IDS systems is through better feature selection rather than new ML models. This is in line with findings in several other domains where AI has been applied. Also, it is the root cause of non-acceptance in industrial applications. The primary goal of any new IDS approach should thus be to minimize false positives, which are bottlenecks to the effective use of IDS in networks [60]. In pursuit of this goal, we evaluate the recent research landscape based on the following criteria, derived from recent advancements in ethical AI approaches, which promise to enable such models to become more usable and better regulated:  
1. Explainability: addressing the need for IDS system decisions to be easily explainable to the user,
2. Bias: addressing imbalance and multicollinearity, treating outliers effectively, and efforts to mitigate dataset imbalance, ultimately affecting the ability of the model to eliminate false predictions,
3. Robustness: evaluating the model against attacks and normal traffic and analyzing the repeatability of the outcomes,
4. Efficiency: the model's inference execution time and whether it is reported. Explainability in ML and IDS is crucial for transparency and trust [55]. Research shows that understanding these models leads to better performance through improved data engineering [59] rather than merely adopting new ML models. The bias criterion addresses the validity of the IDS system or research results. Factors such as multicollinearity [63], outlier impact [64], and dataset class imbalances [65] can impact the results. Many of these issues are not widely discussed in the IDS literature, \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Dataset** & **Attack Diversity** & **Realism** & **Balance** & **Quality** & **Size and Complexity** \\ \hline UNSW-NB15 & \(\star\star\star\star\) & \(\star\star\star\star\) & \(\star\star\star\star\) & Good, some imbalanced classes & Large and complex \\ NSL-KDD & \(\star\star\star\star\) & \(\star\star\star\star\) & \(\star\star\star\star\) & Improved balance and reduced redundancy & Manageable \\ CICIDS2017/2018 & \(\star\star\star\star\star\) & \(\star\star\star\star\star\) & \(\star\star\star\star\) & Generally good, some imbalance & Very large \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Summary of assessment of most prominent datasets covering a wide range of attacks. Attack Diversity, Realism and Balance are rated with a star system where each full star represents a step from Very Low to Very High. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Dataset** & **Audit Logs \& Raw Data** & **Modern Attacks** & **Real or Simulated** & **Labelled** & **AI \& GDPR Compliant** & **Scientifically Accepted** \\ \hline UNSW-NB15 & Yes (raw packet data, audit logs unclear) & Partially (2015) & Simulated (generated in a controlled environment) & Yes & Presumed yes (anonymized) & Yes (widely used) \\ \hline NSL KDD & Yes (raw packet data, audit logs unclear) & No (1999) & Simulated (irpecting attacks into normal flow) & Yes & Presumed yes (KDDV9, privacy concerns addressed) & Yes (it is still a reference dataset in the community) \\ \hline CIC datasets & Yes & Yes (2017/19) & Simulated (attacks to emulate real-world situations) & Yes & Yes (anonymized) & Yes (Recently gaining acceptance, but limited applicability) \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Most prominent datasets with a wide range of attacks against assessment criteria. raising concerns about the result's statistical validity.