# Advancements in Machine Learning-Based Intrusion Detection in IoT: Research Trends and Challenges
## 6 ML Performance Review in IDS
### ML Performance Comparison

Table 6: Multiclass classification performance (sorted by accuracy). \begin{table}
\begin{tabular}{l c c c c l} \hline \hline
**Paper** & **Accuracy [\%]** & **Precision [\%]** & **Recall [\%]** & **F1 Score [\%]** & **Highest Accuracy ML Technique** & **Dataset(s)** \\ \hline
[61] & 100 & 100 & 100 & 100 & DT & TON\_IoT \\
[55] & 100 & 100 & 100 & 100 & RF & TON\_IoT \\
[43] & 100 & 100 & 100 & 100 & RF & Edge-IIoT \\
[59] & 100 & 100 & 100 & 100 & DT & CICIoT2023 \\
[59] & 100 & 100 & 100 & 100 & RF & CICIoT2023 \\
[45] & 99.9987 & 100 & - & 99.993 & XGBoost & BoT-IoT \\
[74] & 99.99 & 99.99 & 99.99 & 99.99 & KNN, LR, DNN & CIC2023 IoT \\
[78] & 99.99 & - & - & - & JRip classifier & BoT-IoT \\
[80] & 99.98 & 99 & 89 & - & DT & CSE-CIC-IDS2018 \\
[33] & 99.96 & 99.99 & 100 & 100 & XGBoost & CIC-IDS2017 \\
[82] & 99.96 & 99.96 & 99.96 & 99.96 & RF & BoT-IoT \\
[47] & 99.95 & 99.66 & 99.61 & 99.63 & RF & Application-Layer DDoS Dataset \\
[44] & 99.94 & 99.87 & 99.83 & 99.85 & RF & SNMP-MB Dataset \\
[76] & 99.92 & - & - & - & Stacking using XGBoost & BoT-IoT \\
[67] & 99.9 & 99.9 & 99.99 & RF & BoT-IoT \\
[10] & 99.79 & 99.09 & 99.77 & 99.43 & RF & Edge-IIoT \\
[71] & 99.73 & 99.97 & 93.3 & 96.54 & MLP & CICIDS2017 \\
[35] & 98.17 & 97.63 & 98.08 & 97.90 & EWEA & TON\_IoT \\
[72] & 99.74 & 100 & 99 & 100 & CNN, VGG19 & CIC-DDoS2019 \\
[40] & 99.7 & 99.7 & 99.7 & 99.7 & RF &
\begin{tabular}{l} \_attacks\_N\_BaloT \\ IoTID20 \\ \end{tabular} \\
[36] & 99.7 & 98.3 & - & 85.9 & CNN & IoTID20 \\
[52] & 99.51 & 99.6 & 99.52 & 99.51 & DENS + GRU) & CSE-CIC-IDS 2018 \\
[57] & 99.45 & - & - & 99.52 & RF & UNSW-NB15, IoT23, TON\_IoT \\
[53] & 99.1 & 98.4 & 99.1 & 98.7 & XGBoost & TON\_IoT \\
[28] & 99 & 100 & 99 & 91.6 & EWM\_ACUSUM+KNN & Bot-IoT \\
[63] & 99 & 99 & 100 & 99 & RF & Bot-IoT \\
[58] & 99 & 100 & 100 & 100 & SVM & TON\_IoT \\
[64] & 98.78 & 99.03 & 99.35 & 98.48 & CNN + LSTM & CICIDS2019 \\
[46] & 98.69 & 99 & - & 99 & XGBoost & CICIoT2023 \\
[77] & 98.6 & - & 98.7 & 98 & RF & IoT-CIDDS \\
[86] & 98.5 & - & - & - & SVM + Bagged Trees & generated dataset \\
[68] & 98.5 & 98.5 & 98.44 & TabNet algorithm & CIC-DDoS2019 \\
[32] & 98.2 & - & - & NB & custom \\
[37] & 98 & 100 & 98 & 99 & SVM & custom \\
[49] & 97.39 & - & - & - & DT + RF + SVM & CICDDoS2019 \\  & & & & & Stacking (DT, MLP RProp, Logistic & TON\_IoT \\
[75] & 97.31 & 95.8 & 96.88 & 96.39 & Regression) \\
[29] & 97.16 & 97.41 & 99.1 & - & CNN + LSTM & CICIDS2017 \\
[50] & 97.06 & 97.31 & 97.02 & 97 & RF & MedBiT \\
[54] & 97 & 98 & - & 95.9 &
\begin{tabular}{l} variational dynamic Bayesian \\ algorithm+HMM \\ \end{tabular} & DARPA2000 \\
[51] & 95.32 & 95.8 & 95.32 & 94.81 & RF & MedBiT \\
[38] & 95 & 95 & 95 & 94 & GADAD-ET & TON\_IoT \\
[65] & 94.5 & 93.3 & 95.3 & 94.3 &
\begin{tabular}{l} K-means + Gaussian mixture + \\ one-class SVM \\ \end{tabular} & CIC-DDoS2019 \\
[66] & 88.73 & 89.92 & 88.73 & 88.53 & APSO-CNN-SE & UNSW-NB15 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Binary classification performance (sorted by accuracy). There were significant performance variations between various ML models. For binary classification, RF, DT, and XGBoost were the overall best performers among other models. On the contrary, for multiclass classification, the RF, K-Nearest Neighbors (KNN), and Long Short Term Memory (LSTM) models performed better, suggesting that the flexibility and robustness of the above methods were more suitable to deal with the increased complexity of the multiclass issue. These findings verify the relevance of algorithmic selection to maximize outcomes. While the majority of studies emphasize accuracy as the primary measure, it is class distribution-prone and can be misleading with high class imbalances in the datasets. In response to these limitations, numerous studies have placed emphasis on measures such as F1 score, which provides a balanced perspective with regard to both recall and precision.