## 4 Results
### Machine Learning-Based IDS Implementation on IoT Device
\begin{table}
\begin{tabular}{c c c c c} \hline
**No. of Features** & **PCA** & **XGBoost + PCA** & **Chi-Square** & **XGBoost + Chi-Square** \\ \hline
10 & 1.1547 & 1.7944 & 1.3518 & 1.9761 \\ \hline
15 & 1.2732 & 1.8900 & 1.4709 & 2.0856 \\ \hline
25 & 1.2629 & 1.8813 & 1.6112 & 2.2363 \\ \hline \end{tabular}
\end{table} 
Table 6: PCA vs chi-square for multiclassification using XGBoost—inference time in milliseconds. is not feasible to implement the RF model with 200 estimators since the required memory is not available on an ESP32-based edge device. The ANN model-based IDS is also studied in this work. Two-layered and three-layered ANN models are used for the IDS. The best accuracy achieved with the ANN model (Layer 1 = 96, Layer 2 = 64, Layer 3 = 32) is 98.93%. Due to the limited storage and computational power of the microcontroller-based edge device, our objective is to identify an IDS that can deliver accurate results within a shorter time frame. The XGBoost-based IDS has proven superior to DT, RF, and ANN in terms of accuracy, FNR, and FPR. The highest accuracy achieved by the XGBoost-based IDS is 99.92%, but it requires 3002.78 \(\upmu\)s for threat detection. In contrast, XGBoost's performance is only 0.07% better than DT, yet DT completes computations in 11.36 \(\upmu\)s. Considering DT's significantly quicker detection time compared to XGBoost, it could be the preferred choice \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**ML Model** & **Accuracy** & **Precision** & **Recall** & **F1-Score** & **FNR** & **FPR** & \
\begin{tabular}{c} **Computation** \\ **Time \(\upmu\)s (Average)** \\ \end{tabular} &
\begin{tabular}{c} **Program** \\ **Storage** \\ \end{tabular} \\ \hline XGBoost (Estimator = 50,  Depth = 8) & 99.91\% & 0.99 & 0.99 & 0.23\% & 0.04\% & 1840.70 & 80.92\% \\ \hline
**XGBoost (Estimator = 100,  Depth = 8)** & **99.92\%** & **0.99** & **0.99** & **0.23\%** & **0.04\%** & **3002.78** & **82.71\%** \\ \hline XGBoost (Estimator = 150,  Depth = 8) & 99.92\% & 0.99 & 0.99 & 0.23\% & 0.04\% & 3514.39 & 83.34\% \\ \hline XGBoost (Estimator = 200,  Depth = 8) & 99.91\% & 0.99 & 0.99 & 0.25\% & 0.04\% & 4100.54 & 83.86\% \\ \hline RF (Estimator = 50,  Depth = 8) & 98.67\% & 0.95 & 0.98 & 0.97 & 1.34\% & 1.32\% & 628.62 & 72.76\% \\ \hline RF (Estimator = 100,  Depth = 8) & 98.89\% & 0.96 & 0.98 & 0.97 & 1.28\% & 1.06\% & 1565.01 & 79.0\% \\ \hline RF (Estimator = 150,  Depth = 8) & 98.72\% & 0.95 & 0.98 & 0.97 & 1.28\% & 1.28\% & 2679.09 & 85.12\% \\ \hline RF (Estimator = 200,  Depth = 8) & 98.71\% & 0.95 & 0.98 & 0.97 & 1.28\% & 1.29\% & – & Memory overflow \\ \hline DT (Depth = 6) & 97.94\% & 0.92 & 0.99 & 0.95 & 1.01\% & 2.34\% & 6.05 & 65.83\% \\ \hline DT (Depth = 8) & 99.05\% & 0.98 & 0.97 & 0.97 & 2.37\% & 0.56\% & 5.09 & 65.94\% \\ \hline DT (Depth = 10) & 99.53\% & 0.98 & 0.99 & 0.99 & 0.92\% & 0.34\% & 6.96 & 66.13\% \\ \hline DT (Depth = 12) & 99.69\% & 0.99 & 0.99 & 0.99 & 0.54\% & 0.25\% & 8.82 & 66.40\% \\ \hline DT (Depth = 14) & 99.78\% & 0.99 & 0.99 & 0.45\% & 0.15\% & 9.90 & 66.56\% \\ \hline DT (Depth = 16) & 99.84\% & 0.99 & 0.99 & 0.39\% & 0.10\% & 11.61 & 66.68\% \\ \hline DT (Depth = 18) & 99.85\% & 0.99 & 0.99 & 0.99 & 0.34\% & 0.10\% & 11.82 & 66.72\% \\ \hline DT (Depth = 20) & 99.85\% & 0.99 & 0.99 & 0.99 & 0.31\% & 0.10\% & 11.36 & 66.73\% \\ \hline ANN (Layer 1 = 32,  Layer 2 = 16) & 98.62\% & 0.97 & 0.96 & 0.97 & 3.31\% & 0.65\% & 131.98 & 75\% \\ \hline ANN (Layer 1 = 64,  Layer 2 = 32) & 98.98\% & 0.97 & 0.97 & 0.97 & 2.59\% & 0.58\% & 296.12 & 76.15\% \\ \hline ANN (Layer 1 = 64,  Layer 3 = 32) & 98.92\% & 0.97 & 0.97 & 0.97 & 2.74\% & 0.62\% & 350.16 & 76.34\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: IDS implementation on microcontroller-based IoT device using ToN_IoT dataset (connection, statistical, and DNS activity features). for implementing the IDS on edge devices. However, the accuracy of the IDS is crucial. Therefore, XGBoost should be preferred for the implementation of the IDS.