## 4 Results
### Machine Learning-Based IDS Implementation on IoT Device  
An ML-based IDS has been embedded in a microcontroller-based smart thermostat, a low-power and cost-efficient IoT device functioning as an HTTP client to upload building temperature and humidity data to the AWS cloud server via WiFi technology. The RF-, XGBoost-, DT-, and ANN-based IDSs are embedded in the smart thermostat to determine their feasibility in terms of accuracy, storage requirements, and inference time. The TinyML library is used to embed the trained IDS on the microcontroller. The trained models of RF, XGBoost, ANN, and DT are embedded in the smart thermostat to classify the network traffic as attack or normal. Low-power IoT devices are vulnerable to various security threats due to their limited resources. Consequently, these devices may be exploited for backdoor, DoS/DDoS, MITM, scanning, and ransomware attacks. Given the restricted access of low-cost IoT devices to network traffic features, researchers opted to deploy an IDS on a Raspberry Pi-based edge device, leveraging tools like tcpdump and Wireshark for extracting network features. This study focuses on implementing the primary defense layer on the microcontroller-based IoT device (smart thermostat), which lacks packet analyzer tools. An ML-based IDS is specifically designed for the binary classification of attacks. The IDS classified backdoor, scanning, DDoS, MITM, and ransomware as attacks (i.e., 1), while normal traffic was classified as 0. 3.1 ML-Based IDS Implementation on Smart Thermostat Using Connection, Statistical, and DNS Activity Features of TON_IoT Dataset  
In this work, ANN, RF, XGBoost, and DT are tested as ways of implementing the binary classifier IDS. The TON_IoT dataset is used for training the IDS at the edge device. The dataset is split into 70% training and 30% testing data. The 10-fold cross-validation has been applied during training to ensure the generalization of the ML model. Due to limited computational and storage resources at the edge device, we trained the IDS with 25 features (i.e., connection activity, statistical activity, and DNS activity). The ML-based IDS is evaluated in terms of accuracy, precision, recall, F1-score, false positive rate (FPR), and false negative rate (FNR). The testing results of the binary classification of the XGBoost-, RF-, DT-, and ANN-based IDSs are shown in Table 7. These results indicate that the RF-based IDS (100 estimators, depth = 8) outperformed the other RF models. Additionally, the DT model with a depth of 18 achieved an accuracy of 99.85%. Regarding ANN, experiments were conducted with 50, 100, 150, and 200 neurons, revealing that all ANN models performed best at 100 epochs. Among these, the three-layered ANN model exhibited the highest accuracy of 98.93%. XGBoost outperformed the other ML models and achieved the highest accuracy of 99.92%. The implementation results are shown in Table 7. The table shows the testing accuracy, program storage, and inference time average of 30 packets. DT outperformed RF, XGBoost, and ANN in terms of inference time, being capable of classifying network traffic as normal/attack in 5.09 us with 99.05% accuracy. The DT-based IDS (depth = 8) consumed 65.94% of program memory of the IoT device. The DT model with a depth of 20 achieved the best accuracy of 99.85% and the inference time for classifying network traffic is 11.36 us. The precision, recall, and F1-score for this model are 0.99, while the program storage required for this model is 66.73%. In terms of accuracy, FNR, FPR, and XGBoost with 100 estimators outperformed RF, ANN, and DT with an accuracy of 99.92%, FPR of 0.04%, and FNR of 0.23%, but the inference time is 3002.78 us. The best accuracy achieved by the RF-based IDS is 98.89% with 100 estimators. The FNR for this model is 1.28% and the FPR is 1.06%. It