# A Comparative Analysis for Optimizing Machine Learning Model Deployment in IoT Devices
## 3 Methodology
### Evaluation Metrics  
The evaluation of our machine learning models encompasses a holistic approach, considering a broad spectrum of metrics to assess both traditional performance and environmental impact. This comprehensive evaluation ensures a nuanced understanding of each model's effectiveness and its sustainability within IoT applications. Performance Metrics on Test Data: The evaluation metrics for model performance on test data include accuracy, F1-score, precision, and recall. These metrics provide a multifaceted view of the model's effectiveness. * Accuracy: Accuracy is the proportion of correctly predicted observations out of the total observations. It provides a general measure of the model's performance. However, in scenarios with imbalanced datasets, accuracy might not be sufficient to understand the model's true performance. In our context, accuracy helps to obtain the overall correctness of our ML models across various IoT tasks. * Precision: The ratio of correctly predicted positive observations to the total predicted positive observations. It reflects the model's ability to return relevant results. Precision is crucial in IoT-based real-world applications, where false positives can lead to unnecessary negative consequences that can be costly. * Recall (Sensitivity): The ratio of correctly predicted positive observations to all observations of individual classes. It measures the model's capability to find all relevant cases. High recall is particularly important in scenarios where missing positive instances can lead to significant issues, such as in security or health monitoring IoT applications. * F1 Score: The harmonic mean of precision and recall. This score takes both false positives and false negatives into account, making it a better measure of the incorrectly classified cases than the accuracy metric alone. The F1 Score is critical in our study as it offers a more summarized measure of the model's performance by accounting for both false positives and false negatives, thus providing a comprehensive understanding of the model in various IoT applications. These performance metrics offer a comprehensive assessment of the model's predictive accuracy and its ability to manage trade-offs between precision and recall, essential in applications where the cost of false positives differs from that of false negatives. Computational and Environmental Impact Metrics: Additionally, the evaluation includes metrics such as execution time, memory utilization, power consumption, and CPU temperature. These metrics are pivotal for assessing the model's computational efficiency, resource demands, and environmental impact, guiding towards sustainable ML deployment in IoT systems. When applicable, we have used command line utilities to remain programming-language-agnostic. * Execution Time: Measures the total time taken by the machine learning model to complete its training process. A shorter execution time is generally preferable, indicating an efficient model capable of learning quickly, which is crucial in time-sensitive or resource-limited environments. In Python, the execution time can be accurately measured using the time library. * Memory Utilization: Reflects the amount of memory required by the machine learning models during their execution. Effective memory management is critical, especially in environments with limited memory resources such as IoT devices. The command line utility free is used to monitor and optimize memory consumption during model training. * Power Consumption: Indicates the electrical power used by the computing device while running the machine learning model. Energy efficiency is paramount in scenarios where power availability is constrained, such as mobile or embedded systems. Power consumption is typically measured using a USB power meter or multimeter to track the voltage and current draw of a device, like a Raspberry Pi, thus allowing for the calculation of power usage in watts. Power draw can be estimated using powertop. The command also logs CPU usage.