# A TDD Framework for Automated Monitoring in Internet of Things with Machine Learning +
## 5 Case Studies
### Remote Lab
The data collection procedure is detailed in the Figure 13. Each second the Python agent queries the IoT devices deployed in the smart home, which send their status to the MQTT broker. All messages are registered in the NoSQL database by the plug-in integrated with the HiveMQ MQTT broker. The TDD procedure is described in the Figure 14. The TDD tool with the trained machine learning model is subscribed in IoT status topics. Each status message from the IoT device under development (sent as a result of the periodic Python Agent request) is relayed to the TDD tool by the MQTT broker, and it is used to compare the current status to the clustered historical values from deployed devices. Figure 8:0 Smart Home study case architecture. Figure 8:1 Deployed IoT device status data collection with periodic request. We collected a total of 595,554 readings from 22 ESP8266 devices with 13 different firmware versions from 10 July 2021 to 17 July 2021. The metrics of the ESP8266 devices are freeheap in bytes (i.e., free memory available), wifi_level in decibels (i.e., local Wi-Fi network level), fs_usedbytes (i.e., total bytes used in the filesystem) and max_loop_wdt_cnt (i.e., the maximum loop watchdog count). The readings count by each of the 13 different firmware versions is presented in Figure 8:2  
A summary of the collected metrics of the 22 ESP8266 devices installed in the smart home is presented in the Figure 8:3  
Figure 8:4 TDD procedure for IoT device under development (smart home scenario). Figure 8:5 Readings count by firmware version. The correlation matrix of the Figure 17 shows the negative correlation between wifi_level and freeheap, and between wifi_level and fs_usedbytes. The K-Means clustering algorithm was performed using the aforementioned four metrics with the total number of cluster of two corresponding to a good and bad status clusters. After the clustering, the cluster label is performed by the IoT developer based on the visual representation, such as the one represented in Figure 8:6  
The clustering results are summarized in Figure 8:7 The firmware versions ArCond2, R4x2, RM6, RM8, and TV4x2 presented the worst status results, resulting from bad Wi-Fi signal and more integrated functionalities that may compromise performance. The firmware versions R4x2o, R4x4 ir5, R4x4o, RM1, RM1t10o, and WifiModem presented the best results, resulting from specialized functionality or better location for Wi-Fi level. The R4x4 ir firmware version presented an average status. Figure 8:8 Summary of the four ESP8266 metrics. Figure 8:9 Smart home metrics correlation matrix. With the trained model, it was possible to classify the IoT device under development status described in (1) in the good status cluster (i.e., cluster 1). \[ \begin{split}&{}^{\prime}f_{S}\_{usedBytes^{\prime}}:340858,\\ &{}^{\prime}max\_{loop\_{wdt}\_cnt^{\prime}}:992,\\ &{}^{\prime}freeheap^{\prime}:39112,\\ &{}^{\prime}wifi\_level^{\prime}:-75\end{split}  \tag{2}\]  
Considering the proposed framework, we describe the five steps considered:  
1. Business drivers: the motivation is to scale the energy efficiency solution to smart homes in the same region, considering the different communication conditions;
2. Non-functional requirements: similar to the remote lab scenario, the performance efficiency requirement is prioritized, considering the resource utilization. Another relevant requirement is the maintainability, considering the testability characteristic. 3. Engineering view: the architectural solution was illustrated in the Figure 12, and described in detail. 4. Dynamic system evaluation: initial descriptions are described in Figures 13 and 14. The historical clustering results are presented in Table 4, and one example of a status reading classified in the cluster 1 (good status according to the IoT developer rationale) is presented in (2). 5. Recommendations: considering that the deployed modules have the filesystem usage presented in Figure 16, it is desirable that the new IoT devices have filesystem usage of less than 300,000 bytes.