# A TDD Framework for Automated Monitoring in Internet of Things with Machine Learning +
## 7 Final Considerations  
Considering the challenge of quality assessment in IoT systems, this work addressed the topic of automated non-functional requirements evaluation. The objective is to enhance the Testability non-functional requirement of IoT systems under development. The proposal is evaluated in academic and smart home study cases with 25 devices, consisting of 15 different firmware versions collected in one week, with a total of over 550,000 IoT status readings. K-Means algorithm was applied to free memory available, internal temperature and Wi-Fi level metrics to automatically monitor the IoT devices under development to identify device constraints violation and provide insights for monitoring frequency configuration of different firmware versions. The clustering results enabled to assess if new data of a specific IoT device fits into bad or good status. In the remote lab, the bad status accounted for around 60% of the total readings. The proposed framework guides the IoT system designer from the business domain to the non-functional requirement dynamic evaluation, based on RM-ODP [7] standard. The TDD tool is built with K-Means clustering algorithm application in real testbed data to acquire a high resemblance to real deployment scenarios. According to the proposed framework, the TDD tool could be used to identify IoT development problems earlier in the development process. The automated test results from the remote lab and smart home study cases indicate that the identification of device constraints violation is possible. To the best of the authors' knowledge, it is the first TDD approach for IoT module automatic assessment which uses machine learning based on real testbed data. Some of the related work carried out experiments in testbeds with real hardware devices but none of them used machine learning, which motivated the novel TDD approach validated with two study cases. As future work, the comparison of K-Means with other unsupervised learning algorithms could be performed. Other possibility is to test different deployment conditions with synthetic firmware versions with known resource misuses to investigate how the number of clusters may affect the proposed automated tool performance. Supervised algorithms for time series prediction and firmware version classification tasks also might be applied to the collected data to expand the TDD approach to the security domain, focusing on anomaly detection. A limitation of the proposed framework and the associated TDD tool based on machine learning is the need to know the total number of clusters beforehand, because the tool uses the K-Means algorithm. Such a limitation may be investigated in future work by testing other machine learning models. For example, graph-based deep learning approaches have been proposed as a promising solution for relevant problems (e.g., anomaly detection in communication networks [34; 35], intrusion detection in IoT environments [36]). Additionally, the framework was validated with two case studies, therefore other scenarios may contribute to enhance its relevance. Conceptualization, V.T.H. and R.A.; methodology, V.T.H., J.C.E. and R.A.; software, V.T.H., M.A.P., B.T., R.B.J. ; validation, V.T.H. and C.R. ; investigation, A.Q.F. and M.A.P. ; resources, V.T.H. and J.C.E. ; data curation, V.T.H. ; writing--original draft preparation, V.T.H., A.Q.F., R.A., C.R. ; writing--review and editing, V.T.H., W.V.R. and J.C.E. ; supervision, J.C.E. and R.A.; project administration, V.T.H. ; funding acquisition, V.T.H. and R.A. All authors have read and agreed to the published version of the manuscript. This research was funded by the Graduate Program in Electrical Engineering (PPGEE) from the Polytechnic School of the University of Sao Paulo. Data Availability Statement: Not applicable.