## 5 ML Setup and Results Analysis
### Tests and Results Analysis  
Now the evaluation and the results of the developed ML models after training and testing can be discussed. The results analysis was carried out by applying Wi-Fi and MF models separately, and the combined WF-MF models on three different indoor locations and other three different buildings (Outdoor) locations. From now on, we call these different types of models WF-ML, MF-ML and WF-MF models. The target of this analysis is to find the best-used metrics in different situations. In Figures 12-14, we can see that the results of the indoor locations indicate different behavior. This is due to the nature of the datasets in each location and the availability of the metrics for the GDS generation. In the WRL Lab, the WF-ML and the WF-MF show nearly the same performance, and they are outperforming the MF-ML. In Mech 606 LH, WF-ML and MF-ML does not have a significant change, and WF-MF is worse than them. In LH 26, WF-MF has superior performance due to the low number of Wi-Fi access points in this LH. We can conclude by studying the three locations that the WF-ML is a better model for the indoor locations in the case of the availability of at least two or three Wi-Fi APs. WF-MF was good in two cases but having a model inference with all these parameters uses a large amount of processing power and time in the long range. Figure 12: The models comparison in WRL Lab (Indoor). Figure 13: The models comparison in Mech 606 LH (Indoor). The buildings or the outdoor environment layer in Figures 15-17 can now be discussed. In the communications department building and Neo Lecture Halls building, the MF-ML have noticeably higher performance. The mechanical department building had better performance with WF-MF. It is important to note that most of the outdoor environments lack the presence of Wi-Fi coverage. Therefore, we did not get good results with WF-MF or WF-ML as in the first two locations discussed. Thus, we can conclude that MF-ML is the best model for the buildings (outdoor) locations. After we chose the WF-ML for the indoor locations layer and MF-ML for the buildings layer, Figures 18 and 19 show the results of each location we tested on. The performance of the WF-ML models in Figure 18 is above the 95% accuracy taking into consideration that each indoor location offers a challenge to the ML models. Each location contains at least two or three Wi-Fi access points that are detected by the mobile application. We can see that the best models are the Random Forest, ANN, and CNN as each one has an advantage to a specific case. The random forest is mostly the highest accuracy model as the classical machine learning models do not require large datasets. However, the deep learning methods require large datasets with at least 1000-1500 data rows or wireless metrics scans. However, for our tests, we achieved good results with the CNN due to its ability of deep feature extraction. In the IPDSC Lab, which  
Figure 14: The models comparison in Lecture Hall 26 (Indoor). Figure 15: The models comparison in Communications department building (Outdoor). Figure 19: The models accuracy in all outdoor buildings geofences using MF-ML. Figure 18: The models accuracy in all indoor geofences using WF-ML. Figures 20-23 show a typical receiver operating characteristic (ROC) curve for the chosen models in some test locations. ROC can be considered as an evaluation metric for binary classification problems that shows the performance of a classification model at all classification thresholds. The higher the area under the curve (AUC), the better the performance of the model at distinguishing between the positive and negative classes. When AUC = 1, then the classifier can perfectly distinguish between all the positive and the negative class points correctly. If the AUC was 0, then the classifier would be predicting all negatives as positives, and all positives as negatives. When 0.5 < AUC < 1, there is a high chance that the classifier will be able to distinguish the positive class values from the negative class values.