# Indoor Localization System Using Fingerprinting and Novelty Detection for Evaluation of Confidence
## 4 Performance Evaluation
### Methodology  
We evaluated the performance of our solution in a large-scale, real-world testbed, shown in Figure 6. Since we are focused on applications based on small, low-processing, low-power, and battery-operated mobile devices, we used nodes and anchors with BLE technology. Figure 7a shows the hardware used for the mobile devices, a smartwatch-based wearable device, while Figure 7b shows the hardware used as anchor nodes for the location data gathering.  
Since our main application is to localize students, our testbed was implemented in a school area of 645 m\({}^{2}\) consisting of 15 spaces (11 rooms and 3 halls). The complete map of the area is depicted in Figure 6. To cover the entire area, 15 anchor nodes were installed on the ceiling of the rooms in places where it was somewhat convenient to connect them to the mains power. These anchor nodes are depicted in blue in Figure 6. This base was collected and used in other works from our research group [20].  
To execute the training phase of LocFiND and generate the fingerprint database, we evenly distributed 150 reference points (RPs) throughout the area. These RPs are depicted  
Figure 6: Real-world application/testbed map: area of 645 m\({}^{2}\) (43 m \(\times\) 15 m), 11 rooms of averaged 42 m\({}^{2}\) (7 m \(\times\) 6 m), 3 halls, 15 anchor nodes, and 150 reference points. For each reference point, 100 samples were gathered to a total of 15,000 samples. 11 different mobile devices were used, 9 for training and 3 for testing.  
Figure 7: Device Nodes [20]. (**a**) Bluetooth-based, smartwatch-like devices are used as mobile devices. (**b**) Bluetooth-based access points are used as anchor nodes.  
as gray dots in Figure 6. For each RP, we went to that location and started sending data packets using 11 different mobile devices. We collected a total of 100 samples per RP to a total of 15,000 samples in our fingerprint database. Finally, in our results shown in the following sections, we used the samples from nine mobile devices as training and the samples from three other mobile devices as testing. Thus, the samples used for testing are from never-seen devices from the training.  
To choose the parameters of KNN, the GridSearchCV method of the scikit-learn was used, which is a tool of selection of parameters using cross-validation on the training base, thus avoiding overfitting. By using GridSearchCV with five folds, we set the main parameters in KNN the value of \(k=9\). Selecting \(k\) is not an exact science because it's hard to estimate how well your fold represents your overall dataset. For this reason, we used the GridSearchCV with a 5-fold cross-validation to find the best \(k\) value without overfitting the system. Finally, we also used different devices for training and testing, so that the values in testing are from never-seen devices. This increases the chances for the proposed solution to work with new devices in a real-world application.  
As a baseline of comparison for the results, a traditional fingerprint-based solution that uses the RSS database and KNN (with \(k=9\)) was used, having a 96% accuracy for room classification and a 36% for point estimation with an average error of 1.9 m. Since in this case there is no evaluation of confidence, 100% of the position estimations were sent to the application. Our goal for LocFiND is to improve these classifications and average localization error while delivering as many position estimates for the application as possible.