# Collaborative Indoor Positioning by Localization Comparison at an Encounter Position
## 4 Experiments
### Evaluation of Localization Comparison  
The localization comparison model was trained on the training dataset using a cross-entropy loss function. The validation dataset was created by extracting one-quarter of the training dataset and the best model was determined by evaluating the validation dataset. Then, we evaluated the model using the walking data from the start to the encounter in the test dataset. We averaged the results obtained by cross-validation. Table 4 shows that an accuracy of 0.83 was obtained on the test dataset. Accuracy was low in Building No. 1 and AER2F, where their localizations were highly accurate. Figure 11 shows examples of comparison pairs of localizations in AER2F. The comparison was correct when the metrics were considerably different. In addition, the number of disappearing particles significantly affected the comparison determination. Many particles disappeared in Figure 11a. The failed example had two close localizations in the metric. The incorrect result was produced by the features related to turns and intersections. Figure 11d had a smaller average distance from each turn to the nearest intersection than Figure 11c. \begin{table}
\begin{tabular}{l l} \hline \hline
**Facility** & **Accuracy (\%)** \\ \hline Bldg. No. 1 & 0.66 \\ AER1F & 1.0 \\ AER2F & 0.68 \\ Fujisaki & 0.77 \\ Mitsukoshi & 0.96 \\ Agriculture & 1.0 \\ Complex & 0.78 \\ SPAL & 0.79 \\ \hline Average & 0.83 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Accuracy of the localization comparison. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Facility** & **Walk** & **Ours** & **Ours** & **Ours** \\  & & \((\boldsymbol{o=0.5})\) & \((\boldsymbol{\theta_{o}+N(0,\sigma)})\) & **(Full)** \\ \hline Bldg. No. 1 & \( \begin{array}{c}1\\ 2\end{array} \) & \( \begin{array}{c}-7.9\\ \mathbf{1.1}\end{array} \) & \(\begin{array}{c  
Figure 12 shows the importance of the features of the comparison model. The importance represents the contribution of each feature to decreasing the loss function. The importance was calculated using the validation dataset. The input features to the model consisted of two localizations. Then, we averaged the corresponding features to obtain the importance of the 15 features. Generally, the importance \(I\) at a node of a decision tree can be calculated by \(I=rL-r_{1}L_{1}+r_{2}L_{2}\), where \(L\) and \(r\) represent the loss value at the node and the ratio of the sample size at the node to the total sample size, respectively. Likewise, \(L_{1}\) and \(r_{1}\) are of the left leaf of the node. In this work, binary cross-entropy was used to calculate the loss value. Thus, the importance of a feature \(f\) is the total importance at all nodes using \(f\). If the importance is larger, the feature significantly reduces the loss value. The results showed that the average distance from each turn to the nearest intersection was the most important feature. The proposed method detects the user's turn positions, and then the distance is measured. Intuitively, the localization performance is high if the predicted turn position is close to the GT intersection. Therefore, the feature is a critical metric for localization performance. Figure 13 shows the average outputs of the comparison model for correct and incorrect cases. Values were unavailable if there were only correct cases. For simplicity, the output was subtracted from 1 if it exceeded 0.5. Thus, a smaller value indicates a higher confident estimation. In contrast, a value close to 0.5 means an ambiguous estimation. The results showed that the model was more confident for correct than incorrect instances. The outputs in Building No.1 were less confident, since both correct and incorrect were close to 0.5. In fact, the comparison accuracy was the lowest, 0.66, among the facilities. Thus, the model did not make a significant mistake in its estimation on this map. Figure 11: Results of localization comparison in AER2F. Red and cyan dots indicate the estimated and starting positions, respectively. Blue dots depict the detected turns.