# URSIM: Unique Regions for Sketch Map Interpretation and Matching
## 6 Experiments
### Evaluation of the \(F_{1}\) Scores
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Algorithm** & **Baseline** & \(\mathbf{T_{1}}\) & \(\mathbf{T_{2}}\) & \(\mathbf{T_{3}}\) & \(\mathbf{T_{4}}\) \\ \hline \multicolumn{6}{c}{_Median values_} \\ \hline Hungarian & 0.27 & **0.50** & 0.42 & 0.40 & 0.47 \\ VF2 & 0.00 & 0.00 & 0.00 & 0.00 & 0.00 \\ Planar & 0.67 & 0.74 & 0.67 & 0.67 & URSIM: **0.80** \\ Human VS Human & & 0.89 & & \\ \hline \multicolumn{6}{c}{_Mean \(\pm\) one standard deviation_} \\ \hline Hungarian & \(0.27\pm 0.23\) & \(0.48\pm 0.24\) & \(0.47\pm 0.24\) & \(0.42\pm 0.22\) & \(\mathbf{0.50\pm 0.24}\) \\ VF2 & \(0.03\pm 0.17\) & \(\mathbf{0.22\pm 0.33}\) & \(\mathbf{0.22\pm 0.33}\) & \(0.13\pm 0.29\) & \(0.13\pm 0.29\) \\ Planar & \(0.61\pm 0.30\) & \(0.68\pm 0.28\) & \(0.70\pm 0.23\) & \(0.70\pm 0.25\) & URSIM: \(\mathbf{0.73\pm 0.22}\) \\ Human VS Human & & \(0.84\pm 0.16\) & & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Median and mean \(F_{1}\) scores over the sketch map dataset. Using the anchors and the standardized sizes as weights on the vertices leads to an increase for both the median and mean \(F_{1}\) score, i.e., increase of the accuracy of the correspondences. Figure 9: Flowcharts describing the attributes used to build the descriptors in the tests of Section 6.2. (**a**) In the first test, \(T_{1}\), only the topology was used to generate the region descriptors. (**b**) In the second test, \(T_{2}\), both the topology and the anchors were used to generate the region descriptors. (**c**) The third test, \(T_{3}\), used the topology and the standardized size of regions. (**d**) The last test, \(T_{4}\), used the topology, the anchors, and the standardized size of regions. Indeed, both the Hungarian matching and VF2 only consider the topology through the region descriptors. On the other hand, the planar matching presented in Section 5.2 starts from a pair of vertices, i.e., the seed, and expands the correspondences one neighborhood at a time, sequentially (see Line 10 of Algorithm 1). Therefore, the topology is taken into account both through the descriptors and by the planar graph matching algorithm during the neighborhood matching step. Looking at the results of the planar matching in Table 1, URSIM has higher scores than the method presented in our previous work [3]. Furthermore, URSIM has the highest median and mean \(F_{1}\) scores, with \(0.80\) and \(0.73\pm 0.22\), respectively, and, removing any attribute of the calculation of the LFS, leads to lower mean and median \(F_{1}\) scores. Thus, we conclude that every attribute used in URSIM to build the descriptors and find correspondences is needed. As a reference, we also calculated the median and mean \(F_{1}\) scores between both sets of ground truth correspondences given by the users. While one set of ground truth correspondences was used in place of the correspondences returned by a matching algorithm, the other set was used as the ground truth set. One could expect the results for humans to be close to the maximum score, 1. However, the mean \(F_{1}\) score is \(0.84\pm 0.16\) and the median is \(0.89\), showing that finding correspondences between a sketch map and a metric representation is not a trivial task, even for humans. The high level of abstraction and error in local scales, as shown in Figure 1 and mentioned in Section 3, make matching sketch maps a challenge even for humans.