# Landmark Generation in Visual Place Recognition Using Multi-Scale Sliding Window for Robotics
## 4 Experiments
### Detailed Analysis of Landmark Scale and Space Distributions  
We first verified through a controlled experiment our assertion that scale distribution of the landmarks is important to the performance of landmark-based place recognition in recognizing scenes with significant viewpoint changes. Figure 7a shows three representative scale distributions used in the experiment where Set 1 is the uniform distribution we used in our proposed method, and Sets 2 and 3 are non-uniform distributions where either small landmarks (Set 2) or large landmarks (Set 3) dominate. The recall-precision curves in Figure 7b,c, obtained from the Mapillary Berlin and Gardens Point Campus datasets, support our conclusion that a uniform distribution of landmark scale provides the optimal result. In contrast, when there is little viewpoint change, as in the Freiburg Across Seasons dataset, recognition performance is independent of the landmark scale distribution (Figure 7d). Qualitative results of the experiment are shown in Figure 8 in which our proposed MSW method is able to handle scenes with both significant and minor viewpoint change (Figure 8a,b, respectively), correctly matching landmarks at different scales and the same scale, respectively. \begin{table}
\begin{tabular}{l l l l l} \hline \hline  & **NetVLAD**[36] & **SP-Grid**[34] & **Edge Boxes**[17] & **MSW(Our)** \\ \hline Mapillary Berlin & **0.8441** & 0.7242 & 0.7285 & 0.8231 \\ Gardens Point Campus & 0.7788 & 0.7670 & 0.8274 & **0.9203** \\ Freiburg Across Seasons & 0.8748 & 0.8359 & 0.8109 & **0.9040** \\ Oxford RobotCar & **0.9193** & 0.8043 & 0.8315 & 0.9074 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of our method with the state-of-the-arts (AP). Figure 7: (**a**) Three representative scale distributions of landmarks. Set 1 shows uniform distributions, while Set 2 and 3 are non-uniform. Results of three settings on: (**b**) Mapillary Berlin; (**c**) Gardens Point Campus dataset including large viewpoint changes; and (**d**) Freiburg Across Seasons dataset with slight viewpoint changes. Another important assumption of our MSW method is that landmarks at small scales are not as reliable as those at the higher scales. This assumption is verified by the result shown in Figure 9 that plots landmark matching success rate with respect to landmark scale, using sample images in the Gardens Point Campus dataset whose ground truth for matching landmarks was produced manually. Clearly, small landmarks are not as reliable as landmarks at higher scales. This contributes to the poor performance of landmarks generation methods whose landmark population is dominated by small landmarks such as Edge Boxes. Furthermore, to analyze the impact of the small-scale landmarks on the recognition performance, from the candidate regions generated by Edge Boxes, we removed the landmarks with scales smaller than 0.1 and used the remaining 100 regions for feature matching. We then compared the performance of the method that uses the original Edge Boxes landmarks vs. scale-filtered Edge Boxes landmarks on Mapillary Berlin and Freiburg Across Seasons datasets. As shown in Figure 10, the performance of the scale-filtered Edge Boxes using only landmarks at large scales is slightly worse than our MSW method, whereas it is significantly better than the original Edge Boxes on both datasets. This indicates that landmarks at higher scales tend to improve performance. In addition, landmarks at scales approaching 1.0 are similar to using the whole image with poor shift-invariance property [13; 34], and their use should therefore be discouraged as well. Figure 8: Examples of matched images with extracted and matched landmarks by our MSW method from: (**a**) Mapillary Berlin dataset; and (**b**) Freiburg Across Seasons dataset. The matched landmarks on Mapillary Berlin dataset are from different scales, while they have same scales on Freiburg Across Seasons dataset. Finally, we include an observation on the spatial landmark distributions of the three competing methods.