## 4 Evaluation and Analysis
### Experimental Results
#### 4.3.1 Qualitative and Quantitative Comparisons
This demonstrates that our feature refinement and weighted similarity modules can significantly enhance the accuracy of LCD. It also shows that MetricNet and SAES achieve the two best results on St. Lucia, which verifies the robustness of the patch-based LCD architecture. As for the other two models designed for changing environments, CALC2.0 [36] and [PERSON] et al. [37] obtain reliable results by using different technical solutions. By taking semantic and geometric information into account, CALC2.0 is able to perform well under changing lighting conditions. [PERSON] et al. 's work also proves the importance of clustering  
Figure 8: The maximum recall rate at 100% precision of different networks. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Algorithm**} & \multicolumn{3}{c}{**Dataset**} \\ \cline{2-4}  & **Gardens Point** & **Nordland** & **St. Lucia** \\ \hline SAES [35] & 0.6997 & 0.7162 & 0.7098 \\ CALC2.0 [36] & 0.7284 & 0.6348 & 0.5869 \\ [PERSON] et al. [37] & 0.6900 & 0.7500 & 0.6800 \\ Place-ResNet [23] & 0.3048 & 0.8209 & 0.6860 \\ NetVLAD [24] & 0.7433 & 0.5320 & 0.5660 \\ SeqSLAM [5] & 0.4300 & 0.7233 & 0.2267 \\ MCN [31] & 0.7400 & 0.8211 & 0.6241 \\ SMCN [32] & 0.6500 & 0.5300 & - \\ CAE-VLAD-Net [25] & **0.8400** & 0.7500 & - \\ [PERSON] et al. [42] & 0.8380 & 0.7280 & - \\ \hline Pairwise & 0.5320 & 0.6843 & 0.5781 \\ MetricNet\_\(\omega\) & 0.7289 & 0.6389 & 0.6920 \\ MetricNet\_\(\alpha\) & 0.7793 & 0.8173 & 0.7422 \\
**MetricNet** & 0.7831 & **0.8458** & **0.7476** \\ \hline \hline \end{tabular} - means that the corresponding method did not experiment on the open dataset. The best performance is in **bold** and the second best is underlined.  
\end{table} 
Table 3: The mAP Comparison of Different Networks. and PCA-based descriptor standardization. However, MetricNet yields more promising and stable results. It can be seen that our method performs well without the addition of extra perceptual information. Additionally, incorporating similarity measurement inspired by metric learning can further enhance the strengths of the proposed method. Although the SOTA methods based on sequence-to-sequence matching, i.e., SeqSLAM [5], MCN [31], and SMCN [32], produce promising average precision by exploiting the sequential characteristic of robotic data streams, their results are inferior to those of MetricNet when it comes to long-term operation with significant appearance variations. The results of Place-ResNet [23], NetVLAD [24], and CAE-VLAD-Net [25] demonstrate their robustness to environmental changes to some extent. The learned feature representation of CAE-VLAD-Net is more effective than the simple CNN-based network used in our method. We conclude this is mainly because of the combination of convolutional neural networks and stacked auto-encoders. [PERSON] et al. [42] propose the multi-tuplet clusters loss function together with a mini-batch construction scheme, which shows advantages when dealing with changing illumination. Nevertheless, our approach yields much better performance on seasonal changes in Nordland. We further conduct an ablation experiment to evaluate the effectiveness of the self-adaptive parameters. Note that the Pairwise baseline is achieved by a cosine similarity-based comparison using deep image descriptors computed by AlexNet. MetricNet_\(\omega\) is the ablation model with \(\omega_{i}=1/4\). MetricNet_\(\alpha\) is the version with the constant parameter \(\alpha\) calculated by:  
\[\alpha=\frac{1}{1+e^{d}} \tag{20}\]  
where \(d\) is calculated by Equation (9). It can be seen that MetricNet delivers the best estimation on the Nordland and St. Lucia datasets and the third-best one on Gardens Point. When compared to MetricNet_\(\omega\), the results of MetricNet_\(\alpha\) are much closer to those of MetricNet. They both yield lower average precision than MetricNet.