## 4 Evaluation and Analysis
### Experimental Results  
#### 4.3.1 Qualitative and Quantitative Comparisons  
In this experiment, comparisons are made on the Gardens Point, Nordland, and St. Lucia datasets. To comprehensively measure the effectiveness of MetricNet, we select multiple typical and representative approaches from different perspectives for comparison, such as the methods based on learned feature representations (Place-ResNet [23], NetVLAD [24], CAE-VLAD-Net [25]), sequence-to-sequence methods (SeqSLAM [5], MCN [31], SMCN [32]), methods designed for appearance variations (SAES [35], CALC2.0 [36], [PERSON] et al. [37]), and viewpoint changes ([PERSON] et al. [42]). To test SAES, we use the model trained on the SPED_900 dataset by the authors. The CALC2.0 is trained on the COCO dataset [54], and the implementation used for the experiments is the same as described in their paper. As for Place-ResNet, the pre-trained models are utilized to make the comparison. The MATLAB source of NetVLAD is available from [24] along with several sets of weights. The results of SeqSLAM [5], MCN [31], SMCN [32], CAE-VLAD-Net [25], [PERSON] et al. [42], and [PERSON] et al. [37] presented in this section are obtained from their published studies. For a fair comparison, the versions of the models chosen for the experiments are relevant to the methodology in our paper and largely follow the settings in their published papers. We evaluate the model's effectiveness from multiple perspectives. Firstly, the PR curve results of the methods designed for appearance variations are compared. The PR curves are drawn based on the experimental results, as shown in Figure 7. It can be seen that, as the similarity threshold increases, all the recall rates show a slow downward trend when the precision is high. MetricNet achieves the most promising performance due to its PR curve closer to the upper right corner. It demonstrates that MetricNet has a relatively prominent generalization ability in dealing with the test set with significant appearance variations. Compared to SAES, the backbone of our method, the improvements of MetricNet are mainly due to the utilization of channel-wise attention distillation and adaptively weighted similarity scores. Although our method achieves slightly lower precision as the recall rate increases up to a certain threshold on the Gardens Point and St. Lucia datasets, MetricNet is more advantageous in terms of the area under the PR-curve. It is noteworthy that our method performs best on the Nordland dataset compared to the experimental results on other datasets, highlighting the robustness of MetricNet to seasonal variations. We attribute this to the fact that the images in the dataset were taken along a fixed train track, which minimizes changes in viewpoint. Illumination variations have long posed a challenge in visual tasks. While our method exhibits some sensitivity to illumination variations in the St. Lucia dataset, it achieves the best performance overall. The corresponding results of the maximum recall rate at 100% precision are shown in Figure 8. It can be seen that MetricNet can achieve the highest recall rate at 100% precision, reaching 43.97%, 44.49%, and 26.71% on the three datasets, respectively. Compared to the other approaches, MetricNet improves performance by up to 10-30%, implying the robustness of MetricNet in reducing false positives for true loop closures. We conclude  
Figure 7: The PR curves of different networks on the (**a**) Gardens Point, (**b**) Nordland, and (**c**) St. Lucia datasets. that improving the discrimination between positive and negative samples in the Similarity Measurement Module is helpful under such conditions. To quantitatively verify the performance of MetricNet and other competitive SOTA algorithms, we compare their mean average precision (area under the PR curve), as shown in Table 3. Compared to the most related method, SAES, MetricNet outperforms it by up to 11.92%, 18.10%, and 5.33% on the Gardens Point, Nordland, and St. Lucia datasets, respectively.