EffoVPR: Effective Foundation Model Utilization for Visual Place Recognition
Authors: Issar Tzachor
Boaz Lerner
Matan Levy
Michael Green
Tal Berkovitz Shalev
Gavriel Habib
Dvir Samuel
Noam Korngut Zailer
Or Shimshi
Nir Darshan
Rami Ben-Ari
Venue: ICLR 2025 Poster (2025)
Keywords: Visual Place Recognition
Geo-Localization
Re-ranking
Vision Transformer
Self-Attention
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: A novel and effective approach for Visual Place Recognition, based on a foundation model that yields SoTA results

Abstract: The task of Visual Place Recognition (VPR) is to predict the location of a query image from a database of geo-tagged images. Recent studies in VPR have highlighted the significant advantage of employing pre-trained foundation models like DINOv2 for the VPR task. However, these models are often deemed inadequate for VPR without further fine-tuning on VPR-specific data.
In this paper, we present an effective approach to harness the potential of a foundation model for VPR. We show that features extracted from self-attention layers can act as a powerful re-ranker for VPR, even in a zero-shot setting. Our method not only outperforms previous zero-shot approaches but also introduces results competitive with several supervised methods.
We then show that a single-stage approach utilizing internal ViT layers for pooling can produce global features that achieve state-of-the-art performance, with impressive feature compactness down to 128D. Moreover, integrating our local foundation features for re-ranking further widens this performance gap. Our method also demonstrates exceptional robustness and generalization, setting new state-of-the-art performance, while handling challenging conditions such as occlusion, day-night transitions, and seasonal variations.

Published as a conference paper at ICLR 2025
EFFOVPR: EFFECTIVE FOUNDATION MODEL UTI- LIZATION FOR VISUAL PLACE RECOGNITION
Issar Tzachor', Boaz Lerner', Matan Levy?, Michael Green', Tal B Shalev', Gavriel Habib!
Dvir Samuel', Noam K Zailer', Or Shimshi', Nir Darshan', Rami Ben-Ari'
OriginAl, Israel The Hebrew University of Jerusalem, Israel issartgori wtteerrignnii..oommmm
ABSTRACT
The task of Visual Place Recognition (VPR) is to predict the location of query image from a database of geo-tagged images. Recent studies in VPR have high- lighted the significant advantage of employing pre-trained foundation models like DINOv2 for the VPR task. However, these models are often deemed inade- quate for VPR without further fine-tuning on VPR-specific data. In this paper. we present an effective approach to harness the potential of a foundation model for VPR. We show that features extracted from self-attention layers can act as a powerful re-ranker for VPR, even in zero-shot setting. Our method not only outperforms previous zero-shot approaches but also introduces results competi- tive with several supervised methods. We then show that single-stage approach utilizing internal ViT layers for pooling can produce global features that achieve state-of-the-au art performance, with impressive feature compactness down to 128D. Moreover, integrating our local foundation features for re-ranking further widens this performance gap. Our method also demonstrates exceptional robustness and generalization, setting new state-of-the-art performance, while handling challeng- ing conditions such as occlusion, day-night transitions, and seasonal variations.
INTRODUCTION
The task of Visual Place Recognition (VPR), also known Geo-Loocalization, aims to predict the place where a photo was taken relying solely on the visual information in the image. This typically done by an image retrieval approach (Arandjelovic et al., 2016; Hausler et al., 2021; Lu et al., 2024b; Berton et al., 2023; Lu et al., 2024a; Keetha et al., 2023) where a database of geo-tagged images is used, often referred as gallery. Real world data, including tagged VPR datasets, rely two major sources for images: 1) car street-view and 2) people personal cameras (commonly mobile phones) (Arandjelovic et al., 2016; Torii et al., 2015; Warburg et al., 2020; Berton et al., 2022a; Barbarani et al., 2023). As a result, images contain natural objects that are irrelevant and sometimes misleading for VPR task. Figure (top-row) shows an example, where people, vehicles, daylight, camera angles might differ between images.
Compact Features: Modern models commonly use a deep neural network to extract a so-called global feature (a.k.a descriptor) for the query and gallery images (Levy et al., 2023; Jia et al., 2021; Radford al., 2021; Li al., 2022b; Liu et al., 2021). This so called single-stage approach is then followed by a nearest neighbor search in the feature space, to retrieve the matching candidates from the gallery. In practice, global features of the entire gallery need be maintained in the RAM memory to enable fast retrieval. In large-scale and real-world scenarios, reducing the memory footprint for each image is crucial to ensure real-time applicability. Therefore, several works often promote compact features to achieve both accuracy and applicability (Zhu et al., 2023; Lu et al., 2024b; Berton et al., 2022a).
Following best practices in computer vision, VPR methods are often initial- ized with ImageNet pre-trained weights (e.g. (Zhu et al., 2023; Berton et al., 2022b)), followed by finetuning on VPR datasets, e.g. MSLS (Warburg et al., 2020), or trained from scratch (Wang al., 2022: Hausler et al., 2021; Arandjelovic et al., 2016). Recent advancements in VPR exploit the capabilities of foundation models (Keetha et al., 2023; Izquierdo & Civera, 2024; Lu et al., 2024b;a)
Foundation Models:

---

Published as a conference paper at ICLR 2025
such DINOv2 (Oquab et al., 2023), transformer (Vaswani et al., 2017) based model trained by self-supervised learning. Recent approaches (Lu et al., 2024b;a; Izquierdo & Civera, 2024) argue that using vanilla DINOv2 (without tine-tuning) is ineffective. They criticize this approach for fail- ing by capturing dynamic and irrelevant elements (pedestrians or vehicles). Some methods propose to address this issue by integrating learned adaptors into the foundation model architecture (Lu et al., 2024b;a) or changing the standard fine-tuning scheme with a unique pooling mechanism (Izquierdo & Civera, 2024), in order to facilitate the foundation model adaptation to the VPR task. In contrast, Anyloc (Keetha et al., 2023) suggested to use DINOv2 as a anneral-purpose feature representation at zero-shot for various localization tasks.
Pooling Methods: Typically, VPR methods (Izquierdo & Civera, 2024; Lu et al., 2024b;a; Berton al., 2023; Zhu et al., 2023; Berton et al., 2022a) adhere to conventional approach of initially extracting local features from pretrained model, then using pooling methods such as GeM (Raden- ović et al., 2018) or NetVLAD (Arandjelovic et al., 2016), to obtain global feature. Methods using VLAD (Jégou al., 2010), such as AnyLoc (Keetha et al., 2023), NetVLAD (Arandjelovic al., 2016) for aggregating local features necessitate learning dictionary for each specific gallery. These models tend to "overfit" to the particular gallery distribution, which hampers their generalization. Additionally, they often require fine-grained clustering, leading to high-dimensional feature vectors. Furthermore Shao al. (2023), highlighted GeM shortcomings, hyper-parameter sensitivity, spa- tial information loss, and convergence complications.
In this paper, we challenge the necessity of the external pooling layers, and suggest utilize the in- ternal ViT self-attention mechanism for an implicit pooling (Li et al., 2022a). Apparently, a straight- forward approach that trains with the class [CLS] token solely (instead of finetuning by all patch tokens) implicitly aggregates local features into a global representation. This approach removes the reliance on external specialized components or learned features, which often result in excessively large feature representations (Izquierdo & Civera, 2024; Lu et al., 2024a;b; Ali-bey et al., 2024). Moreover, this approach facilitates simple dimensionality reduction, allowing to achieve state-of- the-art results even with compact feature representations (see Figure I bottom row).
(a) Query
(b) Top-1 Result
VPR Partormance-Tokyy2477
VPR Performancee MSL5 ChalleCh
100
90
75
Recall@1 (N)
70-
Costacss CWP
M
X2 12Forme CVP1222
Eipenhaace ACCA
Seal HOUUC
60-
creasos cupap-
60-
SALAD CuPag21
OusiGGdedt-1224
CusiChoa 1:004
50-
Curotiess 1:35
0
2000
4003
5000
8000
2000
4000
6000
8000
Global Feature Dimension
Global Feature Dimersion
Figure 1: EffoVPR showcase. Top row: we show a challenging query image (a) and Effo EPR's top- 1 candidate (b), retrieved from a gallery of 2.8M geo-tagged images of SF-Occ. EffoVPR demon- strates high capability in handling transitional objects and large obstructions. Bottom row: We present Recall@) I performance of EffoVVR using global features against feature dimensionality. While the current leading methods achieve their performance using large features, Effo demon- strates top performance even with extremely compact feature size.
2

---

Published as a conference paper at ICLR 2025
Re-ranking: A popular strategy to improve the accuracy is conduct a two-stage search, with sub- sequent similarity search on the top-k ranked results. This step is commonly performed by matching local key-points and their corresponding descriptors. We build prior work highlighting the ro- bust properties of SSL.-trained models, e-g. their semantic and instance-level capabilities Oquab al. (2023); Caron al. (2021); Park et al. (2023); Keetha al. (2023), with Amir et al. (2022), exploring the roles of self-attention facets across various tasks. In this paper, we propose a match- ing method for VPR that uses specific internal ViT features, effective even in a zero-shot setting, outperforming current zero-shot methods and competitive with several fine-tuned VPR approaches. Fine-tuning the DINOv2 backbone for the VPR task further enhances the quality of the features and the overall performance of our matching method, achieving state-of-the-ard results.
We hereby present an Effective foundation based VPR method called EffoVPR that achieves su- perior single-stage performance without requiring additional components. Unlike more intricate DINOv2-based methods (Lu et al., 2024a) that rely on adapter components (Lu et al., 2024b) or special aggregation techniques (Izquierdo & Civera, 2024), EfRo delivers state-of-the-art results across multiple datasets while maintaining a compact feature representation. Additionally, ERR demonstrates strong generalization and robustness across diverse cities and landscapes, effectively handling challenges such as occlusions, time differences and seasonal changes as demonstrated in Figure I, which illustrates significant occlusion.
To summarize, our contributions in this work include:
1. We introduce a novel image matching method for VPR that relies on keypoint detection and descriptor extraction from the intermediate layers of a foundation model. This method outperforms previous zero-shot approaches while showing comparable results with trained VPR methods on few datasets.
2. We challenge the necessity of aggregation methods commonly used in various state-of- the-art methods, and propose simple yet effective alternative that significantly improves performance and feature compactness.
3. We propose effective approach that involves fine-tuning, followed by re-ranking the top candidates using our matching method. This approach significantly boosts performance, even with very small number of candidates.
4. We achieve state-of-the -arthe results on numerous benchmarks, demonstrating outstanding robustness across varying scenes e.g. day night, different seasons and occlusions, while achieving feature compactness that is often orders magnitude better than prior methods.
RELATED WORK
Traditional approaches utilized a single-stage approach for Visual Place Recognition (VPR), using SIFT (Lowe, 2004) SURF (Bay et al., 2006), or RootSIFT (Jin et al., 2021), focused on matching queries to gallery images through the use of image local feature matching. Two-stage methods (Hausler al., 2021; Wang et al., 2022; Zhu et al., 2023; Lu et al., 2024b) entail an initial ranking, based on global representation similarity, succeeded by re-ranking the top-K retrieved candidates in the second stage, utilizing local features.
First deep learning approaches for the VPR task used CNN (Arandjelovic et al., 2016; Radenović al., 2018: Jin Kim et al., 2017) that later replaced with Vision Transformers (ViT) (Wang al., 2022; Zhu et al., 2023; Keetha et al., 2023; Lu et al., 2024b;a; Izquierdo & Civera, 2024). TPR (Wang et al., 2022) and R2Former (Zhu al., 2023) suggested the application of transformers for VPR and adopted two-stage approach that included re-ranking. However. training from scratch or initialized on ImageNet, and lack of effective view-variability in training has restricted their performance as they introduce a significant challenge in selecting appropriate positive and hard negative images (Arandjelovic et al., 2016; Warburg et al., 2020; Zhu et al., 2023; Berton et al., 2022a). EigenPlaces (Berton et al., 2023) introduced novel training paradigm that organizes the training data into classes, with each class containing multiple viewpoints of the same scene. By employing classification loss rather than the conventional contrastive loss (used in (Lu et al., 2024b; lzquierdo & Civera, 2024; Lu et al., 2024a)), the resulting model demonstrated high resilience to varying viewpoints in testing. Hence, adopt similar strategy for training process.
3

---

Published as a conference paper at ICLR 2025
A handful of visual foundation models (VFMs) have recently emerged as the backbones for nu- merous tasks. VFMs like CLIP (Radford et al., 2021), DINOv2 (Oquab et al., 2023), SAM (Kir- illov et al., 2023) use VIT that are trained with distinct objectives, exhibiting unique characteristics for various downstream applications. In VPR, SALAD (Izquierdo & Civera, 2024) proposed fine- tuning the pre-trained DINOv2 model to improve its performance for VPR task. Their single-stage approach involves pooling local features from the oulput layer of DINOv2 replacing NetVLAD's soft-assignment to clusters by an optimal transport methodology. Notably, their most significant results are achieved with large features exceeding 8K dimension, which adversely impacts memory consumption. Using DINOv2 as their backbone, SelaVPR (Lu et al., 2024b) and CricaVPR (Lu et al., 2024a) took a different path and avoided fine-tuning of the model, proposing the incorpora- tion of a trainable adapters integrated into DINOv2 architecture. SelaVPR built above DINOv2's output tokens, while discarding the standard [CI.S] token. Their re-ranking strategy does not require spatial verification and is based on a newly learned patch level (local) features. CricaVPR instead, suggests learning features pyramid above DINOv2's output tokens, to learn a special viewpoint robust encoder. Both, SelaVPR and CricaVPR incorporate adapters in the DINOv2 architecture, using GeM pooling for feature aggregation, trained with the common contrastive loss.
Current VPR methods predominantly, derive a global representation by aggregating local features obtained from the output layer of trained backbone. This aggregation is often conducted using external modules for learnable pooling techniques, such as VL.D (Vector of Locally Aggregated Descriptors) (Jégou et al.. 2010). SPoC (Sum-Pooled Convolutional features) (Babenko & Lempit- sky, 2015), RMAC (Regional Maximum Activation of Convolutions) (Tolias et al., 2015), the known NetVLAD (Arandjelovic et al., 2016) or the nowadays widely-used GeM (Generalized Mean) pool- ing layer (Radenović et al., 2018; Shao et al., 2023), as utilized in various studies (Berton et al., 2021: Garg & Milford, 2021; Ge et al., 2020; Warburg et al., 2020; Hausler et al., 2021; Izquierdo & Civera, 2024; Lu et al., 2024b;a). However, NetVLAD is encumbered by high computational costs and dimensional complexities (with ~ 32K dimensional feature) (Berton et al., 2022a), while GeM encounter convergence issues and require hyperparameter tuning (Shao et al., 2023). Recently, more sophisticated approaches were suggested. SALAD (Izquierdo & Civera, 2024) applied opti- mal transport for local features aggregation, while BoQ (Ali-bey al., 2024) aggregated features by learning a Saa-of--uuriie along cross-attention. In this paper, we propose a method that, unlike previous approaches, utilizes the existing internal aggregation layer in DINOv2 without the need for additional pooling components or adaptors, achieving SoTA results with highly compact features.
DINOv2 deep features have been utilized for various tasks, e.g. part co-segmentation (Amir al., 2022), establishing semantic correspondences between image pairs (Zhang et al., 2023; Shtedritski et al., 2023; Zhang et al., 2024) or visual appearance transfer (Tumanyan et al., 2022), using internal features. Motivated by these studies we present a novel local matching method for VPR that detects keypoints and extracts their corresponding descriptors using a unique combination of VIT features.
Foundation models has shown strong applications even at zero-shot setting (without requiring any fine-tuning). In VPR, AnyLoc (Keetha al., 2023) proposed a single-stage solution that leveraged a DINOv2 pre-trained model alongside dense local (patch-level) features. By employing VLAD for feature pooling across multiple layers, AnyLoc approach produces an extremely large global representation of ~49K dimensions, subsequently reduced to 512D through PCA whitening, but in expense of lower performance. AnyLoc's s VLAD aggregation tends to fail in generalizing to queries with large time-gaps, day vs. night or of different season. In this paper, we propose two-stage zero- shot approach for VPR that first performs global ranking, followed by re-ranking. Moreover, our approach is adaptable fine-tuned models. Remarkably, after fine-tuning -uninun it attains ofee----theerr results while preserving compact features, which is essential given practical memory constraints.
METHOD
We start by outlining our model training strategy, followed by our local feature extraction method. Our approach stems from the insight that DINOy2 foundation model encapsulates robust features suitable for VPR tasks (Keetha et al., 2023). The core idea is to employ specific facets within the self-attention layer as reliable keypoint detectors and others robust descriptors, for re-ranking.
We explore ViT (Dosovitskiy et al., 2021) feature maps as local patch descriptors. In a ViT archi- tecture, an image is split into p non-overlapping patches which processed into tokens by linearly
4

---

Published as a conference paper at ICLR 2025
projecting each patch to d-dimensional space, and adding learned positional embeddings. An additional [CLS] token is inserted to capture global image properties. The set of tokens are then passed through L transformer encoder layers, each consists of normalization layers, Multihead Self- Attention modules, and MIP blocks. In ViT, each patch is directly associated with a set of features, Key, Query, Value that can be used patch descriptors. We utilize the self-attention matrices at layer 1, as { K, Q, V) (x++1) l)xd, with p indicating the number of patches resulting p 1 tokens (number of patches in the image plus one added global token) and standing for the embedding dimension. The self-attention function at layer / is then given by:
KTQ Vd
Attention( @, K, V) = Softmax
VV
Note that for sake of brevity we omit the index I. We indicate the key feature of the [CLS] token layer / by kiots E K. Each image patch therefore is directly associated with the set of features (go k;, helptt1 including its query, key and value at certain layer E [1, LJ. respectively.
In the global search stage, the aim is to perform efficient search across vast corpus of images in a gallery. To this end, a shared embedding space for query and gallery images is typically utilized identify the most similar images and rank them accordingly. In the re-ranking stage, the top-K nearest neighbors (candidates) are further refined by evaluating the mutual similarity between their local features and those of the query image. Throughout both stages, we use a VIT (Dosovitskiy et al., 2021) encoder our backbone model. Our EffoVPP retrieval method is presented in Figure 2.
Extract Features VITAL
Rerank
K Candidates
For each carccilate::
Global
Mutual NN
VITIL
Layer PAPTEE
LaNE
Reranking Features
K
Q O-KEEs
Count:
Keep 9, TT
CRida
Sort by H-r
Figure 2: An overview of VPR. Left: During inference, we identify the nearest neighbors of the query by using the [CLS] token as the global representation for each image (Ug). For the second re-ranking stage, we extract (dashed-line) intermediate features, and utilize S, the partial attention map, for keypoini selection (with a predefined threshold Ti), while employing the Value facet V as the corresponding keypoints descriptors. Right: Lastly. we re-rank the top-K candidates from the first stage based on the count of strongly connected mutual nearest neighbors (MNN) with a score exceeding a predefined threshold (denoted as T2).
3.1 TRAINING STRATEGY
Our training strategy employs a classification loss applied on the output [CLS] token. Follow- ing EigenPlaces (Berton al., 2023), we partition the map into cells measuring 15 15 meters each, where cells define classes. Each class contains images capturing the same location from var- ious viewpoints, enhancing the model's ability to accurately recognize locations despite substantial changes in the captured view angle. To enforee class discrimination while ensuring viewpoint in- variance, we utilize Large Margin Cosine Loss (CosFace) (Berton et al., 2022a; Wang et al., 2018).
We initialize our VIT (Dosovitskiy et al., 2021) backbone with pre-trained DINOv2 weights (Oquab et al., 2023; Darcet et al., 2023). To retain the rich visual representations learned during pre- tre-training, while adapting the model for the VPR task, Note that due to the inherent design of the ViT architecture, which incorporates self-attention mechanism (Eq. (1)), the global feature is implicitly trained to aggregate local features without the
fine-tune only the final layers of our backbone.
5

---

Published as a conference paper at ICLR 2025
need of additional specialized components. For learning more compact global feature we simply add a linear layer on top of the output [CLS] token.
3.2 INFERENCE STRATEGY
Post-training, we extract the global feature of image from the [CLS] output token of the penultimate classification layer. We then retrieve the K nearest neighbors of the query's 's global feature.
Global ranking stage:
We start by extracting patch local features from each candidate among the top-k retrieved images in the previous global stage, and re-rank the candidates based on these features. Local features are derived from the intermediate layer L of ViT, utilizing the self-attention matri- ces, thus their computation is integrated into the process of computing global features and does not necessitate recalculation. For keypoints descriptors, we find that Vi2 matrix in Eq. (1) most effec- tively captures the "instance" properties of the keypoints, making it the most promising facet for the task at hand (see analysis in Tab. S3 in Appendix). Then, for keypoint selection, we lever- age the model's internal prioritization, to identify discriminative features and extract the attention map: S := Softmax((i kets) E I2, which represents the attention of cach Value feature with the global [CTSS) image representation key-token kets- We therefore select subset V of the values V S {vI; Fpp Vi as the image's 's local features, based the score S: V == {o; S; Ti) } for a predefined threshold Tj. Note that the number of selected local features may vary between images.
Re-ranking stage:
Next, given the query's local features V and candidate's 's V", we calculate the pairs of mutual nearest neighbors (MNN) between V and V by the number of feature pairs that are the nearest neighbor of each other. Our observation revealed that applying threshold to the MNN scores enhances the model's 's resilience to clutter and directs its attention to the pertinent key-points for accurate matching. We therefore count only the pairs with cos- similarity higher than a predefined threshold T2. We formulate that process in Equation (2):
MNN(V, V²) == ((,))))) v5)) E XVXVV uy == NN((. vf) and v/j = NN v6), of
T2 (2)
Finally, the re-ranking stage concludes by sorting the top-K candidates based on their MNN counts. Note that our re-ranking strategy, which matches local features, does not require any additional learning, optimization, or spatial verification. The thresholds are established once and remain fixed across all test sets (20 different scenarios).
Zero-shot:
For the first stage ranking we use the [CLS] token from vanilla-DINOv2. The results are then refined by re-ranking, while employing our suggested features V with MNN in Eq. (2).
Our design parameters include the two key values of Ti. Tz. Threshold Ti is used to select strong keypoints for image matching, typically focusing on more significant areas. The threshold T2 facil- itates the selection of strong matching pairs, filtering out weak correspondences that could result in false image matches. For detailed ablation study on these thresholds refer to Appendix B.
EVALUATION
In this section, we compare our approach with several SoTA VPR methods following the common VPR Benchmarks (Berton et al., 2022c), We propose, a single-stage (EffoVPR-G) and two-stage, that includes re-ranker (PR-)) approaches with backbone trained on the publicly available SF-XL (Berton et al., 2022a) dataset containing streetview of San Francisco. For more implementa- tion details see Appendix. We then test EPf on a large number of diverse datasets (20), includ- ing e.g. Pitts30k (Arandjelovic et al., 2016), Tokyo24/7 (Torii et al., 2015), -aleh--ll (Warburg et al., 2020) Nordland (Sünderhauf et al., 2013) and more, exhibiting a wide variety of conditions, including different cities, day/night images, and scasonal changes. Note that MSLS Challenge (Warburg et al., 2020) is a hold-out set whose labels are not released, but researchers sub- mit the predictions to the challenge server. Further details on the benchmarks in the Appendix.
Datasets with gallery made from street-view images and with largest viewpoint variance, include Tokyo 24/7 (Torii et al., 2015) and SF-XL (Wang et al., 2018; Barbarani et al., 2023), where the query images are collected from a smartphone, usually from sidewalks. Most datasets are from urban footage, with the main exception being Nordland (Sünderhauf al., 2013), which is collection of

---

Published as a conference paper at ICLR 2025
photos taken across different seasons with camera mounted on train. Some datasets present various degrees of day-to-night changes, namely MSLS (Warburg et al., 2020), Tokyo 24/7 (Torii et al., 2015), SF-XL (Berton al., 2022a) SVOX-Night (Berton et al., 2021). AmsterTime (Yildiz et al., 2022) contains grayscale historical queries and modern-time RGB gallery images, making the only dataset with large-scale time variations of multiple decades.
We follow common evaluation metric used in previous works e.g. (Berton et al., 2022c; 2023; Zhu al., 2023; Lu al., 2024b;a; Arandjelovic et al., 2016) and use 25 meters radius the threshold for correct localization and report Recall K metrics for K=1,5,10. For Nordland we evaluate 10 frames as the common evaluation protocol used in (Berton al., 2022c; Hausler et al., 2021). For more comprehensive description of all 20 datasets and implementation details see Appendix.
4.1 ZERO-SHOT PERFORMANCE
We present the performance of our zero-shot method (EffoVPR-ZS) in Table I compared to two zero-shot al- ternatives, the recently published AnyLoc, and DINOv2 global feature (using the output [CLS] token), with- out finctuning, where EffoVPR-ZS re-ranks its top-100 global retrieved candidates. The results show that method significantly improves over the baseline and superior AnyLoc. Note the significant gap for more challenging scenarios of Tokyo24/7 and Nordland exhibiting day vs. night and seasonal variations. AnyLoc tends to fail in these challeg- ning scenarios
Table 1: Comparison on Zero Shot with R
Pinc30k Tokyo24/7 MSL.5 A-V.S-Val Val Nordllad
DINOV2
78.1
62.2
47.7
33.0
Anyloc
87.7
60.6
68.7
16.1
BROVPR-ZS
89.4
70.3
57.9
VLAD aggregation learned (in unsupervised manner) on the gallery can generalize well to challenging out-of-distribur di-sstiibuution queries. In Figure 3a we compare our zero-shot approach with several methods that have used VPR datasets for training. Although Effo VPR-ZS was not trained on VPR task, it still achieves comparable results to the trained methods on three popular datasets. This success can be attributed the robust features in DINOv2, specifically those selected from the V facet, combined with our mutual-NN matching and scoring. Figure 3b demonstrates this by showcasing scenario where the original attention mistakenly focusing on an advertisement placed in front of a building. However, method successfully identifies relevant key-point, on the building itself, enabling correct image matching (even though a different advertisement is displayed on the gallery image).
Zero-Shat vs. Taned
08
0
SALSS
21 light
1501047
Attention map
Query image
Top-1 result
(a) Zero-shot vs. Trained
(b) EffoVPR zero-shot visualization
Figure 3: EPR zero-shot. (a) Comparison of EIR-VPR-ZS with other VPR trained methods. Our zero-shot approach shows comparable results. (b) Zero-shot success despite existing dynamic and irrelevant objects and strong visual change. Matching keypoints are indicated by colored lines. Although the pre-trained DINOv2 initially has its strongest attention on the distracting temporal advertisement, Effo VPR effectively identifies correct keypoints for successful matching.
4.2 COMPARISON WITH STATE OE-TTE-ART
In this section, compare single stage (EffoVPR-G) and two-stage methods (EffoVPR-R) with previous state-of-the the-the-art art including the recent works of (Lu et al., 2024a; Izquierdo & Civera, 2024; Lu et al., 2024b; Ali-bey et al., 2024). SeRaVPR and R2Former were trained combination of Pitts30k and MSLS while CricaVPR, SALAD and BoQ were trained on GSV-Cities, and Cosplace and FigenPlaces, on SF-XL (similar ours). We show in Table 2 the global retrieval results (without
7

---

Published as a conference paper at ICLR 2025
re-ranking) with Recall on five different benchmarks. EffoVPR-G - achieves SoTA performance on three out of five datasets, while being ranked second on the other two. This highlights the effec- tiveness of the single global representation learned by method. Notably, it achieves +2.9% on Tokyy24/7, +2.8% on the challenging Nordland dataset that exhibits extreme seasonal changes, and +3.2% the hold-out MSL.S-calleenge dataset. We further demonstrate the performance of global feature learning with reduced dimensions of 256D and even 128D, significantly decreasing the memory footprint and enabling efficient searches within a considerably larger gallery. The find- ings indicate only marginal degradation in performance with lower-dimensional features, while achieving parity with the SALAD on Tokyo24/7 using 128D, compared 8.448D, feature size (a 66-fold reduction in dimensionality). Figure illustrates this quality Tokyo24/7 and the hold-out MSh.Seehalle showing our top performing results even with 128D feature size. Results on more datasets can be found in the Appendix. The strong impact of our finetuning process including the last five layers, visualized Figure
Table 2: Comparison with single stage method Recall@) performance. Two-stage methods are marked with 9, and present lst-stage performance (for fair comparison). The best results are highlighted bold and the second underlined. We present results from VPR with three different feature dimensions.
Method
Dim
Pitts30k
Tokyo24/7
MSLS-val val
MSLS-chall
Nordland
CosPlace
512
88.4
81.9
82.8
61.4
66.5
MixVPR
4096
91.5
86.7
88.2
64.0
58.4
R2Former
256
76.3
45.7
79.3
56.2
50.9
EigenPlaces
2048
92.5
93.0
89.1
67.4
71.2
SelaVPR
1024
81.9
87.7
69.6
CricaVPR
4096
94.9
93.0
90.0
69.0
90.7
SALAD
8448
92.4
94.6
92.2
75.0
76.0
BoQ
16384
91.1
73.3
EfRoVPR G
1024
94.8
97.5
90.9
78.2
93.5
EffoVP--G
256
93.8
95.9
90.4
75.6
79.7
Effo G
128
92.6
94.6
88.2
73.8
70.4
(a) Before training
(b) After training
Figure 4: Attention map visualization: pre- pra-trd DINOv2 focuses on irrelevant foreground objects e.g. vehicles. Whereas attentions of EfRoVPR after training are shifted to scene layout and building structures such as cables and windows.
Next, we showcase the comprehensive performance our two-stage approach (EffoVPR-R) in Ta- ble 3. EPR- achieves top performance across all datasets, taking the second place only on Pitts30K-R@. 1, with very close result. Note that CricaVPR reports using Pitts30k as a validation set, which may have contributed the improved results on this dataset. However, EffoVPR-R demonstrates notable improvements, particularly evident in Tokyo24/7, where it achieves remark- able increase in R@| from 94.6% to 98.7% and MSLS MSLS-challenge (from to 75.0% to 79.0%), These results underscore the generalization capability of our approach, demonstrating its resilience in handling significant variations between query and gallery images, such as viewpoint discrepancies (as in Pitts30k) and changes in illumination (in Tokyo24/7), across diverse range of locations. Following the common practice, we report Effo VPR-R re- re-ranking performance over top- 100 candi- dates retrieved in the first-stage (K 100), however achieve SoTA R @ results even with low
8

---

Published as a conference paper at ICLR 2025
Table 3:
Comparison state-of-the-art methods on four benchmarks. The bests results are high- lighted in bold and the second is underlined. Two-stage methods are marked with f.
P0t
Tokyo24/7
MSLS-val
MSLS-challenge
Method
R@
@@5
R
10
R@
R
R
10
R@
R@5
R 10
R@
R@5 R@ 10
NetVLAD
81.9
91.2
93.7
60.6
68.9
74.6
53.1
66.5
71.1
35.1
47.4
51.7
SFRS
89.4
94.7
95.9
81.0
88.3
92.4
69.2
41.6
$2.0
56.3
Patch-NetVLAN
88.7
94.5
95.9
86.0
88.6
90.5
79.5
86.2
87.7
48.1
57.6
60.5
CosPlace
88.4 94.5
95.7
81.9
90.2
92.7
82.8
89.7
92.0
61.4
72.0
76.6
MixVPR
91.5 95.5
96.4
86.7
92.1
94.0
94.3
64.0
75.9
R2Former
91.1 95.2
96.3
88.6
91.4
91.7
89.7
95.0
96.2
73.0
85.9
88.8
EigenPlaces
92.5 96.8
97.6
93.0
96.2
97.5
89.1
93.8
95.0
67.4
77.1
81.7
SelaVPR
92.8 96.8
97.7
94.0
96.8
97.5
90.8
96.4
97.2
73.5
87.5
90.6
CricaVPR
94.9 97.3
98.2
93.0
97.5
98.1
90.0
95.4
96.4
69.0
82.1
85.7
SALAD
92.4
96.3
97.4
94.6
97.5
97.8
92.2
96.2
97.0
75.0
88.8
91.3
BoQ
92.5
96.4
97.4
91.1
95.9
96.5
90.7
94.7
95.8
73.3
82.9
86.2
EffoVPR-R
93.9
97.4
98.5
98.7
98.7
98.7
92.8
97.2
97.4
79.0
89.0
91.6
number of candidates (from K 5 onwards, see Tab. S4 in Appx). Note that EffoVPR matching method is highly expedient, with an average processing time of just millisecond per match.
In Figure 5, we present failure case of our zero-shot approach, which is resolved after fine-tuning. In this instance, both the query and gallery contain visually identical vehicle (an SF cable car), which leads to incorrect matching. Although such instances are rare general case in context of pedestrians or vehicles in the images, where the objects are commonly not identical, this example highlights a limitation of our zero-shot approach.
(a) Zero-shot
After training
Figure 5: Zero-shot vs trained: failure case of our zero-shot approach, resolved after training.
Finally, we demonstrate the efficacy our approach in the most challenging VPR benchmark sce- narios by conducting experiments six demanding datasets: Nordland (Sünderhauf et al., 2013), which includes extensive seasonal changes; AmsterTime Time (Yildiz et al., 2022), spanning over an extended time period: SF-Occlusion (Barbarani al., 2023), that features queries with significant field-of-view obstructions: SF-Night (Barbarani et al., 2023), with severe illumination changes; and SVOX (Berton al., 2021), with extreme weather and illumination variations. The results, detailed in Table 4, underscore the significant superiority of our method over previous approaches across these datasets. EffoVPR-P shows improvements of +4.3%, +0.8%. +7.9%, and +15%. +2% on Nordland, AmsterTime. SF-Occlusion, SF-Night, and SVOX-Night respectively and comparable re- sults on SVOX- -ain. This demonstrates the high versatility our model, which can handle extreme variations even when trained without seasonal day-to-night changes. Figure shows an examples of this case. We attribute this robustness primarily to the combination our training method and specific re-ranking strategy over the DINOv2 model. We conduct an extensive ablation study various hyperparameters and aspects of our approach in the Appendix.
Figure qualitatively highlights the superior performance of our method. While other methods fail in challenging scenarios, such as viewpoint changes, seasonal variations, illumination differences, and severe occlusions, VPR demonstrates high robustness against these challenges.

---

Published as a conference paper at ICLR 2025
Table 4: Comparison @R@) SoTA methods on more challenging datasets.
Amster Time
SF-XL Occlusion
SF-XL Night
SVOX Night
SVOX
Method
Nordland
Rain
EigenPlaces
71.2
48.9
32.9
23.6
58.9
90.0
SelaVPR
85.2
55.2
35.5
38.4
89.4
94.7
CricaVPR
64.7
42.1
35.4
95.0
SALAD
76.0
58.8
51.3
46.6
95.4
98.5
BoQ
83.4
51.3
36.8
26.8
86.5
95.5
ERfo
95.0
65.5
59.2
61.6
97.4
98.3
Query
ErfovPR(Ours)
SALAD
CricaVPR
SclaVPR
EiganPlaces
Changes
Seasonal Changes
Illuminatoon Difference
Severe Occlusion
Figure 6: Qualitative comparison to SoTA Methods with challenging examples.
ABLATION STUDY
We conduct a comprehensive ablation study using two different datasets to analyze the key parame- ters of our EffoVPR method. Key findings are summarized below, with detailed results provided in the appendix. Analyzing the different VIT self-attention facets, we show that Value facet (V) offers the most effective local features for VPR re-ranking (Table S3). Moreover, the selection of the layer for feature extraction is important, with the n layer delivering the best performance, whereas us- ing the final layer leads to decline results (Table S1). In Table S4 we show that our re-ranking stage remains effective even with as few as five candidates, achieving SoTA results, highlighting the method's efficiency and effectiveness. Fine-tuning only the last five layers of the backbone net- work proves optimal (Figure S1), Although EffoVPR achieves sfaee-of-the-ar results with lower resolutions, increasing image resolution up to 504px slightly improves performance (Table S7).
SUMMARY
In this paper, we present both single-stage and two-stage approaches for VPR that leverage the internal layers of a foundation model for pooling and re-ranking. We challenge the necessity of recent complex architectural modifications, particularly the reliance on external pooling methods that lead to extended feature lengths in pursuit of high performance. By utilizing the model's 's existing pooling layers, we demonstrate rlate-of-the-aa results with impressive feature compactness. We introduce novel matching method for re- re-ranking based self-attention facets of foundation model, which proves effective even in a zero-shot setting.
Demonstrating strong robustness to various appearance changes while maintaining compact fea- tures, Effo VPR offers promising solution for tackling the VPR task real-world, large-scale applications.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Amar Ali-bey, Brahim Chaib-draa, and Philippe Giguère. GSV-Cities: Toward appropriate super- vised visual place recognition. Neurocomputing. 513:194-203, 2022. 16
Amar Ali-bey, Brahim Chaib-draa, and Philippe Giguère. Mix VPR: feature mixing for visual place recognition. In WACV, pp. 2998-3007, 2023. 16
Amar Ali-bey, Brahim Chaib-draa, and Philippe Giguère. BoQ: A place is worth bag of learnable queries. Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR), pp. 17794-17803, June 2024. 4,7
Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. Deep ViT features dense visual descriptors. ECCVW, 2022. 3,
Relja Aranjjelovic, Petr Gronat, Akihiko Torii, Tomas Pajdla, and Josef Sivic. NetVLAD: CNN architecture for weakly supervised place recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5297-5307, 2016. 1, 3, 4, 6, 7, 15, 16
Artem Babenko and Victor Lempitsky. Aggregating local deep features for image retrieval. In Proceedings of the IEEE international conference on computer vision, pp. 1269-1277, 2015. 4
Giovanni Barbarani, Mohamad Mostafa, Hajali Bayramov, Gabriele Trivigno, Gabriele Berton, Carlo Masone, and Barbara Caputo. Are local features all you need for cross-domain visual place recognition? In CVPRW, pp. 6155-6165, June 2023. 1, 6, 9, 15
Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. SURF: Speeded up robust features. In ECCV, pp. 404-17.. Springer, 2006. 3
Gabriele Berton, Carlo Masone, and Barbara Caputo. Rethinking visual geo-localization for large-
scale applications. In CVPR (CVPR), pp. 4878 4878-4888, June 2022a. 1, 2, 3, 4, 5, 6, 7, 15, 16,
21
Gabriele Berton, Riccardo Mereu, Gabriele Trivigno, Carlo Masone, Gabriela Csurka, Torsten Sat- tler, and Barbara Caputo. Deep visual geo-localization benchmark. In CVPR, pp. 5396 5407, 2022b.
Gabriele Berton, Riccardo Mereu, Gabriele Trivigno, Carlo Masone, Gabriela Csurka, Torsten Sat- tler, and Barbara Caputo. Deep visual geo-localization benchmark. CVPR, June 2022c. 6, 7, 15, 16, 17
Gabriele Berton, Gabriele Trivigno, Barbara Caputo, and Carlo Masone. Eigenplaces: Training viewpoint robust models for visual place recognition. In Proceedings of the IEEE/CVF Interna- tional Conference Computer Vision (ICCV), pp. 11080-11090, October 2023. 1, 3. 5.
21
Gabriele Moreno Berton, Valerio Paolicelli, Carlo Masone, and Barbara Caputo. Adaptive-attentive geolocalization from few queries: A hybrid approach. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 2918-2927, 2021. 4, 9, 9,9,16 16
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski. and Armand Joulin. Emerging properties self-supervised vision transformers. In ICCV, pp. 9650- 9660, 2021.
David M Chen, Georges Baatz, Kevin Köser, Sam Tsai, Ramakrishna Vedantham, Timo Pylvänäinen, Kimmo Roimela, Xin Chen, Jeff Bach, Mare Pollefeys, et al. City-scale landmark identification on mobile devices. In CVPR 2011, pp. 737-744. IEEE, 2011.
Zetao Chen, Lingqiao Liu, Inkyu Sa, Zongyuan Ge, and Margarita Chli. Learning context flexible attention model for long-term visual place recognition. IEEE Robotics and Automation Letters, (4):4015-4022, 2018. 15
Mark Cummins and Paul Newman. Highly scalable appearance-only Slam-fab-map 2.0. In Robotics: Science and systems, volume 5, pp. 17. Seattle, USA, 2009. 15

---

Published as a conference paper at ICLR 2025
Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers, 2023. 5, 21
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICIR, 2021. 4,5
Sourav Garg and Michael Milford. Segnet: Learning descriptors for sequence- based hierarchical place recognition. IEEE Robotics and Automation Letters, 6(3):4305 4312, 2021. 4
Yixiao Ge, Haibo Wang, Feng Zhu, Rui Zhao, and Hongsheng Li. Self-supei tne-grained region similarities for large-scale image localization. Computer Eision-ECCi 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part IV 16, pp. 369-386. Springer, 2020.
Stephen Hausler, Sourav Garg, Ming Xu, Michael Milford, and Tobias Fischer. Patch-NetVLAN Multi-scale fusion of locally-global descriptors for place recognition. In CVPR, pp. 14141-14152, 2021. 3, 4,
Sergio Izquierdo and Javier Civera. Optimal transport aggregation for visual place recognition. In Accepted CVPR, June 2024. 2, 4, 7, 16, 18
Hervé Jégou, Matthijs Douze, Cordelia Schmid, and Patrick Pérez. Aggregating local descriptors into compact image representation. In CVPR, pp. 3304-3311, IEEE, 2010. 4
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hicu Pham, Quoc V. Le, Yun- Hsuan Sung, Zhen Li, and Tom Duerig. Scaling Up Visual and Vision-Language Representa- tion Learning With Noisy Text Supervision. In Marina Meila and Tong Zhang (eds.), ICML, volume 139 of Proceedings of Machine Learning Research, pp. 4904 4916. PMLR, 2021. URL http:/ http:///ooeedddeee mlr. pwwss/vvlss /e//jja220 jiallb. html.
Yuhe Jin, Dmytro Mishkin, Anastasiia Mishchuk, Jiri Matas, Pascal Fua, Kwang Moo Yi, and Ed- uard Trulls. Image matching across wide baselines: From paper to practice. International Journal of Computer Vision, 129(2):517-547, 2021. 3
Hyo Jin Kim, Enrique Dunn, and Jan-Michael Frahm. Learned contextual feature reweighting for image geo-localization. In CVPR, pp. 2136-2145, 2017. 3
Nikhil Keetha, Avneesh Mishra, Jay Karhade, Krishna Murthy Jatavallabhula, Sebastian Scherer, Madhava Krishna, and Sourav Garg. AnyLoc: towards universal visual place recognition. IEEE Robotics and Automation Letters, 2023. 1, 2, 4,
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan- Yen Lo, et al. Segment anything. In CVPR, pp. 4015 4026, 2023. 4
Matan Levy, Rami Ben-Ari, Nir Darshan, and Dani Lischinski. Chatting makes perfect: Chat- based image retrieval. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 61437-61449, Curran Associates, Inc., 2023.
Jiahao Li, Greg Shakhnarovich, and Raymond A Yeh. Adapting clip for phrase localization without further training. arXiv preprint vrXiv:2244.03647 2022a. 2
Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. Hoi. BLIP: Bootstrapping Language-
Image Pre-training for Unified Vision-Language Understanding and Generation. In ICML, pp. 12888-12900, 2022b.
Zheyuan Liu, Cristian Rodriguez-Opazo Damien Teney, and Stephen Gould. Image Retrieval on Real-life Images with Pre-traines trained Vision-and- and-Language Models. In ICCV, pp. 2105-2114, 2021. doi: 10.1 C21109/CC994992.22222 202.2222....2.0000 00013..
12

---

Published as a conference paper at ICLR 2025
David G Lowe. Distinctive image features from ccale-invariant keypoints. International journal of computer vision, 60:91-110, 2004.
Feng Lu, Xiangyuan Lan, Lijun Zhang, Dongmei Jiang, Yaowei Wang, and Chun Yuan. Cricavpr: Cross -msgee correlation-awann aware representation learning for visual place recognition. In Accepted to CVPR, June 2024a. 1, 2, 4, 7, 16, 21
Feng Lu, Lijun Zhang, Xiangyuan Lan, Shuting Dong, Yaowei Wang, and Chun Yuan. Towards seamless adaptation of pre-trainn pre-trained models for visual place recognition. In ICLR, 2024b. 1, 2, 4, 7, 16, 17, 21
Will Maddern, Geoffrey Pascoe, Chris Linegar, and Paul Newman. 1 year, 1000 km: The oxford robotcar dataset. The International Journal of Robotics Research, 36(1):3-15, 2017. 16
Michael J Milford and Gordon F Wyeth. Mapping suburb with single camera using biologically inspired slam system. IEEE Transactions on Robotics, 1038- 1053, 2008. 16
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo. Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojeiech Galuba, Russell Howes, Po-Yao Huang, Shang- Shang-Wen Li, Ishan Misra, Michael G. Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jégou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: learning robust visual features without supervision. CoRR, abs/2304.07193, 2023. 2, 3, 4, 5, 21
Namuk Park, Wonjae Kim, Bycongho Heo, Tackyung Kim, and Sangdoo Yun. What do self- supervised vision transformers learn? ICLR, 2023. 3
Filip Radenović, Giorgos Tolias, and Ondřej Chum. Fine-tuning cmn image retrieval with no human annotation. IEEE transactions on pattern analysis and machine intelligence, 4157):1655,1668,
2018. 2, 3,4
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. In Marina Meila and Tong Zhang (eds.), ICML, 2021. URL http::/pprceedinng/ mll ptt.spesss/.../..///em
v139/radfordee htmll. 1,4
Shihao Shao, Kaifeng Chen, Arjun Karpur, Qinghua Cui, André Araujo, and Bingyi Cao. Global features are all you need for image retrieval and reranking. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 11036-11046, 2023. 2,
Aleksandar Shtedritski, Andrea Vedaldi, and Christian Rupprecht. Learning universal semantic cor- respondences with no supervision and automatic data curation. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 933-943, 2023. 4
Niko Sünderhauf, Peer Neubert, and Peter Protzel. Are there yet? challenging seqslam 3000 km journey across all four seasons. In Proc. of workshop on long-term autonomy, IEEE international conference on robotics and automation (ICRA), pp. 2013. Citeseer, 2013. 6, 6,9, 15
Giorgos Tolias, Ronan Sicre, and Hervé Jégou. Particular object retrieval with integral max-pooling of cnn activations. arXiv preprini arXiv: 1.05887 2015. 4
Akihiko Torii, Relja Arandjelovic, Josef Sivic, Masatoshi Okutomi, and Tomas Pajdla. 24/7 place recognition by view synthesis. In Proceedingy of the IEEE conference on computer vision and pattern recognition, pp. 1808-1817, 2015. 1, 6, 7, 16
Narek Tumanyan, Omer Bar- Bar-Tal, Shai Bagon, and Tali Dekel. Splicing vit features for semantic appearance transfer. In CVPR, pp. 10748-10757, 2022. 4
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pp. 3998-6008, 2017. 2
13

---

Published as a conference paper at ICLR 2025
Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2265-5274, 2018.
Ruotong Wang, Yanqing Shen, Weiliang Zuo, Sanping Zhou, and Nanning Zheng. Trans TransVPR: transformer-based place recognition with multi-level attention aggregation. In CVPR, pp. 13648- 13657, 2022. 3,3
Frederik Warburg, Soren Hauberg. Manuel Lopez-Antequera, Pau Gargallo, Yubin Kuang, and Javier Civera. Mapillary street-level sequences: A dataset for lifelong place recognition. In CVPR, pp. 2626-2635, 2020. 1,3, 3,3,4, 4, 15, 16
Burak Yildiz, Seyran Khademi, Ronald Maria Siebes, and Jan Van Gemert, Amstertime: visual place recognition benchmark dataset for severe domain shift. In 2022 26th International Confer- ence on Pattern Recognition (ICPR), pp. 2749-2755. IEEE, 2022. 7, 9, 15
Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Polania Cabrera, Varun Jampani, Deqing Sun, and Ming- Hsuan Yang. A Tale of Two Features: Stable Diffusion Complements DINO for Zero- Shot Semantic Correspondence. arXiv preprint arxiv:3305 2023. 4
Junyi Zhang, Charles Herrmann, Junhwa Hur, Eric Chen, Varun Jampani, Deqing Sun, and Ming- Hsuan Yang. Telling left from right: Identifying geometry- semantic correspondence. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024. 4
Sijie Zhu, Linjie Yang, Chen Chen, Mubarak Shah, Xiaohui Shen, and Heng Wang. Formre::: unified retrieval and reranking transformer for place recognition. In CVPR, pp. 19370-19380,
2023. 1, 2, 3, 7, 16, 17
14

---

Published as a conference paper at ICLR 2025
Appendix
A DATASETS
We evaluate EffoVPR performance across a large number of datasets, to underscore its top- performance in variable scenarios and cities. Following prior work, we have used (Berton et al., 2022c) open-source code for downloading and organizing datasets, to ensure maximum repro- ducibility. the following we shortly describe each of the datasets.
A.1 DATASETS SUMMARY
AmsterTime (Yildiz et al., 2022) consists of 1, 232 image pairs from Amsterdam, Holland, exhibit- ing long-term changes. The queries are historical grayscale images, where for cach query there is reference of modern-day photo which represents the same place. The pairs curated by human experts, and provide multiple challenges over different viewpoints and cameras, color vs grayscale and long-term changes.
Eynsham (Cummins Newman, 2009) is a collection a car street-view camera, capturing photos around the same route of Oxford countryside twice. The grayscale images divided to 23, 935 queries and 23, 935 gallery.
Mapillary Street-Level Sequences (MSLS) (Warburg al., 2020) is image and sequence baseed based VPR dataset. The dataset consists of more than 1.6M geo-tagged images collected during over seven years from 30 cities, in urban, suburban, and natural environments. There 3 non-overlap subsets a training set, validation (MSLS-val), and withheld test (MSLS-challenge), MSLS-val and MSLS- challecee provide various challenges, including viewpoint variations, long-term changes, and illumination and seasonal changes.
Nordland (Sünderhauf al., 2013) was collected by mounted camera on the top of riding train in the Norwegian countryside, presenting rural and natural scenes. The data collected over the same route across four seasons, providing seasonal and illumination variability. Following (Berton al., 2022c; Sünderhauf et al., 2013) we use the post-processed versions of winter queries and summer as database, determining correct localization by retrieval of an image that is in less than 10 frames away. This dataset consists of 27, 592 query images and 27, 592 gallery images
Pittsburgh30k (Arandjelovic et al., 2016) is collected from Google Street View 360 panoramas of downtown Pittsburgh, split into multiple images. Ensuring queries and gallery were taken in different years, it provides splits training set, validation and test. Pitts30k-test consists 10k gallery images and 6816 queries. Pitts250k consists of 8280 queries including these of Pitts3kk, and its gallery size is 83, 952.
San Francisco Landmark (SF-R) (Chen et al., 2011) is dataset from downtown San Francisco, which provides viewpoint variations. It presents a collection of 598 of smartphone camera queries and gallery of 1, 046, 587 images.
San Francisco eXtra Large (SF-XL) (Berton et al., 2022a; Barbarani et al., 2023) is an enormous dataset covering the whole city San Francisco. it consists of training set, which includes also raw 360 panoramas, a small validation set of 7, 983 queries and 8, 015 gallery images, and a test gallery of 2, 805, 840 images.
There are four sets of queries:
SF-XL-v/ (Berton et al., 2022a) consists of 1, 000 queries curated from Flickr, and provides view- point and camera variations, illumination changes and even some occlusions.
SF-XL-v12 (Berton al., 2022a) is the queries of San Francisco Landmark (SF-R).
SF-XL-Night (Barbarani et al., 2023) is collection of 466 Flickr images of night scenes from San- Francisco. It provides viewpoint variations and very-challenging illumination changes. SF-XL-Occlusion (Barbarani et al., 2023) is collection of 76 Flick images from the city of San Francisco, which suffers from severe occlusions, mostly by vehicles and crowd.
SPED (Chen et al., 2018) is collection of surveillance cameras images consists of 607 pairs of queries and gallery, captured accros time. It provides challenging viewpoint with seasonal and illumination changes.
15

---

Published as a conference paper at ICLR 2025
St Lucia (Milford & Wyeth, 2008) is collection of nine videos of car-m muu-modtedd camera from the St Lucia suburb of Brisbane. Following (Berton et al., 2022c) open-source code, we select the first and last videos queries and database, and sample one frame every 5 meters of driving. The gallery consists of 1, 549 images and there are 1464 query images.
SVOX (Berton et al., 2021) is dataset which presents multiple weather conditions VPR challenge. consists of 17, 166 gallery images, of the city of Oxford. The queries were collected from the Oxford RobotCar dataset (Maddern et al., 2017), providing multiple weather conditions queries sets, such as night (823 queries), overcast (872 queries), rainy (937 queries), snowy (870 queries) and sunny (854 queries).
Tokyo 24/7 (Torii et al., 2015) is dataset from downtown Tokyo, which provides viewpoint changes and challenging illumination variations. It consists of gallery of 75, 984 images, and a collection of 315 smartphone camera queries from 185 places. Each place is portrayed by three photos one taken during the day, one at sunset and one at night.
A.2 TRAIN DATASET
Following VPR classification methods, Eigneplaces and CosPlace, train on SF-XL (Berton al., 2022a) while other studies (Ali-bey et al., 2022; Izquierdo & Civera, 2024; Lu et al., 2024a; Ali-bey et al., 2023) train GSV-Cities (Ali-bey al., 2022) combinations of Pittsburgh30k (Arand- jelovic et al., 2016) and MSLS (Warburg et al., 2020; Arandjelovic et al., 2016; Lu et al., 2024b; Zhu al., 2023), including large mixture of different cities around the world (introducing higher variability). Note that similar to EigenPlaces, our approach is designed for training on panoramas with heading information, and requires slicing them for lateral and frontal views, which cannot be applied other training datasets.
B ADDITIONAL ABLATION STUDIES
We conduct extensive experiments on two different datasets to ablate over key-components of our EffoVPR method.
Re-ranking features:
We explore various configurations for selecting features the re-ranking stage. Our initial focus is on the choice of the layer from which features are extracted. 'able S1 demonstrates that extracting features from the n layer yields the most significant enhancement in overall performance. Gen- erally, employing re-ranking with any layer, except of the last layer, improves results compared to omitting re-ranking entirely (i.e., relying solely on the global feature from the first stage). Subse- quently, upon extracting the Q K, V components from the chosen layer, we find that the Value set (V) represents the most effective local features for re- re-ranking, as detailed in Table S3. We ablate in Table S2 the impact of two thresholds, the Attention Map threshold T2 and the threshold on the countable local feature matching score threshold T2.
Table $ 1: Ablation study on the choice of the layer for the re-ranking stage. We find the n layer to be the optimal for re-ranking feature extraction. Notably, the last layer is ineffective and downgrades global performance. Results (in %) are the R 1.
Dataset
Global
n-5
n-4
n-3
n-2
n-
n
MSLS-val
90.9
90.3
90.9
92.0
92.3
92.8
88.2
Tokyo-24/7
97.5
98.1
98.1
98.1
98.7
98.7
97.1
16

---

Published as a conference paper at ICLR 2025
Table S 2: Ablation study on the impact of the thresholds. Results (in %) are the R @ 1. Ti the Attention Map threshold and T2 is the threshold on the countable local features matching score
Table 3: Ablation on the choice of the lo- cal features. Results (in %) are the R R@) Query, Key and Value are respectively Q.K.V Equation (1)
Tokyo24/7
MSLS-val
Query
Key
Value
no thr.
95.9
86.4
Tokyo24/7
96.5
96.8
98.7
T
97.1
91.5
MSLS-val
89.7
90.1
92.8
+72
98.7
92.8
Number of Candidates Re-rank: The second re-ranking stage is applied to the top-K candidates retrieved during the global stage. Although common choice in literature is K 100 (e.g. (Lu al., 2024b; Zhu et al., 2023; Berton et al., 2022c)), we explore different choices of K, as detailed in Table S4. We achieve SoTA results even with K 5. It is important to note that in some cases, an increase in can introduce greater number of "distractor" candidates, potentially leading to a decrease in performance. However, EffoVPR SoTA performance is consistent for all tested K's
Table S 4: Re-ranking ablation. K' indicates re-ranking over top- K results. We achieve SoTA results even with = 5. Bold values indicate SoTA results.
Top-K
Pitts30k
Tokyo24/7
MSLS val
Nordland
SF-XL-Occ.
SF-XL-Night
SPED
K=5
94.2
97.8
92.4
95.3
59.2
61.6
93.4
K=10
94.2
98.1
92.2
95.3
59.2
61.2
92.9
K=15
94.1
98.4
92.3
95.3
60.5
60.3
93.1
K=20
94.0
98.4
92.4
95.3
59.2
60.3
93.1
K=50
93.9
98.7
92.7
95.2
57.9
60.9
93.2
K=100
93.9
98.7
92.8
95.0
59.2
61.2
93.2
Choice of Trainable Layers: Figure SI presents a few different sets trainable layers in our back- bone model. We find the vanilla fine-tuning of the entire model, end-to-end, that includes all layers, drastically harms the performance of EffoVPR-G in the global descriptors level. We attribute this decline to the fact that the DINOv2 backbone trained significantly larger datasets compared to those typically used in VPR. Subsequently, we establish that training only the last five layers represents a "sweet-spot", yielding peak performance. Both increasing decreasing the number of trainable layers from this configuration leads to lower results.
Performance number fireduned Giobal teuture
30
80
MSLS Way02/4/2
28:0 Shot layer 2-byerss 31 alayyess Blayers lavers ayers layyrrs alllayers
Layer Fineuund
Fig. S I: Ablation on the number of trainable layers
T. and T2 thresholds. Ti parameter shown in Fig.2 sets a threshold for selecting strong keypoints, while T2 sets threshold for identifying strong matching pairs. In practice, keypoints with scores (s:
17

---

Published as a conference paper at ICLR 2025
in Fig. 2) below Ti filtered out, and matching pairs with scores below T2 discarded. We set T1 and T2 values according to a validation set, and ablate the sensitivity for each value independently (while fixing the other) in Tables S5, S6.
Table S 5: Ablation study on the value of Ti (T2 = 0.65). Results (in %) are the R@1 1.
Table S 6: Ablation study on the value of T2 (T1 = 0.05). Results (in %) are the R@
Ti
0.00
0.05
0.10
T2
0.50
0.65
0.80
Tokyo24/7
97.8
98.7
98.4
Tokyo24/7
97.5
98.7
98.4
MSLS-val
89.1
92.8
92.4
MSLS-val
91.8
92.8
91.5
Image Resolution. Different methods employed varying image resolutions. For instance, SALAD (Izquierdo & Civera, 2024) used resolution of 322px. During inference, we used images with resolution of 504px enable more patches (of 14 14), particularly for re-ranking that uses local keypoint detection and matching. The increased number of patches primarily contributes to improved performance on more challenging datasets. However, image resolution impacts both in- ference time and memory footprint in the feed-forward path. Table S7 presents an ablation study on performance and feature extraction efficiency using intermediate resolutions of 224px and 322px across five datasets. EffoVPR-R consistently maintains satt--ofthh-asl performance across various resolutions. Higher resolution enhances performance. Both GPU memory footprint and feature extraction time increase as the resolution rises.
Table 7: Ablation study on image resolution. Results (in %) are the R@ 1 of Effo
GPU
Latency
Tokyo 24/7
MSLS val
Nord land
SF-XL Night
Amster Time
Res.
(GB)
(s)
224px
2.33
0.018
98.4
91.1
92.7
57.5
65.1
322px
2.38
0.028
98.4
92.8
94.3
61.2
65.3
504px
2.57
0.067
98.7
92.8
95.0
61.6
65.5
C ADDITIONAL RESULTS
C.I PERFORMANCE VS. FEATURE COMPACTNESS
In Figure S2 we present performance comparison of our global feature (EffoVPR-G) versus feature dimensionality for more datasets. While the current leading methods achieve their performance using large features, EPR demonstrates high performance even with an extremely compact feature size.
Parfomarce Plb32
Parformance NSL5
Porfaraanos Nordand
Recaloa 196)
02.5
On Naw PYRPS
Roc M
35.8
425
Egen COND3
018
Ka de Comio4
dud settooosaaa 204
3000
4000
1993
8008
2000
4006
5000
8000
2000
4000
5000
eaturre Dimension
Gicbal Tweture Dimerriion
Global Feature Dimensson
Fig. S 2: Recall@1 performance of EffoVPR-G global feature versus feature dimensionality for more datasets.
18

---

Published as a conference paper at ICLR 2025
C.2 COMPARISON OF EFFOVPR MODEL VARIANTS
We further summarize the results of different variations of EffoVPR over five datasets in Table S8.
Method
Pitts3kk
Tokyo24/7
MSLS-val
MSLS-chall.
Nordland
Effo 89.4
90.8
70.3
41.3
57.9
EffoVPR-G
94.8
97.5
90.9
78.2
93.5
EffoVPR-R
93.9
98.7
92.8
79.0
95.0
Table S 8: Recall @
performance for different variations of EfRo
D VISUALIZATIONS
D.I ADDITIONAL ZERO-SHIOT VISUALIZATIONS
Figure we show more visualizations Eff-VP method. While the attention map of pre-trained DINOv2 doesn't 't focus discriminative VPR elements. EffoVPR is able to fill the gap in zero-shot with local features matching. the first row The pre-trained attention-map is mainly focused on temporal traffic signs and a far ad and almost not attend the building, and in the second row it is mainly focused on an insignificant back of a traffic sign. However Effo VPR method finds multiple local matches the right geo-tagged image in the gallery.
(a) Attention Map
(b) Query Image
(c) EffoVPR Matching
Fig. S: Additional EffoVPR-ZS zero- shot visualizations.
D.2 ADDITIONAL RE-RANKING VISUALIZATIONS
Figure S4 exhibits EffoVPR-R R local features matching invariability to highly challenging scenes with top-1 results. From the top left to the bottom right to camera rotation, nature scene, color variance across time (building renovation), tree matching, challenging day-time change with hardly noticed electric cables matching, night to day significant change.
D.3 FAILURE CASES
Figure S5 presents several failure cases where EffoVPR-R produces incorrect predictions. In the first row, the left, cable obscures most of the scene, and the model erroneously matches its caster-style roof with a gallery image featuring the same cable car. On the right, car and a tree obscure most of the scene, leaving insufficient features for accurate matching. As result, the
19

---

Published as a conference paper at ICLR 2025
Fig. 4: EffoVPR-R top-1 matching visualizations. For each pair matching, the left image is the query and the right is the top-1 result.
model incorrectly associates the tree and the white stripes of a pedestrian crossing with the wrong location. In the bottom row, a significant time-gap change is evident between the query and gallery images, making it challenging to identify the house of interest in the gallery, as it is depicted during period of construction. Figure S6 illustrates a more common scenario observed our results, where the model produces visually accurate prediction that is nevertheless classified incorrect. This occurs because the distance between the gallery image and the query image exceeds 25 meters, the threshold defined for the VPR task by previous studies.
(a) Query Image
(b) Matching
(c) Ground Truth
Fig. S:5: EfR- failure cases stem from obscured views, lack of significant features and time gaps.. In the first row for each pair matching, the left image is the query and the right is the top-1 result.
20

---

Published as a conference paper at ICLR 2025
Fig. 6: Effo failure cases due to distance from the camera. For each pair matching, the left image is the query and the right is the top-1 result.
E IMPLEMENTATION DETAILS
We use ViT-L/14 as the backbone, initialized with pre-trained weights of DINOv2 with registers (Oquab et al., 2023; Darcet et al., 2023). We only train the last five layers of the backbone, which appeared to be most beneficial. We employ EigenPlaces's (Berton et al., 2023) group and class partitioning with its default hyper-parameters, and both lateral and frontal views, on the publicly available SF-XL street-view panoramas dataset (Berton et al., 2022a). We set an AdamW optimizer the backbone, and an Adam optimizer to the classification heads, both with constant learning rate of 1 x 10-5, We train EffoVPR with a batch size of 16, for 25 epochs, on a single NVIDIA- A100 node. We otherwise follow EigenPlaces training recipe. We choose the best epoch by SF-XL validation set, measuring Recall global ranking performance. Given that ViT is independent of the image input size (provided can be segmented into x 14 patches), we evaluated using images sized 504 x 504, but trained on 224 x 224 images to expedite training. For benchmarking EffoVPR-G of the global feature, we report nearest-neighbors performance on normalized output class token. In the re-ranking stage extract the V self-attention facet from layer n - 1 (with being the output layer), measure cosine similarity, we filter the features by class attention map with threshold Ti 0.05, and count only mutual neighots with score above the threshold To = 0.65.
E. ADDITIONAL INFORMATION
1. ZERO-SHOT
In evaluating AnyLoc (Keetha et al., 2023), we tackle the significant memory requirements of its VLAD pooling algorithm by implementing an online clustering scheme. We observed that their rec- ommendation for layer 31 outperformed layer 23. In our zero-shot evaluation, we assess Effo ZS method by extracting the V features from layer trieved from the first-stage global [CLS] feature. In this framework, our performance is constrained by the first-stage Recall @ 100, achieving rates of 99.2%, 96.8%, 81.5%, and 78.1% on Pitts30k, Tokyo24/7, MSLS-VA. Val, and Nordland, respectively.
to re-rank the top-100 candidates
E.1.2 BENCHMARKING
Generally, for consistent benchmarking, we adhere to (Berton et al., 2022c). In addition, we report the results of other methods in accordance with the evaluation choices of Sela (Lu et al., 2024b) and Cri (Lu et al., 2024a), including the specific versions trained models utilized. For the recent state-of-the-art methods SeR, CricaVPR and SALAD, we provide results from the original publications whenever available. When such results are not directly available, we utilize their code and published weights Specifically for VeR, which has two sets of weights (trained on Pitts30k and MSLS), we report the best west-performing for each dataset. For BoQ, we evaluated by the model provided weights across all datasets, due to inconsistency in reported results in the paper, and project page and code.
E.2 OTHER
We evaluate VPR matching runtime and local features dimension by averaging matching func- tion runtime and local features number the Tokyo 24/7 dataset.
21

---

Published as a conference paper at ICLR 2025
F MEMORY FOOTPRINT AND RUNTIME
One important aspect of the memory footprint VPR stems from the size of the model's features. This directly affects the gallery size in RAM, influencing the efficiency of fast search and retrieval. In Table S9, we present a detailed breakdown of the memory footprint for our global ranking stage, comparing other methods. This includes an analysis based on collection of million im- ages, demonstrated using the real-world SF-XL gallery, which covers the city of San Francisco with 2.8 million images. Effo VPR-G achieves high performance even with 228-iimessional features, requiring as little 500MB per million images, and 1.34GB for the city of San-Francisco.
Method
Features Dim.
1M images (GB)
SF-XL Gallery (GB)
R2Former!
256
0.98
2.68
SelaVPR
1024
3.81
10.7
CricaVPR
4096
15.26
42.81
SALAD
8448
31.47
88.3
BoQ
16384
61.04
171.25
EffoVPR-Gi
1024
3.81
10.7
EffoVPR-G
256
0.98
2.68
EffoVPR-G 128
0.49
1.34
Table S 9: Feature dimension and memory usage for different methods.
In Table S10, we evaluate memory footprint and runtime of our re-ranking stage comparing to R2Former and SelaVPR as reported in their corresponding paper.
Local Features Dim.
Mem. Footprint
Latency (s)
Tokyo24/7 R@1
MSLS-Val Val
Method
(GB)
R@1
R2Former
500 x (128 + 3) 61 x x 128
0.025
0.202
88.6
89.7
SclaVPR
0.182
0.085
94.0
90.8
EffoVPR-R
649 x 1024
0.254
0.035
98.7
92.8
Table 10: Local features dimension, memory footprint, latency and performance for different methods. Different methods utilized different GPUs for runtime evaluation: R2Former runtime was measured using RTX A5000, SelaVPR with RTX 3090, and EffoVPR with A100.
G ADDITIONAL QUANTITATIVE RESULTS
To ensure comprehensiveness, the following Table S11 presents the complete results for datasets that were only partially presented in the main paper, well for some datasets that were previously omitted. Our method, EffoVPR, demonstrates SoTA performance on the majority of these datasets, and remains competitive with the SoTA on others.
22

---

Published as a conference paper at ICLR 2025
Table 11: Comparison SoTA on more datasets
SPED
SF-K
SF-XL-v]
SI-XL-v2
SI-XL-OsS
Method
R@1
R@5
R@:10
R@1
R@8
R@:0
R@
R@5
R@ 10
R@
R@5
R 10
R@1
R@5
R@
EigenPlaces
70.2
83.5
87.5
89.5
94.3
95.3
84.1
89.1
90.7
90.8
95.7
96.7
32.9
48.7
52.6
SelaVPR
88.6
95.1
97.2
88.5
92.0
93.0
74.9
80.7
82.1
89.3
95.7
96.3
35.5
47.4
$5.3
CricaVR
SALAD
92.1
96.2
96.5
92.3
95.7
96.8
88.6
93.5
94.4
94,8
97.3
98.3
51.3
65.8
68.4
BoQ
94.4
96.1
95.3
83.7
87.8
89.2
92.8
95.0
96.3
36.8
47.4
52.6
EffoVPR
93.1
97.9
98.4
93.0
96.0
96.3
95.5
98.1
98.3
94.5
97.8
98.2
59.2
68.4
73.7
SP-XI-Night
Amster Time
SVOX
SVOX Night
SVOX Overcast
Method
R:610
EigenPlaces
23.6
30.7
14.5
48.8
69.55
76.0
98.0
99.0
163.2
58.9
76.9
82.0
91.1
97.8
98.3
SelaVPR
38.4
50.9
55.4
78.0
97.2
89.4
95.5
96.6
CricaPR
35.4
48.3
64.7
82.5
87.9
97.8
99.2
99.3
86.3
95.3
96.6
96.7
99.0
99.0
SALAD
46.6
59.0
62.2
58.8
78.9
84.2
98.2
99.3
99.4
95.4
99.3
99.4
98.3
99.3
99.3
BoQ
26.8
40.6
51.3
71.7
78.1
98.8
99.4
99.5
86.5
94.4
96.4
97.9
99.2
99.3
ENoVPR
61.6
73.4
77.0
65.5
87.2
90.7
98.7
99.5
99.6
97.4
99.5
99.5
98.4
99.3
99.7
SVOX Rain
SVOX Snow
SVOX Sun
Sr. Lacin
Eynsham R
Method
EigenPlaces
90.0
96.4
98.0
93.1
97.6
98.1
86.4
95.0
96.4
99.6
99.9
100.0
90.7
94,4
95.4
CricaVPR
94.8
98.5
98.7
96.0
99.2
99.2
91.8
98.1
98.8
99.9
99.9
99.9
91.6
95.0
95.8
95.1
95.9
BoQ
95.5
99.3
99.7
98.5
99.5
99.7
96.1
98.8
98.9
100.0
100.0
100.0
91.2
94.9
95.9
23


Reviews:
[{"review_id": "Mk3Jw8a2o0", "reviewer": "Reviewer_ZQCU", "summary": "This paper presents an effective approach for visual place recognition (VPR) using a foundation model (DINOv2). The features extracted from the self-attention layers are used as re-ranker for VPR. Experiments on several datasets show the proposed method is robust and effective.", "questions": "Please see the weakness section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed method is easy to follow. It leverages the foundation model’s feature representation capability to improve the effectiveness of image retrieval and re-ranking. \n\n2. The proposed method outperforms some existing approaches on several VPR benchmarks.", "weaknesses": "1. Since the proposed method mainly leverages the strong representation power of a foundation model (DINOv2), the technical novelty of the proposed method in VPR is somewhat incremental. \n2. Certain design choices, such as using the V_l matrix for keypoint descriptors, lack clear motivation and justification, particularly in explaining why such choices are effective.\n3. Although the proposed method outperforms several existing methods, the improvement could come solely from using a stronger foundation model feature representation. Therefore, the comparison may not be fair. \n4. The computational complexity and memory footprint of the proposed method should be discussed and compared with other existing methods. \n5. It would be good to also show some failure cases and provide analysis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hdLjtXiS9L", "reviewer": "Reviewer_2roT", "summary": "This paper proposes a new method, called EffoVPR, to effectively use vision foundation models for visual place recognition. This method leverages the output class token of the DINOv2 model to generate global features to retrieve candidates. It also uses ViT internal attention matrices to obtain local features for re-ranking these candidates. EffoVPR achieves good zero-shot performance. After training on the SF-XL dataset with the EigenPlaces framework, it outperforms previous SOTA methods on several VPR datasets.", "questions": "see weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThe methods proposed in the paper are generally novel.\n2.\tThe experiments provided in the paper are sufficient. The author provides the results of other SOTA methods on datasets such as SF-XL. As far as I know, completing such evaluation experiments requires a lot of time and effort.\n3.\tThe performance of the proposed method is really good. It achieves excellent results even with compact global features.", "weaknesses": "1.\tThe methodological contribution of this paper is a little weak. Its training strategy follows the EigenPlaces work. The global feature is the class token (following a linear layer) of the ViT model. The local matching in the re-ranking process is based on the mutual nearest neighbors searching, and it also already exists. I think the authors should state that the previous SelaVPR work also does not require spatial verification in re-ranking. The biggest contribution of this work seems to be the use of the internal self-attention matrices to yield selected local features, which is meaningful but maybe not enough.\n2.\tThe proposed method is trained on SF-XL, which is a very large dataset and time-consuming to train. If training on other datasets, such as GSV-Cities or even MSLS/Pitts30k. Can it still achieve such good results? If the proposed method cannot extend to other training sets, it is better to have an appropriate explanation in the paper.\n3.\tThe proposed method uses 224×224-pixel images for training, while 504×504-pixel images for evaluation. Using the DINOv2-large model to process such high-resolution images is time-consuming and memory-consuming. Although the paper gives results using low-resolution images on MSLS and Tokyo247, I think it is also necessary to give a comparison of GPU memory usage, and better to provide results on more datasets (e.g., Nordland and SF-XL-Night).\n\nOverall, the strengths of this paper outweigh its weaknesses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AzJ5vnhkgK", "reviewer": "Reviewer_P6DW", "summary": "The paper introduces a method that uses DINOv2 foundation model to address the Visual Place Recognition task. This is achieved by retrieving the top-K most similar candidates in the first stage by measuring the similarity of (finetuned) DINOv2’s [CLS] tokens between the query and the candidate images. In the second stage, the proposed approach re-ranks the retrieved candidates using local features computed using the Queries, Keys and Values from the self-attention module of an intermediate DINOv2 layer. The method outperforms other state-of-the-art zero-shot approaches, as well as supervised methods when the last layers of the DINOv2 network are finetuned. Moreover, the proposed method achieves results competitive with the supervised methods even when using only the first stage of the method.", "questions": "- Q1: The paper shows that EffoVPR-R achieves SOTA results even with K=5. I am curious, how much does this change for the zero-shot setting. I.e., when the last layers of DINOv2 are not finetuned, is it also sufficient to use K=5, or would K need to be increased?\n\n- Q2: It should be clearly stated that the ablations in the appendix we done using the EffoVPR-R. Would it be possible to confirm this?\n\n- Q3: The re-ranking stage requires storing local features for the re-ranked images. It would be good to add discussion and analysis of memory requirements for the proposed approach and in particular the memory requirements to store the local features for spatial re-reranking.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- S1: The paper is clear and easy to read.\n\n- S2: Simple, yet effective and robust method that improves the performance over the DINOv2 baseline and other state-of-the-art methods, while reducing the required dimensionality of the used features.\n\n- S3: The method finetunes DINOv2 to achieve better performance but is still effective even in a zero-shot setting.", "weaknesses": "- W1 Even though the results of the three proposed variants of the method (EffoVPR-ZS, EffoVPR-G and EffoVPR-R) are provided, the paper is missing their comparison in a single table, to directly see how large is the effect of finetuning and re-ranking. It would be good to see this comparison in a single table to understand (and compare) the direct benefits of the different components of the proposed approach. \n\n- W2 The notation for Queries, Keys and Values and corresponding local features would be more readable if the layer index would be consistently used always as superscript or always as subscript ($V_l$ versus $v^l_i$). Alternatively, since the Queries, Keys and Values are always taken from the same layer, maybe the layer index could be completely omitted for better readability.\n\n- W3 What are the failure cases and limitations of the main presented method, EffoVPR-R? The paper contains only failure cases for the zero-shot setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dDddvp1vlJ", "reviewer": "Reviewer_KXCa", "summary": "The paper presents a model for visual place recognition, specifically aimed at achieving robust zero-shot retrieval performance. The model uses a Vision Transformer (ViT) with DINOV2 weights as its backbone to identify nearest neighbors based on global feature similarity, followed by a mutual nearest neighbor (NN) re-ranking strategy to pinpoint the closest reference image using local features. For training, the Eigenplace paradigm is applied to promote view invariance in the ViT. Only the final layers of the backbone are fine-tuned, with the rest of the model kept frozen. Fine-tuning was performed on the SF-XL dataset, and testing was conducted on several benchmarks: Pitts30k, Tokyo24/7, MSLS-val, and MSLS-challenge, as well as more challenging datasets, including Nordland, Amsterdam Time, SF-XL Occlusion, SF-XL Night, SVOX Night, and SVOX Rain.  For contribution, EffoVPR proposed a novel reranking architecture that does not require training and works well even with a small number of candidates.  The paper also claims to demonstrate the lack of need for aggregate methods to capture local features", "questions": "The paper mentions time evaluation, but only shows time evaluation on the first step and not the reranking.  From the look of the model, the time of reranking doesn’t seem to be particularly time efficient.  How fast is the reranking?  \n\nThe paper mentioned memory efficiency and reduced memory footprint.  Quantitatively, how much memory consumption does the modal overall take compared to the other models? \n\nOn the ablation on T1 and T2, table only shows the result with different T1 and different T2, but didn’t specify what the other filter value is (ex. On table S5, When showing performance when T1 equals 0.05 and 0.10, it didn’t specify what T2 is when evaluating).  What is the value of T2 when evaluating for table S5 and what is the value of T1 when evaluating for table S6? \n\nHow well does the model do on the Pittsburgh 250k dataset? \n\nAre there experimental results of EffoVPR trained on other datasets other than SFXL? \n\nIn the Eigenplaces paper which the training process of EffoVPR seems to be heavily based on, the backbone was completely trained.  How well would the model perform if every layer of the backbone was trained? \n\nPaper mentions that only the “final layers” are trained.  What are the final layers that are trained? \n\nPaper calls itself a “foundational model”, which should be able to be used for multiple tasks, yet only same view matching capability is shown in the paper.  Can the model be used for other tasks like cross view matching or detection? \n\nCould the global descriptor be used also during the reranking phase to further filter out the nearest neighbors, perhaps to only neighbors that belong in the same predicted cell? \n\nHow well does the model perform with anyloc as backbone rather than DINOV2?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Performance: \nEffoVPR achieves SOTA performance by performing better than previous recent SOTA models such as SALAD and EigenPlaces.  In particular, EffoVPR performs significantly better on the more challenging datasets  \nEffoVPR is able to fine tune on cities in SF XL but then perform very well on Nordland which is mainly rural and natural scenes taken from a train. \nEffoVPR has very impressive zero shot performance compared to other zero shot methods and some trained models, showing effectiveness of reranking model \nEffoVPR with only the first stage also performs very well  \nArchitecture: \nThe architecture for EffoVPR is intuitive and simple to understand \nEfficiency: \nEffoVPR utilizing only reranking during evaluation makes training efficient and fast, as the reranking does not need to be trained \nExperimentation: \nPaper presents very detailed ablation studies on the choice of layer for reranking, the different thresholds, which layer to choose reranking features for, etc.   \nPapers shows the model being performed on a wide range of datasets.", "weaknesses": "Performance: \nModel does not perform better than other models by a significant margin on the less challenging datasets.  However, the less challenging datasets are already inferred very well by the other datasets so there’s not as much room for improvement.   \nPaper is not entirely clear how much the reranking contributes to performance compared to DINOV2 \nNovelty: \nIn contribution, the paper claimed “We introduce a novel image matching method for VPR that relies on keypoint detection and descriptor extraction from the intermediate layers of a foundation model”.  However, R2Former also uses filtered local features from intermediate layers from the VIT backbone. Perhaps need to change wording to reflect true level of novelty. \n\nThe paper mentions reduced memory footprint, but only shows dimension of global feature and not other aspects such as parameter size (ex. R2Former uses VIT-S whereas EffoVPR uses VIT-L but is not reflected anywhere on paper) \n\nTraining: \nT1, T2, along with which layer to get feature map all being fixed hyperparameters means there’s more manual tuning involved to achieve optimal results", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)