SuperVLAD: Compact and Robust Image Descriptors for Visual Place Recognition
Authors: Feng Lu
Xinyao Zhang
Canming Ye
Shuting Dong
Lijun Zhang
Xiangyuan Lan
Chun Yuan
Venue: NeurIPS 2024 poster (2024)
Keywords: Visual Place Recognition
Image Descriptors
VLAD
SuperVLAD
1-Cluster VLAD
Primary area: machine_vision

Abstract: Visual place recognition (VPR) is an essential task for multiple applications such as augmented reality and robot localization. Over the past decade, mainstream methods in the VPR area have been to use feature representation based on global aggregation, as exemplified by NetVLAD. These features are suitable for large-scale VPR and robust against viewpoint changes. However, the VLAD-based aggregation methods usually learn a large number of (e.g., 64) clusters and their corresponding cluster centers, which directly leads to a high dimension of the yielded global features. More importantly, when there is a domain gap between the data in training and inference, the cluster centers determined on the training set are usually improper for inference, resulting in a performance drop. To this end, we first attempt to improve NetVLAD by removing the cluster center and setting only a small number of (e.g., only 4) clusters. The proposed method not only simplifies NetVLAD but also enhances the generalizability across different domains. We name this method SuperVLAD. In addition, by introducing ghost clusters that will not be retained in the final output, we further propose a very low-dimensional 1-Cluster VLAD descriptor, which has the same dimension as the output of GeM pooling but performs notably better. Experimental results suggest that, when paired with a transformer-based backbone, our SuperVLAD shows better domain generalization performance than NetVLAD with significantly fewer parameters. The proposed method also surpasses state-of-the-art methods with lower feature dimensions on several benchmark datasets. The code is available at https://github.com/lu-feng/SuperVLAD.

SuperVLAD: Compact and Robust Image Descriptors for Visual Place Recognition
Feng Lu Lu 1.2
Xinyao Zhang 1
Canming Ye
Shuting Dong" 1.2
Lijun Zhang²
Xiangyuan Lan Chun Yuan
1 TTinghua Shenzhen International Graduate School, Tsinghua University
"Pengeheng Laboratory *CIGIT, Chinese Academy Sciences
Paazoou Laboratory (Huangpu)
(112200ails,,,unccce (asaaccez)..sinnuuaa edu. cn lanxy0pcl.ac.
Abstract
Visual place recognition (VPR) an essential task for multiple applications such augmented reality and robot localization. Over the past decade, mainstream methods in the VPR area have been to use feature representation based on global aggregation, exemplified by NetVLAD, These features suitable for large- scale VPR and robust against viewpoint changes. However, the VLAD-based aggregation methods usually learn large number of (e.g., 64) clusters and their corresponding cluster centers, which directly leads to high dimension of the yielded global features. More importantly, when there is a domain gap between the data in training and inference, the cluster centers determined on the training set are usually improper for inference, resulting in performance drop. To this end, we first attempt to improve NetVLAD by removing the cluster center and setting only a small number of (e.g.. only 4) clusters. The proposed method not only simplifies NetVLAD but also enhances the generalizability across different domains. We name this method SuperVLAD. In addition, by introducing ghost clusters that will not be retained in the final output, we further propose very low- dimensional 1-Cluster VLAD descriptor, which has the same dimension the output of GeM pooling but performs notably better. Experimental results suggest that, when paired with a transformer-based backbone, our SuperVLAD shows better domain generalization performance than NetVLAD with significantly fewer parameters. The proposed method also surpasses state-of-the-and methods with lower feature dimensions on several benchmark datasets. The code is available at foom//uueen//SuurrrLAA
1
Introduction
Visual Place Recognition (VPR) is a task that aims at quickly estimating the coarse geographical location of place image (i.e.. query) by retrieving the most similar images from geo-tagged database [10.4] It has garnered significant attention in both computer vision and robotics communities, driven by its wide applications in augmented reality 42] and robot localization [59], etc. However, VPR still faces some challenges: On the one hand, images captured at the same place may vary dramatically due the changes in conditions (e.g., lighting and weather) and viewpoints. On the other hand, multiple different places can show high similarity, which may lead to perceptual aliasing [36. quite challenging address these issues simultaneously, especially for VPR methods that use compact global descriptors to represent the place images.
"Equal contribution.
t Correspondiig authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a)
(b)
(d)
Figure 1: VLAD and SuperVLAD similarity measures under different clusterings (Voronoi cells). Orange triangles and blue diamonds depict local descriptors from two different images. In (a) and (b), orange and blue arrows are the sum of residuals (for VLAD). With different training data distributions, the different cluster centers are yielded, causing opposite similarity results using cosine similarity (or normalized 1.2 distance). Compared to VLAD, our SuperV as shown in (c) and (d), simply calculates the distance between the weighted sum of local features directly, freeing from the impact of cluster centers. Thus, only minor changes will occur when dealing with two different distributions.
The VPR is typically implemented with the image retrieval approaches 48 The place images are first described with global features and then the nearest neighbor search is performed over the database to get the matched place images of the query. The global descriptors are usually obtained by applying the aggregation methods, such as Bag of Words [ and Vector of Locally Aggregated Descriptors (VLAD) 228 35 31 533, to process local (or patch) features. Although some hierarchical (le.two-stage) VPR methods improve robustness under challenging environments by cross- moss-mat local features in the query and candidate images for re-ranking, they incur considerable additional computational latency and memory footprint. Taking step back, the first stage of these hierarchical methods still requires the use of global features to retrieve candidates. As such, representing place images with compact and robust global descriptors is always the most essential and important issue in VPR.
In the VPR methods based on neural networks, the NetVLAD 4 and GeM pooling 48 are the most commonly used aggregation/pooling methods. The former aggregates the feature maps extracted by neural networks with trainable generalized VLAD layer. Its main difference from the vanilla VLAD 28 lies in its differentiable soft assignment and better performance. However, still constructs the output vector by summing the residuals of descriptors assigned to each cluster (like VLAD), which is calculated by subtracting the learned cluster center from the local descriptors. When training data distributions differ, the learned cluster centers also vary. Even if we use a well-genere -eneerazle backbone to extract the local features of two images, the differences in cluster centers can lead to completely different results in the similarity of the output global features (see Fig. D. In other words, when there a domain gap between the training and inference data, the cluster center learned on the training set is usually not suitable for inference, which can cause performance drops (5). Besides, the global descriptor yielded by NetVLAD has a large dimension. Even with dimensionality reduction methods like PCA (or other learnable linear projections), the dimension often remains notably higher compared to the output of pooling methods. As a result, some VPR approaches requiring very cow-dimensional features use GeM pooling to yield global descriptors at the cost of certain performance.
To address the above issues, we propose a compact and robust global descriptor for VPR, named SuperVLAD. Our method uses the transformer based backbone to produce initial patch features. Similar to NetVLAD, SuperVLAD calculates the weights of assigning patch features to clusters in the soft-assignment way. The key difference is that SuperVLAD does not compute cluster centers. Instead, we directly perform a weighted summation of the local features assigned to each cluster to achieve the purpose of aggregating local features. This means SuperVLAD only needs to learn the assignment of local features to clusters, without needing to learn the cluster centers, which directly improves its domain generalization. Mcanwhile, magnitude lower than vanilla NetVLAD), resulting in more compact global descriptors. Furthermore, by introducing supernumerary ghost clusters during the soft assignment and retaining one real (useful) cluster in the final output vector, we obtain a 1-cluster VLAD descriptor with extremely low dimensionality and promising performance. Our work brings the following contributions:
set small number of clusters (an order of
(1) We propose a SuperVLAD aggregation method that does not require cluster centers to produce robust global descriptors. It can mitigate the performance degradation in NetVLAD caused by the training and testing data bias, while having fewer parameters (i.e., more lightweight). Besides, can use only small number of clusters to yield compact descriptors.

---

(2) We also design I-cluster VLAD method by employing supernumerary ghost clusters during the soft assignment. It produces very low-dimensional features similar to the pooling methods. Compared to the same-dimensional GeM feature or class token, it shows notable performance advantages.
(3) We conduct extensive experiments using various transformer- backhones, demonstrating the effectiveness of our method. Furthermore, our SuperVLAD with the DINOv2 backbone outperforms state-of (SOTA) methods on several benchmark datasets with lower-dimensional descriptors.
2 Related Work
In the early development of VPR, the methods predominantly relied on hand-crafted local features
such as SURF [ 13]. These local descriptors were then aggregated into global features using
algorithms such as Bag of Words [3) Fisher Vector 477, and VLAD 28 53 32 The resulting
global feature vectors were subsequently used to perform nearest neighbor search over the database
to retrieve the most similar images. As deep learning techniques have advanced, the representation
capability of deep features has been widely recognized in the VPR community [SI 4 [2]] [2]
44 18, 20 58 60, 52 33
I 21 77 10]. Arandjelovic et al. 4 designed a pioneering architecture
that combines deep neural networks with the proposed differentiable VLAD aggregation approach called NetVLAD. Likewise, other traditional aggregation algorithms were also transformed into differentiable modules as the aggregation layer of neural networks for end-to-end training [26] 45 61].
Despite remarkable performance over global max (or average) pooling, NetVLAD-rellted methods 29 34 21 tended to have high-dimensional feature representations, which can be drawback in terms of real-time performance (9]. To address this, the Generalized Mean (GeM) pooling [48] was considered simple alternative that obtains low-dimensional global features. This method simply extended global average pooling by using the p-norm of local features instead of the average, where p is also trainable parameter. Additionally, Berton et al. [9] established an open-source benchmark to fairly compare these standard global-retrieval www.rreeeeelbbsssseeeee VPR approaches under unified framework.
rewrieeal--bbbeeee methods have achieved reasonable performance, most of them exhibit limited robustness in challenging environments and are susceptible to perceptual aliasing. Two common methods to improve the robustness of VPR systems are leveraging temporal consistency constraints and spatial consistency constraints. The first method matches image sequences (i.e., [9 to achieve robust VPR with extreme condition changes. The second method usually involves a two-stage VPR process (25] 24
Although
maintains temporal continuity) [6 22
37
8 55 67
38 41],
where it first retrieves the top-k candidate images in the database with global features and then re-ranks these candidates by matching local features with spatial consistency verification. However, these methods always introduce extra consumption computational latency or/and memory footprint.
In the last two years, with the emergence of purpose-buill built large-scale VPR training datasets α and pre-trained foundation models 62 46) some robust global-retrieval-bassee VPR methods have been proposed. CosPlace 7 and EigenPlaces [0] cast the training of VPR as a classification problem and trained the VPR model the San Francisco eXtra Large (SF-XL) datasets. MiXVPR (2 incorporated the deep features with the multi-layer perceptrons and trained the model with the Multi-Similarity loss (56) on the large-scale and appropriate supervised GSV-Cities (1] dataset. These works all have achieved outstanding performance using only the CNN models. Other works [30
39 27
were based on visual foundation models and achieved better results. DINOv2 [46] has been the most widely used foundation model in VPR. It is a ViT-based [17] model trained on the large-scale
curated LVD-142M dataset using a sulf-suuer strategy and can offer powerful visual features for downstream tasks. SeR [41] proposed a hybrid global-local adaptation method to adapt the DINOv2 model for two-stage VPR. CricaVPR (39 also adapted DINOv2 as the backbone and proposed a cross-image corellttion-wwae representation learning method to enhance the robustness of image features. closely related work to our SuperVLAD is the SALAD work 127]. which used DINOv2 as the backbone and presented novel aggregation algorithm to improve NetVLAD. SALAD redefined the soft assignment of local features in NetVLAD as an optimal transport problem and employed the Sinkhorn algorithm [1] to solve it. Different from this work, our SuperVLAD maintains the ooft-assignment way of NetVLAD but effectively solves the issue caused by the learned cluster centers in NetVLAD being unsuitable for various inference data distributions.
3

---

Soft-asignment
remove ghost
conv (w, b) 1x1xD*K
softmax
Sost-assign.
assign.
asss-
KXD VLAD
Patch
Backbone (ViT)
Tokens
VLAD core: Og (x) X
intra-norm
vector
(Reshape)
Local Reatures
1.2 norm
Image
SuperVL.AD layer
Figure 2: Illustration of the proposed SuperVLAD layer. It aggregates the patch tokens output by the transformer- based backbone and produces K D vector the global descriptor. Note that the VLAD core of SuperVLAD has no cluster center, which is the main difference from NetVLAD.
3
Methodology
Fig.
shows the overview of SuperVI.AD. We first use backbone to extract local features (Sec. then aggregate them with SuperVLAD (Sec. 3.2). I-Cluster VLAD is an alternative to get more
3.1 D.
compact descriptors (Sec. 3.3) and cross-image encoder is optional to boost performance (Sec. 3.4)
3.1 Local Features Extraction
The Vision Transformer (ViT) [7 and its variants possess superiority capturing long-range feature dependencies and have shown remarkable performance
various computer vision tasks [66 63 64], including VPR [55] 4.) In this work, we use VYT (or its variants) as the backbone for feature extraction. VIT imitially divides the image into N patches and linearly projects them into D-dim patch
embeddings pp E R RNDD, after which it prepends a learnable class token to Tp as zQ E (1++) Following the addition of positional embeddings, zo input into a sequence of transformer encoder layers to generate the feature representation. The final output of ViT includes one class token and N patch tokens. We directly discard the former and use the latter as local features to be input the subsequent SuperVLAD aggregation layer, thereby obtaining the final global descriptor of the place image.
3.2 SuperVLAn Layer
SuperVLAD produces the global descriptor of an image by first associating all local features of this image with K clusters and sub- sequently aggregating these features into each cluster. The basic process of the SuperVLAD layer is similar to that of NetVLAD [4) so we start by introducing our work from NetVLAD (and
VLAD (8
CULAD
For NetVLAD, given N local descriptors {x,} (x, E RP) of an image input, and К cluster centers {cx } (c& E RD) parame- ters, it computes a matrix V E RKXD xD as the image representation. The k-th row in the matrix V accumulates the (weighted) residuals (x; - C₂) of local descriptors assigned to cluster Cj- More formally, the (k: jih element in the matrix V is computed as follows:
Figure 3: Unlike VLAD, since the parameters Wx and by used for soft-assignment in (1) NetVLAD are decoupled from cluster center Ca. Cs does not necessarily coincide with the true centroid of the cluster (Voronoi cell). Its robustness against domain shift can be im- proved to some extent. Super- VLAD completely eliminates the need for cluster centers and avoids their negative impact.
N Σ an (x:) (Ii5 - ck.),
VR.
where vijj and Ca.j are the j-th element of the i-th local descriptor and k-thh cluster center, respectively. ax (x:) is the weight of the local descriptor x; assigned to the cluster cg. In VLAD, the as- signment is hard, i.e,, (x(x;) equals I. In contrast, NetVLAD replaces it with the soft assignment and computes an (x.)
-allx
ag (xi) Eg c -a

---

Expanding the squares in Eq. 2 we can readily observe the cancellation of the term e e=]]] between the numerator and the denominator, yielding the following form:
w x,tbb
0g (x.) = Ex C "ixttb
(3)
where vector Wa= = 20rc and scalar by = -a ||cx/12. In the implementation of NetVLAD, {W&]. {br}, and } are set as three independent sets of trainable parameters (while VLAD has only {c&))
That is, {w&} and {br} ) are actually decoupled from {c As shown in Fig. 3 NetVLAD, with its greater flexibility than VLAD, has some potential to alleviate the performance drop caused by domain shift [4. However, NetVLAD retains cluster centers, which would be hardly suitable for various data distributions in inference as it is learned on one training set [3]. Since the assignment of local features has been decoupled from the cluster centers in NetVLAD (in VLAD, it is based on the distance to the cluster center), we can directly remove the cluster center and aggregate the first-order statistics of local features (rather than residuals) assigned into each cluster. So, the matrix V in SuperVLAD can be formulated as
N
ewf x
Vi.5 = Σ ak (x.) Dzs i=1
=
Ene e ew Tt.j"
(4)
For each cluster, SuperVLAD has the parameters We and by compared to NetVLAD with the parameters WE. bg. and Cg. and VLAD with the parameter Cy. This makes SuperVI.AD not affected by the cluster center C2: and provides greater flexibility compared to NetVLAD. Besides, the number of parameters in SuperVI.AD is about half less than that in NetVLAD. It is worth noting that Eq. is different from applying the attention mechanism to local features. Our approach involves assigning features to clusters (rather than applying attention to features). More specifically, given feature, the sum of the weights of assigning to all clusters is the constant 1. However, for a given cluster, the sum of the weights of assigning all features to it is not fixed (for attention, it is the constant 1). The recent work SALAD [27] also sums the local features with the soft-assignment weights to get the global deseriptor. However, its soft assignment relies clusters (the so-called optimal transport between clusters and local features), i... not free from clusters, and requires iterative computations to solve. Its parameters are less than NetVLAD. These are different from our lightweight and low-computation SuperVLAD. Besides, In SuperVVLADD we set only a small number of clusters, namely 4, which makes the output compact without additional dimensionality reduction techniques. As shown in Fig. [] the main module of SuperVLAD is I 1x1 convolution (conv) and softmax for soft-assignment, and VLAD core for aggregation. Unlike NetVLAD, the VLAD core of SuperVLAD has no parameters, and all trainable parameters exist only in the conv layer. The matrix V is finally intra-normalized, flattened into a vector, and entirely 12-normalized as the output global descriptor.
Moreover, we also absorb GhostVLAD(65) which extends NetVLAD by introducing "ghost" clusters. Specifically, it adds additional G ghost clusters that are used for the soft assignment in the same way as the original K clusters, but the ghost clusters are disregarded in the aggregation process and do not directly contribute the final output. Ghost clusters can be used correspond useless objects in VPR, such as sky, ground, dynamic objects, and so forth. In SuperVLAD, the number of ghost clusters is set to only one (i.e., G = 1).
3.3 1-Cluster VLAD
Due to the high dimensionality of the descriptors output by NetVLAD, some SOTA VPR methods use learnable linear projections or GeM pooling for low-dimensional representations, e.g., CosPlace 77, FigenPlaces 10), and VPR [2]. The GeM pooling particularly popular because it easily produces global representations with the same dimensionality as the pooled local/patch features, although it may be lacking performance. Here, we design a 1-cluster VLAD algorithm that can produce very low dimensional descriptors, which has the same output dimensionality as GeM but performs better. Specifically, we only need to set the number of useful clusters in SuperVLAD to 1. and the number of ghost clusters to greater than 1, (we set it to 2), i.e., K=1 and G=2, to get the 1-cluster VLAD. This means that all descriptors representing objects relevant to VPR are assigned to the same cluster, and their weighted sum (the weight is aj (x;)) )) is used as the final global descriptor.
In implementation. by can also be removed (i.e., bz=0), as in the different implementation versions of NetVLAD (refer to https://github..om//
more. woooe.mnfneevlad-veette versioons).
5

---

3.4 Cluster-wise Cross-image Interaction
This subsection introduces an optional process for SuperVLAD (not for 1 1-Cluster VLAD). To further enhance the performance, we draw inspiration from the cross-image encoder proposed in CricaVPR [39.. which can use the cross-image variations as a cue to guide the representation learning and produce more robust image features through cross-image in- teraction in batch. However, directly using the transformer encoder to process the entire SuperVLAD vector would incur significant memory and running time overhead, proportional to the square of the vector size. To overcome this issue, we first split the SuperVLAI descriptors by clusters and then input them into the cross-image encoder ( (ie. two stacked transformer encoders) for cluster-wise cross-image interaction. That is, this encoder will model the correlation between the aggregated features belonging to the same cluster of all images in a batch. We then concatenate the aggregated features belonging different clusters of an image and L2 normalize again to get the final descriptor of this image.
Table 1: Summary of the evaluation datasets.
Number
Dataset
Description
Database Queries
Pitts30k-test urhan, panorama
10,000
6816
MSLS-val
urban, suburban
18,871
740
Nordland SPED-test various scenes
natural, scasonal
27,592
27,592
607
607
4 Experiments
4.1 Datasets and Performance Evaluation
We conduct experiments on several VPR benchmark datasets, which cover various challenges in VPR such as viewpoint changes, condition changes, and perceptual aliasing. Table | provides concise summary of them. Pitts30k (54) mainly exhibits severe viewpoint changes in the urban environment. MSLS [57] is comprehensive dataset comprising images collected in urban, suburban, and natural scenes over 7 years, and encompasses a wide range of visual changes (viewpoint and condition changes). Nordland (10] exhibits seasonal changes in natural and suburban scenes. SPED 11 consists of low-quality and high-scene-depth images captured from diverse scenes with various condition changes. More details are in Appendix G
In our experiments, we employ the Recall (i.e., R@N) to assess the recognition performance of VPR methods. This metric quantifies the percentage of queries for which at least one of the retrieved reference images falls within a predefined threshold of the ground truth. We set the threshold to 25 meters for Pitts30k and MSLS, +100 frames for Nordland, unique counterpart for SPED, following common evaluation procedures [54 57
4.2 Implementation Details
Here we describe the implementation details of training the DINOv2- based SuperVLAD model on the GSV-Cities [I] dataset for SOTA comparison. More details of training the other transformer-base models for the ablation study are Appendix(E) We implement our experiments on two NVIDIA GeForce RTX 3090 GPUs using PyTorch. The size of the input image is 224x224 in training (322x322 inference) and the token dimension of the DINOv2-base backbone is 768. The descriptor dimension of SuperVLAD is 3072 and that of 1-cluster VLAD is 768. We only fine-tune the last four transformer encoder layers (freeze the previous layers) of the DINOv2 backbone. For the loss function, we utilize the multi-similarity loss (56) and set its hyperparameters as in [I] The model training is performed using the Adam optimizer with an initial learning rate 0.00005, halved every 3 epochs. Considering that the cross-image encoder is not initialized, (0.0001) to train separately. Each training batch consists of 120 places, with 4 images per place, resulting in a total of 480 images. An inference batch consists of 8 images (except for the SPED dataset where the batch size is 4). The training process is terminated when the performance on -SL-- does not improve for three epochs. The actual number of effective training epochs is and the training time is 81.6 minutes.
a larger learning rate
4.3 Comparisons with State-of-the-Art Methods
In this section, we compare the proposed SuperVLAA with several excellent VPR methods, including seven one-stage methods using global feature retrieval (like ours): NetVLAD [), SFRS [21].

---

Table 2: Comparison to satee-of-the-eee methods on four VPR benchmark datasets. The best results are highlighted in bold and the second are underlined. The descriptor dimensionalities of two-stage methods are not displayed.
Pitts30k
MaL
Nordland
SPED
Method
Backbone
Dim
RØl
R@5
R 10
RØl
R@5
R
10
R
R@
R 10
RØ
R@
5 R
R@I
NetVLAD
4
VGG16
32768
81.9
91.2 93.7
53.1
66.5
71.1
6,4
10.1
12.5
70.2
84.5
89.5
SFRS 2
VGG16
4096
89.4
94.7
95.9
69.2
80.3
83.1
16.1
23.9
28.4
80.2
92.6
95.4
Trans VPR 55
89.0
94.9
96.2
86.8
91.2
92.4
63.5
68.5
70.2
85.7
90,9
91.8
CosPlace
7
VGG16
512
88.4 94.5
95.7
82.8
89.7
92.0
58.5
73.7
79.4
75.5 87.0
89.6
MixVPR 2
ResNet50
4096
91.5 95.5
96.3
88.0
92.7
94.6
76.2
86.9
90.3
84.7
92.3
94.4
EigenPlaces
10
ResNe150
2048
92.5 96.8
97.6
89.1 93.8
95.0
71.2
83.8
88.1
70.2
83.5
87.5
SelaVPR 41
DINOv2-L
92.8 96.8
97.7
90.8
96.4
97.2
87.3
93.8
95.6
89.5
94.6
95.9
CricaVPR 39
DINOv2-B 4096
94.9
97.3
98.2
90.0
95.4
96.4
90.7
96.3
97.6
91.4
95.2
96.2
SALAD
27
DINOv2-B
8448
92.5
96.4
97.5
92.2
96.4
97.0
89.7
95.5
97.0
92.1
96.2
96.5
Super LLAD
DINOv2-B
3072
95.0
97.4
98.2
92.2
96.6
97.4
91.0
96.4
97.7
93.2
97.0
98.0
Query
SuperVLAD (Oira)
NerVLAD
SERS
CosPlace
EigenPlacess
SelaVPR
Vieupoint
changes
alinsing
Occlusions
Peresptual
aliasing
Sensonal
changes
Pereeptual
Lightnn
chinges
Pereeptual
aliasing
Figure 4: Qualitative results. In these four challenging examples (covering viewpoint variations, condition variations, dynamic objects, etc.), our SuperVI.AD successfully retrieves the right database images, while other methods get the wrong results.
CosPlace 77. Mix 21, Figenplaces [T], CricaVPR (39 and SALAD 27 as well as two two- stage methods with re-ranking: Trans (55) and SeRaVPR [4]. Note that MixVPR. Crica SALAD, and our SuperVLAD use the same training dataset, i.e., GSV-Cities. Meanwhile, CosPlace and EigenPlaces are trained on the purpose-built extra large-scale (SF-XL) datasets. Additionally, the latest works SelaVPR, CricaVPR, and SALAD all use the foundation model DINOv2 as the backbone (Sela using DINOv2- and others using DINOv2-base) and achieve the SOTA performance on the VPR task. So here we follow them and use the DINOv2-bar base backbone. The details of the above methods can be seen in Appendix H Table 2 shows the quantitative results and our SuperVIAA achieves the best results on all these datasets.
The methods based on the DINOv2 backhone, including SelaVPR, CricaVPR, SALAD, and our SuperVLAD, all achieve excellent performance and outperform the remainder methods on these datasets with diverse challenges. These methods all fine-tune DI in different ways, which shows that based on the powerful feature representation capability of DINOv2, coupled with appropriate fine-tuning, it is sufficient to cope with most challenges in the VPR task. However, our method uses more compact feature representation (than Crica and SALAD) and outperforms other methods on the four datasets. In particular, it achieves 93.2% R R@1 I on the SPED dataset, demonstrating significant advantages compared to other global-retrieaal based methods and two-stage methods. This indicates the high robustness of our method handle condition variations in datasets containing low-quality and high-scene-dee -epth images. Additionally, both CricaVPE. and Super VLAD use the cross-image encoder and thus achieve significantly better performance than other methods on Pitts30k and Nordland, which are able to provide different images of the same place in batch and mutually improve condition invariance and viewpoint invariance [39]. However, such benefit cannot be obtained on MSLS-val and SPED (all query images from different places), our approach still outperforms all other methods (and is obviously better than CricaVPR).
7

---

Fig. qualitatiivel shows the superior perfor- mance of our SuperVLAD in some difficult scenes. These examples exhibit various chal- lenges, such as drastic condition changes, view- point changes, and occlusions (caused by dy- namic objects). Other methods mostly return similar images from different places, L.., suffer from perceptual aliasing and fail to get the right results. However, our SuperVLAD retrieves the correct reference images, showing high robust- ness against these challenges.
96
SuperVLAD
95
94
Rocall@1
93
SALAD
92
MixVPR
SFRS
Fig. 5 simultaneously shows the R@1 on Pitts30k, the inference time of single image, and the descriptor dimensionality. Among the four methods (SFRS, MixVPR, SALAD, and Su- perVLAD), MiX uses the CNN backbone and the feature mixing method to get global de- scriptors, which achieves the shortest inference time. The remaining three methods use VLAD- related methods to get global features, and our SuperVLAD achieves the fastest inference speed among them. Although SFRS is based on the CNN model, it uses NetVLAD with 64 clusters to obtain high-dimensional image features and then uses PCA for dimensionality reduction (time-consuminn)) SALAD and our SuperVLAA are based the foundation model DINOv2. However, SALAD uses the Sinkhorn algorithm to iteratively compute optimal transport assignment between clusters and local features, which takes more inference time than our method (17.8ms vs. 13.5ms). Besides, the number of parameters in our SuperVLAD aggregator is much lower than that of SALAD (less than 3/1000), as shown in Table 3 Our SuperV uses lightweight and low-compute aggregation layer to get the compact global descriptor (without extra dimensionality reduction), which simultaneously has fast inference speed, low-dimensional descriptors, and excellent recognition performance.
Inference time (mS/ single Figure 5: The comparison of some global-retrieval- based methods in Recall I (on Pitts30k), infer- ence time (ms/single image), and descriptor dimen- sionality. The diameter of each dot is proportional to the descriptor dimension. Our SuperVLAD gets the best R@ I with the most compact descriptor.
5 10 15
20
4.4 Ablation Study
In this section, we conduct series of abla- tion experiments to demonstrate the effec- tiveness of the proposed SuperVLAD and 1- Cluster VLAD. The cross-image encoder is not used by default in ablation experiments. The SuperVLAD using the cross-image en- coder denoted as SuperVLAD'.
Table 3: The number of parameters of SALAD and SuperVLAD that both use the DINOv2-base backbone. The value in parentheses is the number of parameters in the optional cross-image encoder.
Method SALAD SuperVLAD 86.6 2-11.)) 0.0038
Total (M) Trainable (M) Aggregator (M)
88.0
29.8
1.4
Effect of Super We first validate the
effectiveness of SuperVLAD by comparing it to NetVLAD. To be fair, we do not use the cross-image encoder in Super VLAD, and apply multiple different transformer-based models as the backbone, including the ViT [17 and CCT [23] models pre-trained on ImageNet [15), and the foundation model DINOv2 [46] (that is also a ViT model) pre-trained on the large-scale curated dataset. We conduct experiments with three training sets (MSLS, Pitts30k, and GSV-Cities) and two test sets (MSLS and Pitt30k). To validate that SuperVLAD has superior generalization ability over LetV in addressing the domain shift issue mentioned above, we focus on the cross-domain inference performance of models. For example, the performance of model trained on Pitts30k when tested on MSLS, or vice versa. The results are shown in Table 4 Note that P0tts30k contains only urban scene images, while MSLS covers urban, suburban, and natural scenes, and GSV-Cities is a large-scale training set with accurate supervision and diverse visual variations. Thus, the performance of model trained on Pitts33k and tested on MSLS best reflects domain generalization, while the reverse is less indicative. The model trained on GSV-Cities encounters almost no domain drift issues on these test sets (Pitts30k and MSLS-val) The experimental results largely align with these facts and the above theory. Training the VuperVLAD model (based on CCT and DINOv2) on Pitts3kk and testing it on Pitts30k yields similar results to NetVL.D.. However, when tested on MSL S, shows an absolute R@ improvement of 6.8% (with CCT) and 2.4% (with DINOv2), respectively. The SuperVLAD model trained on MSLS using VIT
8

---

Table 4: Comparison of NetVLAD and SuperVLAD with different backbones and training sets. Since the token dimension of CCT is only half of ViT. models based on it have clusters, while models based on ViT/DINOv2 have 4 clusters. The values in parentheses indicate the change in results of SuperVLAD relative to NetVLAD: red for increase, green for decrease, and black for no change.
Pius30k
MSLS-val
Method
Backbone
Training set
R@1
R@S
R@10
R@1
R@5
R@10 I0
NetVLAD
79.0
90,9
94.1
75.7
89.5
91.8
VIT
MSLS
SuperVLAD
79.4(40.4)
91.9c+10)
94.7(40.)
75.9(+0.)
88.8
91.8(+0.0)
NetVLAD
84.7
93.0
95.3
55.4
68.5
73.1
CCT
Pitts30k
SuperVLAD
84.7(+0.0)
93.0(+0.0)
95.2(-0.1)
62.2(+6.8)
73. ((+4.))))
77.3(+4.2)
NetVLAD
89,4
95.7
96,9
72.7
84.7
87.2
SuperVLAD DINOv2
Pitts30k
89.4(+0.0)
95.8(+0.1)
97.1(+0.2)
(+.1(+2.4)
84.6
87.4(+0.2)
NetVLAD
92.3
96.8
97.7
91.6
96.2
96.8
DINOv2
GSV-Cities
SuperVLAD
92.6(+0.3)
96.4(-0.4)
(7.5(-0.2)
92.2(+0.6)
(5.9(-0.) 96.8(+0.0)
Table 5: Comparison of SuperVLAD with and without the ghost cluster. "SV" is short for Su- PerVLAD. The methods with the "-ng" suffix those without the ghost cluster. Specifically, DINOv2-SV is the model based on DINOv2 and trained on GSV-Cities as detailed in Table
Table 6: Comparison of the very low-
dimensional global descriptors with the same dimensions as the local descriptors. That is, all methods produce 768-dim global descriptors (us- ing DINOv2-base backbone). All models are trained on GSV-Cities.
Pitts30k
MSLS-val val
Method
Pius30k
MSL.-val
R@ R@ RØ R R R 10
Method
R T R R 10
68
R R 10
CCT-SV-ng
84.1 92.9 95.1
60.3
72.2
76.9
GeM Pooling
89.5 95.0
96.3
85.4
93,0
94.3
CCT-SV
84.7 93.0 95.2
62.2
73.1
77.3
class token
91.4 96.2
97.4
88.4
95.1
96.4
DINOv2-SV-ng
92.4 96.5 97.6
92.4
96.4
96.9
1- 1-ClusterVL.AD
91.6 96.2
97.4
90.4
95.3
96.4
DINOv2-SV
92.6 96.4 97.5
92.2
95.9
96.8
exhibits similar test results to NetVLAD on MSLS-val but still has a slight advantage when tested on Pitts30k. These results demonstrate that our SuperVLAD has better domain generalization than NetVLAD. Since the GSV-Cities dataset covers the various visual variations in VPR and provides accurate supervision for fine-tuning, both models trained on perform very well on Pitts30k and MSLS-val. However, this does not mean that our SuperVLAD has no advantage over NetVLAD with such a large-scale VPR training set. SuperVLAD achieves the same performance as NetVLAD with fewer parameters and computations, e.g., offering equivalent performance at a lower cost.
Effect of ghost clusters and the performance of 1-Cluster VLAD. To investigate the impact of ghost clusters, we conduct two sets of experiments based on CCT and DINOv2 to compare the performance with and without ghost clusters. All models also do not use the cross-image encoder. The based model is trained on Pitts30k and the DINOv2- shown in Table[5] When based on DINOv2, whether using ghost clusters or not achieves excellent performance, with little to no difference in their performance (without using ghost clusters even slightly better). However, based on CCT and trained on Pitts30k, the use of ghost clusters resulted in certain improvement. This indicates that the ghost cluster is still necessary. More importantly, we leverage it to achieve 1-Cluster VLAD. We compared the proposed 1- 1-Cluster VLAD with two commonly used methods (GeM pooling and class token) that can ouiput very compact features of the same dimension as local features (as does our I-Cluster VLAD). The results, as shown in Table 6 demonstrate the obvious superiority of 1 1-Cluster VILAD compared to the other two methods. So, our 1- I-Cluster VLAD can be used as a new choice for very low-dimensional descriptors.
model is trained on GSV-Cities. The results are
Effect of the number of clusters. To investigate the impact of the number of clusters on global de- scriptors, we conducted two sets of experiments using SuperVLAD with and without the cross-image encoder. All models utilize DINOv2 as the backbone and are trained on GSV-Cities. Results (see Ta- [ indicate that even with small number of clusters, there is no significant performance decrease. This is primarily attributed to the powerful feature representation capability of the transformer-bast foundation model DINOv2, while the large-scale VPR training dataset GS G-Ctiis offers appropriate supervision for fine-tuning. Even with a small number of clusters, it allows for reasonable and effective classification/ctussrrnng of objects that are relevant to place recognition (e-g., buildings, vegetation). Unlike most VLAD-related methods that use dozens of clusters, our experiments show

---

that just 4 clusters can achieve robust VPR in most cases, which provides compact global descriptors without the need for additional dimensionality reduction techniques.
Effect of the cross-image encoder. Ta- ble 7 also presents the comparison of Su- per VLAD with and without cross-image encoder (i.e., our cluster-wise cross-image interaction). It is evident that using a cross- image encoder consistently leads perfor- mance improvements. However, the per- formance enhancement on Pitts30k sig- nificantly greater than that on MSLS. This because Pitts30k can provide a batch of images with different viewpoints from the same place during inference, which allows different images to directly improve the ro- bustness of each other through the cross-image interaction, while MSLS cannot. Considering that conditions like the Pitts30k dataset might not be available in practical application, we consider the cross-image encoder as an optional component for performance enhancement. is worth noting that, even without it, our method still achieves performance comparable to the SOTA methods.
Table 7: Comparison of the SuperVIAD ablated ver- sions with different numbers of clusters, as well as with and without the cross-image encoder (the former de- noted as SuperVLAD
No.
Pitts
MSL.Svvl
Method
Clusters
R
1 R
R@5
R@
RO
1 R@5 R TO
4
92.6
96.4
97.5
92.2
95.9
96.8
SuperVIAD
16
92.4
96.7
97.7
90.5
96.2
96.8
92.5
96.5
97.7
91.4
95.8
96.5
95.0
97.4
98.2
92.2
96.6
97.4
SuperVLAD'
16
93.6
96.9
97.9
91.4
96.2
97.2
64
94.6 97.5
98.1
92.3 96.8
97.3
Conclusions
In this paper, we introduced SuperVLAD, compact and robust global descriptor for VPR. By climinating the need for cluster centers and utilizing a small number of clusters, SuperVLAD achieved improved domain generalization and produced more compact descriptors. We also designed a I-cluster VLAD descriptor with extremely low dimensionality by introducing supernumerary ghost clusters during the soft assignment. Experimental results on various transformer-basee backbones validated the effectiveness of SuperVLAD, producing more robust features than NetVLAD with a lighter aggregation layer. Furthermore, the DINOv2-based SuperVLAD also outperformed SOTA methods on several VPR benchmark datasets with more compact global descriptors.
Limitations & Future Work. While our study presents some improvements in VPR. particularly in enhancing domain generalization ability, reducing the dimensionality of global descriptors, and reducing the number of parameters for the aggregation layer, we acknowledge two limitations of our work: Firstly, the current success of Super LLAD relies on the use of the transformer backhone. Although our experimental results provided in Appendix demonstrate that SuperVLAN still has certain advantages over NetVLAD when using a CNN backbone with a small number of clusters, it is not good NetVLAD in more commonly used settings. This can be SuperVLAD. Secondly, our method uses the cross-image encoder to further enhance performance, similar to CPR (39]. It is necessary to set the inference batch size Setting it to directly renders the cross-image encoder ineffective during inference, resulting in a performance drop due to the gap between training and inference. Since we can only use a single-frame frame query in some practical applications, we compromise to treat the cross- cross-image encoder as an optional module that is not used (during both training and inference) when multiple-framt inference is not feasible. Even without the cross-image encoder, our SuperVLAD can achieve good performance. Some additional discussion on the limitations of SuperVLAD is in Appendix B In future work, we will try to address the above limitations, and further assess the long- term stability and potential performance changes of the model, which are important for practical deployments.
limitation of
appropriate value.
Acknowledgments and Disclosure of Funding
This work was supported by the National Key R&D Program of China (2022YYBB400144004000000)
SSTIC Grant (KJZDD02309231 15 5106012, KIZID202302231 14916032, 1 11644008)))) Beijing Key Lab of Networked Multimedia, the Project of Pengcheng Laboratory (PCL.2023008, PCL2024Y02), and National Natural Science Foundation of China (62402252).
10

---

References
[1] Amar Ali-bey, Brahim Chaib-draa, and Philippe Giguère. Gsv-cities: Toward appropriate supervised visual place recognition. Neurocomputing, 513:194-2001 2033, 2022.
[2] Amar Ali- Biy, Brahim Chaib Draa, and Philippe Giguere. Mixvpr: Feature mixing for visual place recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2998-3007, 2023.
[3] Adrien Angeli, David Filliat, Stéphane Doncieux, and Jean-Aready Meyer. Fast and incremental method for loop-closure detection using bags of visual words. IEEE transactions robotics, 24(5): 1027-10777 2008.
[4] Relja Arandjelovic, Petr Gronat, Akihiko Torii, Tomas Pajdla, and Joset Sivic. Netvlad: Can architecture weakly supervised place recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5297-5307, 2016.
[5] Relja Arandjelovic and Andrew Zisserman. about vlad. Proceedings of the IEEE conference on Computer Vision and Puttern Recognition, pages 1578-1585, 2013.
[6] Herbert Bay, Andreas Ess, Tinne Tuytelaars. and Luc Van Gool. Speeded-up robust features (surf). Computer vision and image understanding, 110(3)346-33 2008.
(7) Gabriele Berton, Carlo Masone, and Barbara Caputo. Rethinking visual gco-localization for large-scale applications. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4878 4888, 2022.
[8] Gabriele Berton, Carlo Masone, Valerio Paolicelli, and Barbara Caputo. Viewpoint invariant dense matching for visual geolocalization, IEEE/CVF International Conference on Computer Vision, pages 12169- 2021.
[9] Gabriele Berton, Riccardo Mereu, Gabriele Trivigno, Carlo Masone, Gabriela Csurka, Torsten Sattler, and Barbara Caputo. Deep visual localazziinttion benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 5396-5407, 2022.
[10] Gabricle Berton, Gabriele Trivigno, Barbara Caputo, and Carlo Masone. Eigenplaces: Training viewpoint robust models for visual place recognition. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 10800-1090. 2023.
[11] Zetao Chen, Adam Jacobson, Niko Sünderhauf, Ben Upcroft, Lingqiao Liu, Chunhua Shen, Ian Reid, and Michael Milford. Deep learning features at scale for visual place recognition. In 2017 IEEE international conference on robotics and automation, pages 3223-3230. IEEF, 2017.
[12] Zetao Chen, Fabiola Maffra, Inkyu Sa, and Margarita Chli. Only look once, mining distinctive landmarks from convnet for visual place recognition. In 2017 JEEE/SS International Conference on Intelligent Robots and Systems (IROS), pages 9-16. IEEE, 2017.
[13] Mark Cummins and Paul Newman. Fab-map: Probabilistic localization and mapping in the space of appearance. The International Journal of Robotics Research, 2766)6647-665 2008
[14] Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances neural information processing systems, 26, 2013.
[15] Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. Imagenet: large-scale hicrarchical image database. In IEEE conference on computer vision and pattern recognition, pages 248-255, 2009.
[16] Anh-Dzuna Doan, Yasir Latif, Tat-Jun Chin, Yu Liu, Thanh- Toan Do, and Ian Reid, Scalable place recog- nition under appearance change for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9319 90099932,2112 2019.
[17] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2020.
[18] Sourav Garg, Adam Jacobson, Swagat Kumar, and Michael Milford. Improving condition and environment- invariant place recognition with semantic place categorization. In 2017 JEEE/RSJ International Conference Intelligent Robots and Systems, pages 6863-6870. IEEE, 2017.
11

---

[19] Sourav Garg and Michael Milford. Sequet: Learning descriptors for sequence- hierarchical place recognition. IEEE Roboticy and Automation Letters, 6(3):4305-4312, 2021.
[20] Sourav Garg, Niko Suenderhauf, and Michael Milford. Don't look back: Robustifying place categorization for viewpoint-and condition-invarian place recognition. 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 3645-3652, 2018.
[21] Yixiao Ge, Haibo Wang, Feng Zhu, Rui Zhao, and Hongsheng Li. Self-supervising fine-grained region similarities for large-scale image localization. In European conference computer vision, pages 369-386. Springer, 2020.
[22] Peter Hansen and Brett Browning. Visual place recognition using hmm sequence matching. 2014 JEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4549-4555. IEEE, 2014.
[23] Ali Hassani, Steven Walton, Nikhil Shah, Abulikemu Abuduweili, Jiachen Li, and Humphrey Shi. Escaping the big data paradigm with compact transformers. arXiv preprint 104.05040 2021.
[24] Stephen Hausler, Sourav Garg. Ming Xu, Michael Milford, and Tobias Fischer. Patch-netvlad: Multi-scale fusion of socally-global descriptors for place recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14141-14152 2021.
[25] Stephen Hausler and Michael Milford. Hierarchical multi 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 3327-3333 IEEE, 2020.
multi-process process fusion for visual place recognition. In
[26] Yi Hou, Hong Zhang, and Shilin Zhou. Bocaf: efficient image matching with bag of convnet features for scalable and robust visual place recognition. Autonomous Robots, 42(6): 169-1185, 2018
[27] Sergio Izquierdo and Javier Civera. Optimal transport aggregation for visual place recognition. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2024.
[28] Hervé Jégou, Matthijs Douze, Cordelia Schmid, and Patrick Pérez. Aggregating local descriptors into a compact image representation. In 2010 IEEE computer society conference on computer vision and pattern recognition, pages 3304-3311. IEEE, 2010.
[29] Hyo Jin Kim, Enrique Dunn, and Jan-Michi Michael Frahm. Learned contextual feature reweighting for image geo-localization. In Proceedings of the IEFF Conference on Computer Vision and Pattern Recognition, pages 2136-2145, 2017.
[30] Nikhil Keetha, Avneesh Mishra, Jay Karhade, Krishna Murthy Jatavallabhula, Sebastian Scherer, Mad- hava Krishna, and Sourav Garg. Anyloc: Towards universal visual place recognition. arXiv preprint arXiv:2308.. 00688, 2023.
[31] Ahmad Khaliq, Shoaib Ehsan, Zetao Chen, Michael Milford, and Klaus McDonald M.DDnaadd-Maie.... A holistic visual place recognition approach using lightweight enns for significant viewpoint and appearance changes. IEFE transactions on roboiics, 36(2):561 -569, 2019.
[32] Hyo Jin Kim, Enrique Dunn, and Jan-Michael Frahm. Predicting good features for image geo-localization using per-bundle vlad. In Proceedings of the IEEE International Conference on Computer Vision, pages 1170 1178, 2015.
[33] María Vallina Nicola Strisciuglio, and Nicolai Petkov. Data-efficient large scale place recognition with graded similarity supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 23487-23496, 2023.
[34] Liu Liu, Hongdong Li, and Yuchao Dai. Stochastic attraction-eepulsion embedding for large scale image localization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2570 2579, 2019.
[35] Stephanie Lowry and Henrik Andreasson. Lightweight, viewpoint-invariint visual place recognition in changing environments. IEEE Robotics and Automation Letters, 3(2):957-964, 2018.
[36] Stephanic Lowry, Niko Sünderhauf, Paul Newman, John J Leonard, David Cox, Peter Corke, and Michael J Milford. Visual place recognition: A survey. ieee transactions on robotics, 32(1):1-19, 2015.
[37] Feng Lu, Baifan Chen, Xiang-Dome Zhou, and Dezhen Song. Sta-vpr: iipatiotteppore alignment for visual place recognition. IEEE Robotics and Automation Letters, 6(3):4297-43333 4304, 2021.
12

---

[38] Feng Lu, Shuting Dong, Lijun Zhang. Bingxi Liu, Xiangyuan Lan, Dongmei Jiang, and Chun Yuan. Deep homography estimation for visual place recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10341-10349, 2024.
[39] Feng Lu, Xiangyuan Lan, Lijun Zhang. Dongmei Jiang, Yaowei Wang, and Chun Yuan. Cricavpr: Cross-image vorrelation-aware representation learning for visual place recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024.
[40] Feng Lu, Lijun Zhang, Shuting Dong, Baifan Chen, and Chun Yuan. Aanet: Aggregation and alignment network with semi-hard positive sample mining for hierarchical place recognition. In IEEE International Conference on Robotics and Automation, pages 11771-11778, 2023.
[41] Feng Lu, Lijun Zhang. Xiangyuan Lan, Shuting Dong. Yaowei Wang, and Chun Yuan. Towards seamless adaptation of pre-trained models for visual place recognition. In The Twelfth International Conference on Learning Representations, 2024.
[42] Sven Middelberg, Torsten Satdler Ole Untzelmann. and Leif Kobbelt. Scalable 6-dof localization mobile devices. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part 13. pages 268-283. Springer, 2014.
[43] Michael Milford and Gordon F Wyeth. Seqslam: Visual route-based navigation for sunny summer days and stormy winter nights. In 2012 IEEE international conference on robotics and automation, pages 1643-1649) IEEE, 2012.
[44] Tayyab Naseer, Gabriel L Oliveira, Thomas Brox, and Wolfram Burgard. Semantics-wwaee visual localiza- tion under challenging perceptual conditions. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 2614-2620. IEEE, 2017.
[45] Eng-Jon Ong. Sameed Husain, and Miroslaw Bober. Siamese network of deep fisher-vector descriptors for image retrieval. arXiv preprint arXiv:7002.00333, 2017.
[46] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafranicc, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:230..07193, 2023.
[47] Florent Perronnin, Yan Liu, Jorge Sánchez, and Hervé Poirier. Large-scale image retrieval with compressed fisher vectors. In 2010 IEFE computer society conference on computer vision and pattern recognition, pages 3384-3391. IEEE, 2010.
[48] Filip Radenović, Giorgos Tolias, and Ondrcj Chum. Fine-tuning enn image retrieval with no human annotation. IEEE transactions pattern analysis and machine intelligence, 41(7):1655-1668 2018.
[49] Xun Sun, Yuanfan Xie, Pei Luo, and Liang Wang. A dataset for benchmarking image-based localization. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pages 7436-7444, 2017.
[50] Niko Sünderhauf, Peer Neubert, and Peter Protzel. Are we there yet? challenging seqslam on a 3000 km journey across all four scasons. page 2013, 2013.
[51] Niko Sünderhauf, Sareh Shirazi, Feras Dayoub, Ben Uperoft, and Michael Milford. On the performance of convnet features for place recognition. In 2015 IEFE/RST international conference on intelligent robots and systems (IROS), pages 4297-4304. TEEE, 2015.
[52] Janine Thoma, Danda Pani Paudel, and Lue Gool. Soft contrastive learning for visual localization. Advances in Neural Information Processing Systems, 11111119111111111111111111111111 2020.
[53] Akihiko Torii, Relja Arandjelovic, Josef Sivic, Masatoshi Okutomi, and Tomas Pajdla. 24/7 place recognition by view synthesis. In IEEE conference on computer vision and pattern recognition, pages 1808-1817, 2015.
[54] Akihiko Torii, Josef Sivie, Tomas Pajdla, and Masatoshi Okutomi. Visual place recognition with repetitive structures. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 883-890, 2013.
[55] Ruotong Wang, Yanqing Shen, Weiliang Zuo, Sanping Zhou, and Nanning Zheng. Transvpr: Transformer- based place recognition with multi-level attention aggregation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 13648 13657,, 2022.
13

---

[56] Xun Wang. Xintong Han, Weilin Huang, Dengke Dong, and Matthew R Scott. Multi-similarity loss with general pair weighting for deep metric learning. In Proceedings of the IEEE/CVF conference computer vision and puttern recognition, pages 5022-5030, 2019.
[57] Frederik Warburg, Soren Hauberg, Manuel Lopez-Antequera, Pau Gargallo, Yubin Kuang, and Javier Civera. Mapillary street-level sequences: A dataset for lifelong place recognition. Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 2626-2633. 2020.
[58] Zhe Xin, Yinghao Cai, Tao Lu, Xiaoxia Xing, Shaojun Cai, Jixiang Zhang. Yiping Yang, and Yanqing Wang. Localizing discriminative visual landmarks for place recognition. In 2019 International conference on robotics and automation (ICRA), pages
JEEE, 2019.
[59] Ming Xu, Niko Snderhauf, and Michael Milford. Probabilistic visual place recognition for hierarchical localization. IEEE Robotics and Automation Letters, 6(2): 1-318, 2020.
[60] Peng Yin, Lingyun Xu, Xueqian Li, Chen Yin, Yingli Li. Rangaprasad Arun Srivatsan, Lu Li, Jianmin Ji, and Yuqing He. A molli-domain feature learning method for visual place recognition. In 2019 International Conference on Robotics and Automation (ICRA), pages 319-324, TEEE, 2019.
[61] Jun Yu, Chaoyang Zhu, Jian Zhang, Qingming Huang. and Dacheng Tao. Spatial netvlad with weighted triplet loss for place recognition. IEEE transactions on neural networks and learning systems, 31(2):61 674, 2019.
[62] Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, et al. Florenee: new foundation model for computer vision. arXiv preprint or Xiv:22111. 1432, 2021.
[63] Lijun Zhang, Kangkang Zhou, Feng Lu. Xiang-Dong Zhou, and Shi. Deep semantic graph transformer multi-view 3d human pose estimation. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7205-7214, 2024.
[64] Quan Zhang. Xiaoyu Liu, Wei Li, Hanting Chen, Junchao Liu, Jie Hu, Zhiwei Xiong, Chun Yuan, and Yunhe Wang. Distilling semantic priors from sam to efficient image restoration models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 25409-25419, 2024.
[65] Yujie Zhong, Relja Arandjelović, and Andrew Zisserman. Ghostvlad for set-based face recognition. In Computer Vision ACCV 2018, pages 35-50, 2019.
[66] Kangkang Zhou, Lijun Zhang, Feng Lu, Xiang-Dong Zhou, and Yu Shi. Efficient hierarchical multi-view fusion transformer for 3d human pose estimation. In Proceedings of the ACM International Conference on Multimedia, pages 7512-7520, 2023.
[67] Sijie Zhu, Linjie Yang. Chen Chen, Mubarak Shah, Xiaohui Shen, and Heng Wang. R2former: Unified retrieval and reranking transformer for place recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19370-19380, 2023.
14

---

A Broader Impacts
Our research on VPR, particularly the development of the SuperVLAD and I-Cluster VLAD methods, has been motivated by the potential to significantly enhance applications in augmented reality, robot localization, and autonomous driving. The potential benefits and disadvantages depend on how people choose to use our method and how they handle failure cases.
Positive Impacts on Technology Advancement. The introduction of our novel method offers a more compact and generalizable feature representation, which could lead to more efficient and robust retrieval capability for downstream applications.
Potential Failure Cases. Although our method has shown high performance on evaluation metries, there remains small probability of failure cases, which could potentially lead people or autopilots astray. Nevertheless, suppose our method is used in conjunction with other methods and information for autonomous driving systems, then our method mostly introduces additional safety and efficiency to the overall system.
Potential Malicious Uses. Although our research better feature representation is foundational and not tied to particular applications, the VPR system may be exploited for invasive surveillance or social media monitoring, leading to privacy concerns. It is important to avoid unethical use of VPR studies for negative purposes.
B More Comparisons between NetVLAD and SuperVLAD
We have detailed the benefits of removing cluster centers in SuperVLAD in the main paper. This section provides more comparisons and analyses between NetVLAD and SuperVLAD, mainly analyzing the possible negative effects of removing cluster centers (i.e., the advantages of using cluster centers in NetVLAD) to illustrate the limitations (or characteristics) of SuperVLAD.
First, the cluster centers have the potential to play a guiding role in improving the accuracy of assigning local features to different clusters. Especially when local features are assigned to fine-grained clusters (i.e., large number of clusters), this strength of NetVLAD over Super VLAD will gradually become apparent. Although the DINOv2- performance comparable to a large number of clusters when training on the large-scale dataset GSV-Cities, it still makes sense to set a large number clusters for a simpler backbone and a smaller training dataset. Here we conduct complementary experiments using the CCT backbone and Pitts3kk training dataset. The results are shown in Table [ In this case, SuperVLAD has clear advantage over NetVLAD with small number of clusters, but this advantage gradually disappears with large number of clusters. That is, NetVLAD shows a more pronounced improvement with a larger number of clusters. However, when using large number of clusters, the memory occupied by the aggregation layer usually becomes significant in the training stage. The Super can significantly reduce GPU memory usage because it has only half the trainable parameters of NetVLAD. Moreover, from another perspective, SuperVLAI experiences less performance degradation with fewer clusters, indicating its good ability produce low-dimensional global features by directly reducing the number of clusters. Therefore, we do not consider this as limitation of SuperVLAD but rather as one of its characteristics (i.e., more suitable for small number of clusters than NetVLAD).
SuperVLAD
NetVLAD model with small number of clusters can achieve
Secondly, using the local features to subtract the cluster center to get the residual can more accurately capture the subtle differences of local features (improve the ability to distinguish details). However, when we use the transformer-buse local features (i.e., patch tokens), this can basically be ignored. So we use the transformer backbone for our SuperVLAD, which can be considered a limitation. We will discuss the use of CNN backbone in the next section.
backbone (especially the foundation model) to provide powerful
C Additional Comparisons Using the CNN Backbone
Although we have mentioned in the main paper that our SuperVLAD requires a transformer-based backbone to extract local features (i.e., patch tokens) of images, here we also present experimental results when applied a CNN backbone (specifically VGG16) in order to observe the limitations of our method. Considcring that previous works using NetVLAD to aggregate CNN feature maps
15

---

Table 8: Comparison of NetVLAD and SuperVLAD with different numbers of clusters using the CCT backbone and the Pitts30k training dataset. The smaller the number of clusters, the greater the advantage of SuperVLAD over NetVLAD. The values in parentheses show the change in results of SuperVLAD relative to NetVLAD: red for increase, green for decrease, and black for no change.
No.
Pitts30k
MSLS- val
Method
Cluster
R@1
R@5
R@ 10
R@l
R@5
R@ IO
NetVLAD
78.3
90.5
93.5
42.4
53.4
57.2
2
SuperVLAD
80.1(+18)
90.9(+0.4)
93.9(+0.4)
48.0(+5.6)
61.9(48.5)
65.5(+8.3)
NetVLAD
84.7
93.0
95.3
55.4
68.5
73.1
8
SuperVLAD
84.7(=0.0)
93.0(+0.0)
95.2(-0.)
(2.2(46.8)
73.1(+4.))
77.3(+4.2)
NetVLAD
85,4
93.3
95.3
62,0
72.6
77.2
64
SuperVLAD
85.1(:0.3
93.1(.22)
95.4(+0.1)
63.00+1.0)
73.5(40.9)
77.7(+0.5)
Table 9: Comparison of NetVLAD and SuperVI.AD using the CNN backbone (VGG16) with different numbers of clusters. We also provide the results of the corresponding DINOv2-based models as reference. All models are trained on GSV-Cities.
No.
Pitts3kk
MSLS-val
Method
Backbone
Cluster
R@
R@5
10
R@I
R@5
R@10
NetVLAD
92.3
96.8
97.7
91.6
96.2
96.8
DINOv2
4
SuperVLAD
92.6ca0.3)
96.4(-0.4)
97.5(.0.)
(2.2(+006)
95.9(-03)
96.8(.0.))
NetVLAD
91.9
96.1
97.3
92.3
96.2
96.6
DINOv2
64
SuperVI.AD
(2.5
(6.5(+
(7.7(+0.4)
(1.4(009)
95.8(.04)
96.5..o.)
NetVLAD
83.8
91.9
94.7
73.5
84.1
87.0
VGG16
4
SuperVLAD
84.5()0.7)
92.0(+0.1)
94.4(-0.)
76.4(+229)
85.7 6)
87.3(-0.3)
NetVLAD
87.6
93.5
95.4
80.7
88.4
90.3
VGG16
64
SuperVLAD
86.3(-13)
93.0
94.5(.0.)
(8.8(-199)
87.8(-06)
90.3(+0.0)
typically set large number of clusters (e.g., 64), we provide configurations with both 4 and 64 clusters. The experimental results are shown in Table g We also provide the results using DINOv2 as the backbone for reference. All models are trained on the GSV-Cities dataset. For the VGG16 backbone, can be observed that with only 4 clusters, our SuperVLAD also has an advantage over NetVLAD. However, under the more commonly used setting of 64 clusters, SuperVLAD lags behind NetVLAD. Although there is no significant difference in performance between 4 and 64 clusters when using the DINOv2 backbone, both NetVLAD and SuperVIAD perform significantly better with 64 clusters when using the CNN backbone. Considering that the CNN- (NetVLAD and SuperVLAD) models with 4 clusters do not perform well even using the large-scale GSV-Cities dataset for training, í.e., a large number of clusters is necessary in this case, we acknowledge that SuperVLAD is inferior to NetVLAD when using CNN backbone. Therefore, recommend its primarily with the transformer backbone (especially foundation models to provide powerful patch features).
D Comparisons on More Datasets
This section provides comparisons with SOTA methods (SelaVPR 41 and SALAD (7 on more datasets, shown in IO] Pitts250k and Tokyo247 are two datasets with large databases (more than 75k images), on which our method outperforms other methods. Baidu Mall [49 30 is the only indoor dataset, so there is clear gap between it and other datasets. Our work also achieves good results on this dataset. However, it should be noted that this is not the domain gap that we wanted to solve in our work. This because the objects that need to be paid attention to in indoor images are significantly different from those in outdoor images. Although different outdoor datasets may have domain gaps due to scenes (e.g., urban v.s. suburban) or conditions (i.e., illumination), the objects that need to be paid attention to are basically always buildings and vegetation, which is not true for indoor datasets (see Fig. 6 for some examples). Therefore, we think that the issue indoor VPR should be solved from other insights, rather than the idea of this work.
16

---

(a)
(b)
(c)
(d)
Figure 6: The example of data gap between different datasets (Pitts3kk, MSLS and Baidu Mall). Pitts30k is a dataset consisting of urban scene images, as shown in (a). MSLS contains urban and suburban (even natural) scene images, as shown in (b) and (c). There is an obvious domain gap between these two datasets, which is the issue our method attempts address. Besides, (d) shows an example from the Baidu Mall indoor dataset. There is another gap between indoor data and outdoor data, which is mainly caused by the fact that the objects that need be paid attention to in the two types of data are usually different.
Table 10:
Comparison to SOTA methods (SelaVPR and SALAD) on additional datasets. The inference batch size is 8 for Pitts250k and Tokyo247, and 4 for Baidu Mall.
Pitts250k
Tokyo247
Baidu Mall
Method
R@1
R @@5
R@10 10
R@1
R@5
R@
10
R@D 1
R@5
R@ 10
SelaVPR 41
95.7
98.8
99.2
94.0
96.8
97.5
66.6
79.6
85.9
SALAD 2
95.1
98.5
99.1
94.6
97.5
97.8
68.3
82.1
86.8
SuperVLAD
97.2
99.4
99.7
95.2
97.8
98.1
69.4
81.9
88.0
E
Additional Implementation Details
In the main paper, we have described the implementation details of training DINOv2- DINOV2-SuperVLAA models GSV-Cities for comparison with the SOTA methods. Here we introduce the details of the implementation of the ablation experiment. For the training of the models based on ViT and CCT backbones (and trained on MSLS/Pius30k), we follow some previous work [ 38). The last two transformer encoder layers of the ViT backbone (ViT-Base) are truncated and the size of the input image is 224x224. The last four transformer encoder layers of the CCT backbone (CCT-14) 14) are truncated and the layers before the third layer are frozen, using the input images with 384x384 pixels. For the training of VGG16-based models on GSV-Cities, the implementation is basically the same as the DINOv2-based model. However, the layers before the last block of the VGG16 backbone frozen, as in 19. ViT, CCT, and VGG16 are all pre-trained in ImageNet [13]. Following the visual geo-localization benchmark (9), we use the triplet loss to train models on Pitrs30k and MSL.S. Ten hard negative images are used in each triplet. The model training is performed using the Adam optimizer with a learning rate of 0. 0.00001. The batch size is set to 4. The training process on MSLS is terminated when the result does not improve for 6 epochs (40000 images/epoch), and that on Pitt33kk are 3 epochs (5000 images/epoch).
F
Additional Qualitative Results
Fig. in the main paper has presented a few qualitative results. As supplement, Fig. Fig. g Fig. and Fig. provvide additional qualitative results for MSLS, Pitts30k, Nordland, and SPED. Examples from these datasets highlight challenging scenarios including severe condition changes, viewpoint variations, dynamic interference, few or no landmarks, etc. Ours SuperVLAD achieves correct results, while others fail. Moreover, we also provide failure cases of our approach as shown in Fig. IT which suggests that future improvements of our method may need to consider how to retrieve images that are closer to the query image and improve robustness in natural scenes.
17

---

Query
SuperVLAD (Ours) NetVLAD
SFRS
CosPlace
EigenPlaces
SelaVPR
Figure 7: Qualitative results on the MSLS dataset. The first query lacks distinctive landmarks. The first two examples show significant viewpoint changes between the query and the correct reference image. For the third query, most of the other methods returned the same erroneous place due the inability to distinguish small-scale differences in the building surface. Only our method provides the correct result.
Query
SuperVLAD (Ours) NeVLLD
SPRS
CusPlace
EigenPlacess
SeI
Figure 8: Qualitative results on the Pitts30k dataset. In the second example, Although most of the image area the query image the sky, our SuperVLAD accurately returns the correct image using only small useful image area, whereas other methods return incorrect images. In the first and third examples, other methods retrieve images very similar to the query (still wrong), but our method can distinguish the different places through subtle differences, yielding the correct image.
Query
SuperVI.AD (Ours)
NetVLDD
SFRS
CosPlace
FigenPlaces
SelaVPR
Figure 9: Qualitative results on the Nordland dataset. These three queries contain discriminative landmarks and are subject to significant scasonal and lighting variations. The third example is particularly challenging, most of the image region is obscured by snow and clouds and there few discernible features along the road. Only ours SuperVLAM correctly returns the matched image.
18

---

Query
SuperVLAD (Ours)
NetVLAD
SFRS
CosPlace
EigenPlaocs
Sela
Figure 10: Qualitative results the SPED dataset. In the second example, daytime query images mislead other methods into retrieving method can return the right image night, with significant visual changes. Due low light, some discriminative objects are difficult to see clearly, and is hard to recognize the places even by human experts. The third example shows a snowy query image (without discriminative landmarks), causing other methods to return snowy or foggy images. Only our method finds the correct image of the same place in different weather conditions.
daytime images from different places. Our
Query
SuperVLAD (Ouns)
NetVLAD
SFRS
CosPlace
EigenPlaces
SelaVPR
Pitk20k
Pktt:3kk
MSLS
Nordland
SPED
MSLS
Figure 11: Failure Cases. For the first two examples (first two rows), our method (and some other methods) obtains database images that geographically close to the query image but the radius exceeds the set threshold (25m). This phenomenon usually occurs when the camera is too close or far away from the scene. For the third fifth examples, all methods fail because the images are captured in natural scenes without landmarks, which seems to be noteworthy challenge for current VPR methods. For the last challenging example (bad weather and large viewpoint changes), all global retrieval methods (including ours) fail, while VPR based on local feature re-ranking returns the correct result. This shows that even though our method achieves good results using only global features, it is possible to further improve the results by local feature re-ranking.
19

---

G
Dataset Details
MSLS [57]. The MSLS (Mapillary Street-Level Sequences) dataset is large-scale VPR dataset with over 1.6 million images collected in urban and suburban scenes from 30 cities on six continents, provided with GPS coordinates and compass angles. It covers many causes of environmental changes, such as illumination, weather, season, and viewpoint changes, as well as dynamic objects. The dataset contains three sets: training, public validation (MSISSvvll), and withheld test (MSSSS-challenge)) We evaluate models on the MSLS-va val set. It should be noted that the MSLS val set used in the Visual Geo-localization Benchmark [9] and CosPlace 7 contains different number of query images the official version 2 [57]. To facilitate comparison with previous methods, we use the official version and run CosPlace on obtaining better results than those reported in 77]
Pitts30k [54]. The Pitts30k dataset is generated from Google Street View panoramas with GPS labels available. It provides 24 viewpoint images at each location, with large niewpoint variations well as some dynamic objects. Pitts30k contains 10k database images in each of the training, validation, and test sets. Its test set is adopted to evaluate models in experiments.
Nordland [50. The Nordland dataset recorded from the same viewpoint the front of a train in four seasons. So, there are severe condition (e.g., season and weather) changes but viewpoint changes in this dataset. mainly contains suburban and natural place images. The ground truth of this dataset is offered by the frame-level correspondence. Following previous works [10) we use the summer images as the database and the winter images as queries, each containing 27592 images.
SPED [ The SPED dataset is made up of low- low-quality and high-scened segh--cene-depth images taken from CCTV cameras around the globe. The images in this dataset show various condition variations, such as lighting, weather, and seasonal changes. This dataset covers a diverse range of outdoor scenes, including forest landscapes, country roads, and urban environments. We use the SPED-test set for experiments, with 607 images in both its database and query set.
H Compared Methods Details
NetVLAD NetVLAD a classic VPR method with a learnable VLAD layer that is pluggable into any CNN architecture. We use the Py implementation with the VGG16 backbone trained on the Pitts30k dataset.
SFRS 21
This work mines hard positive samples using self-supervised image-to-region similarities for training a more robust NetVLAA model. We use the official implementation with the model trained on Pitts30k in the comparison experiment.
TransVPR (55 This work combines attentions from multiple levels of vision Transformer to yield global features for the candidate retrieval, then uses an attention mask to filter feature maps to get key-patch descriptors for cross- matching in re-ranking. We use the official implementation with the model trained on Pitts30k for the testing on the Pitts30k test set and the model trained on MSLS for others.
CosPlace This work casts the training of VPR as a classification problem and trains the VPR model on the San Francisco eXtra Large (SF-XL)) datasets. We use the official VGG16 model (with the 512-dim output feature) for testing.
VPR 7 This work introduces an innovative holistic feature aggregation approach, which utilizes feature maps extracted from the pre-trained backbone as global features. A stack of Feature- Mixer is employed to integrate global relationships into each feature map yield global descriptors of place images. MixVPR is trained on the GSV-Cities [1] dataset and we conduct comparisons using the best configuration (ResNet50 with 4096-dim output features).
https///ghhbbbcoom///eeee////ppp
htts://giiuboocommmmmmllllyyxyy
20

---

As an improvement of CosPlace, this work enhances the viewpoint robustness of learned global representations by training networks on images captured from various perspectives of the same places. FigenPlaces uses the CNN model as the backbone and attains superior performance on most VPR benchmark datasets. We adopt the official implementation and the best configuration based on the ResNet50 backhone to generate 2048-dim global descriptors for comparison.
EigenPlaces 10
SeIaVPR 41 The work introduces novel hybrid global-local adaptation method for seamlessly adapting the pre-trained foundation model to the VPR task. This method only fine-tunes lightweight adapters without altering the pre-trained backbone, and the fine-tuned model can yield both global and local features for two-stage VPR. Additionally, mutual nearest neighbor local feature loss is designed to train the local adaptation module. The produced dense local features are used in cross- spatial verification. The official implementation
matching for re-ranking, without with the model trained on Pitts30k used for testing on the Pitts30k test set, and the model trained on MSLS is used for others.
Crica VPR (39 This work presents cross-image correlation-aware representation learning method for VPR, which employs the attention mechanism to establish the correlation among multiple images within a batch. This method can leverage the cross-image variations cue guide representation learning, and when producing the feature of each image, it can obtain useful information from other image features to improve its robustness against viewpoint variations, condition variations, and perceptual aliasing. To further enhance robustness, multi-scale convolution -enoonneeeheeddddeeeeee adaptation method is proposed in CricaVPR, adapting the pre-trained foundation model DINOv2 to the VPR task. We used the official implementation and configuration for comparison.
SALAD 27 This method utilizes the pre-trained foundation model DINOv2 the backbone, and treats the soft assignment in NetVLAD as an optimal transport problem, solving it using the Sinkhorn algorithm. This soft assignment takes into account both the relationships between features and clusters, as well as between clusters and features. Additionally, it introduces a "dustbin" cluster, which is aimed at selectively discarding features identified as non-informative, thereby improving the overall quality of descriptors. We used the official pre-trained MINOv2-SALAR model for the evaluation.
http::/github.com http:///hhhbbccooomm////////////
https//gtthub.com///riibuuuuuu
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction of our paper explicitly state our contributions, including the development of SuperVLAD and 1 1-Cluster VLAD and their advantages over NetVLAD (and GeM). These claims are consistent with the empirical results presented, ensuring clear reflection of the scope and contributions of our work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Our paper includes dedicated "Limitations & Future Work" subsection where we thoroughly discuss the constraints of our approach.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate Limitations section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
22

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Our paper only involves a small amount of theory, which has been described in the Methodology section 3 and relevant references are also provided.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Our paper provides detailed formulations and descriptions of our proposed feature aggregation algorithms called SuperVLAD and I-Cluster VLAD in the Methodology section. We also provide implementation details in the Experiments section 4.2 and Appendix E In addition, open access our code and model checkpoints will be provided.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If contribution primarily new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
23

---

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The full codes and model checkpoints for reproducing methods will be publicly available upon paper publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cct
guuides/CodeSubmisse
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Our experimental setting is presented in Section 4.2 and Appendix D with im- portant hyperparameters included. Details for compared methods are included in Appendix
H
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[No]
Justification: Error bars are not commonly used in the VPR research.
Guidelines:
24

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report I-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We presented compute resources in Section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: /atee://////ii/////e http:////w....p..pp/aaa//aaaaaaa hctppppllccctiicccu..e..........
Answer: [Yes]
Justification:
Yes, is.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiciii)..))
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discussed both potential positive societal impacts and negative societal impacts of the work in Appendix A
25

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., dep.oymmly of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: Our model reached SOTA performance in the task of VPR and there is indeed a small possibility of unintended and malicious use. Nevertheless, our project is relatively not at high risk since we don't release new datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: In the paper have properly cited the original paper of the datasets. We respect the license of the referenced code and data and will properly acknowledge them in the project README.md md.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
26

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: Codes and model checkpoints for reproducing our methods will be publicly available upon paper publication.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper neither involves crowdsourcing nor research with human subjects.
Guidelines:
27

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "HsPbJSX7sp", "reviewer": "Reviewer_TmXY", "summary": "This work introduces SuperVLAD, a novel image descriptor for Visual Place Recognition. It eliminates the need for cluster centers, addressing NetVLAD's performance degradation due to data bias while being more lightweight. SuperVLAD generates compact descriptors with fewer clusters. Additionally, a 1-cluster VLAD method employs supernumerary ghost clusters for soft assignment, producing low-dimensional features that outperform same-dimensional GeM features or class tokens. Extensive experiments with transformer-based backbones demonstrate the effectiveness of this approach.", "questions": "1.\tHow do these new image descriptors integrate with visual localization and SLAM systems? Specifically, it would be beneficial to understand how SuperVLAD can improve the accuracy and robustness of these systems in real-world applications.\n2.\tPlease provide some efficiency results on SuperVLAD, such as inference time and training time. Additionally, information on the training data used would be helpful. Understanding the computational demands and data requirements is crucial for assessing the practical applicability of the method.\n3.\tTo attract readers in a machine learning conference, it would be valuable to offer general insights into how deep learning can contribute to spatial scene perception. Discussing the importance of this task to the machine learning community and the broader implications of improving visual place recognition through advanced descriptors like SuperVLAD would enhance the paper's appeal.", "limitations": "please see my questions. Since this work is within a subarea in CV, please provide more insights into how ML can impact this area and how this application is important to ML.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This work is well-written and easy to follow. The method and framework are technically robust and novel, offering a fresh perspective on image descriptor design for Visual Place Recognition. SuperVLAD achieves state-of-the-art performance across several place recognition benchmarks, demonstrating its effectiveness and competitiveness in the field. By eliminating the need for cluster centers, SuperVLAD addresses NetVLAD's performance degradation due to data bias while being more lightweight, which is crucial for real-time applications and resource-constrained environments. SuperVLAD generates compact descriptors with fewer clusters, leading to reduced memory and computational requirements. The introduction of a 1-cluster VLAD method employing ghost clusters for soft assignment is a novel approach, producing low-dimensional features that outperform same-dimensional GeM features or class tokens. Extensive experiments with transformer-based backbones highlight SuperVLAD's versatility and adaptability, leveraging state-of-the-art architectures to enhance visual place recognition performance. The method's broad applicability, including in autonomous navigation, robotics, and augmented reality, makes it a valuable contribution to the field. Additionally, the paper provides a comprehensive evaluation of SuperVLAD's performance, validating its effectiveness and establishing a solid foundation for its claims.", "weaknesses": "Provide more technical and experimental details. Please see my questions below The paper could benefit from more technical and experimental details. Please see my questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive recommendation and encouraging words. The following are Responses to the Questions.\n\n**Q1. Integrate SuperVLAD with visual localization and SLAM systems:**\n\nSince SuperVLAD is a general visual place recognition (VPR) method, it can be used for visual localization and SLAM systems like previous standard VPR methods such as NetVLAD. \nFor SLAM systems, our SuperVLAD can be used as the loop-closure detection method to rectify the accumulated mapping error. As you know, our SuperVLAD is more lightweight and can produce compact descriptors, so it is more suitable for resource-constrained environments like mobile robots SLAM. Given the good recognition performance of SuperVLAD, it may not require spatial consistency verification in loop-closure detection, which requires us to further explore its recall performance at 100% precision in future work. For visual localization, it usually only requires the VPR method to retrieve top-N candidate images (obtain location hypotheses), then perform local matching and compute 6-DoF pose [1]. Our experiments have shown that SuperVLAD can achieve good Recall@N performance and is therefore suitable for visual localization. \n\n**Q2. Experimental details: inference time, training time, and training data:**\n\nThe experimental details including training data (i.e., GSV-Cities) and some other settings are described in Section 4.2 and Appendix D. The inference time is 13.5ms for a single image, as shown in Section 4.3 (Figure 5). The training time of our SuperVLAD is 81.6 minutes (training for 7 epochs on GSV-Cities) using two NVIDIA GeForce RTX 3090 GPUs, compared to 210 minutes for CricaVPR [2], which uses the same backbone and training dataset as ours. We will add the training time to the final paper.\n\n**Q3. How this application is important to ML, how ML can impact this area and what can SuperVLAD bring:**\n\nThis is a very meaningful question. We should think more about the impact that the VPR task and our method (SuperVLAD) can bring to the ML community.\n\n-\tFirst, the VPR research is a foundational part of studying how machine learning models perceive, understand, and represent geographic space.\n-\tSecond, the emergence of better VPR methods/models requires unifying the efforts of the machine learning, computer vision, and robotics communities. Especially when a novel machine learning model is proposed, it usually brings new solutions to the VPR task.\n-\tLast, as a type of aggregation algorithm, VLAD-related methods can not only be used to aggregate image local features, but are also widely used for video sequences aggregation [3][4], speech clustering [5], and multi-modal information processing [6][7]. As a pure aggregation algorithm, our SuperVLAD does not add any prior information related to the VPR task. It also has the potential to be applied to other fields to aggregate other data. From this perspective, it is not just a vision algorithm but can be viewed as a machine learning algorithm. As for the broader implications of improving VPR through our SuperVLAD, we have discussed it in Appendix B.\n\nThanks again for your meaningful questions.\n\n**References**\n\n[1] Sarlin, Paul-Edouard, et al. \"From coarse to fine: Robust hierarchical localization at large scale.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.\n\n[2] Lu, Feng, et al. \"CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[3] Xu, Youjiang, et al. \"Sequential video VLAD: Training the aggregation locally and temporally.\" IEEE Transactions on Image Processing 27.10 (2018): 4933-4944.\n\n[4] Naeem, Hajra Binte, et al. \"T-VLAD: Temporal vector of locally aggregated descriptor for multiview human action recognition.\" Pattern Recognition Letters 148 (2021): 22-28.\n\n[5] Hoover, Ken, et al. \"Putting a face to the voice: Fusing audio and visual signals across a video to determine speakers.\" arXiv preprint arXiv:1706.00079 (2017).\n\n[6] Wang, Xiaohan, Linchao Zhu, and Yi Yang. \"T2vlad: global-local sequence alignment for text-video retrieval.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.\n\n[7] Wang, Hui, et al. \"Pairwise VLAD interaction network for video question answering.\" Proceedings of the 29th ACM international conference on multimedia. 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TUGlo4IIJV", "reviewer": "Reviewer_utGd", "summary": "The paper introduces SuperVLAD for addressing Visual Place Recognition (VPR) task. The proposed SuperVLAD descriptor is an improvement to NetVLAD descriptor, by reducing the dimensionality, by using fewer clusters and", "questions": "1. Results in Table 2 (NetVLAD) don't match the results in SALAD paper ([27] -- also wrongly cited as [40] in Tab 2). Which are the correct ones?\n2. How would SuperVLAD perform if the backbone was ResNet-50? ( to make it easier to compare with existing methods, like MixVPR[2])\n\n(minor) remark: adding Venue column in Tab 2 doesn't add any value to the evaluation -- might be worth using the space to add the backbone for the specific method, to make comparison easier.\n\n3. What are the total number of parameters and number of trainable parameters of the methods compared in Tab2?", "limitations": "Somewhat unclear ablation -- information is present, but not aggregated, making it difficult to understand the contribution of individual components (e.g. removing cluster centers, reducing the number of clusters -- this could be a nice plot / individual rows in a table), adding ghost clusters, changing backbone -- i.e. highlighting the contribution of each change individually.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Simple but effective method, reducing the number of parameters in VLAD (by reducing the number of clusters and removing the cluster centers influence).\n* In dept comparison with other (especially recent methods)\n* Achieving results on-par with NetVLAD (Tab 3) with fewer parameters.", "weaknesses": "1. It is a bit difficult to understand what contributes to the results; there is a table (Tab 6) comparing the influence of number of clusters, and comparison on backbones (e.g. Tab 3). It would be nice to have a number of clusters column in Tab3 - to show the influence of number of clusters for a given backbone, i.e. help understand where the improvements come from.\nFrom Tab 3 -- the results, on the same backbone, seem on par with NetVLAD -- it might be worth emphasizing that this is achieved with fewer parameters, due to the smaller number of cluster centers.\n2. What is the impact of discarding cluster centers? It is not clear from the text / tables if discarding the cluster centers influences the results or not.\n3. Evaluation on SPED dataset (lower quality images) might benefit even more from adding ghost cluster (Tab 4).\n4. Missing evaluation on Pittsburgh-250k and Tokyo 24/7 -- How does SuperVLAD perform for changing environment conditions, such as day-night?\n5. MixVPR achieves comparable results on Pitts30k -- with a much smaller architecture (ResNet-50). In Tab2 -- no mention on what data are the models trained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your insightful comments and valuable suggestions. The following are Responses to the Questions (denoted as **Q**) and Weaknesses (denoted as **W**).\n\n**Q1. Results of NetVLAD in Table 2:** \n\nBoth of our results and the results in the SALAD paper are correct. We use the original version of NetVLAD as described in Appendix G (also can be seen in NetVLAD paper), i.e., VGG16 backbone and trained on Pitts30k. The results of DINOv2-based NetVLAD trained on GSV-Cities (basically the same as in SALAD) are shown in Table 3 and Table 7 of our paper, which basically matches the results in SALAD. Thank you for pointing out our wrong citation in Table 2.\n\n**Q2. SuperVLAD perform with ResNet-50:** \n\nOur SuperVLAD with ResNet-50 cannot outperform MixVPR. To take full advantage of SuperVLAD we need to use the Transformer-based backbone, which is a limitation of SuperVLAD and has been discussed in Appendix A (Limitations) and Appendix C (Results with VGG16). Considering that more and more VPR methods are using Transformer models and even the DINOv2 model, we can also use the DINOv2 backbone uniformly for fair comparison. There is a DINOv2-based MixVPR proposed, i.e. DINO-Mix [1], and SuperVLAD outperforms DINO-Mix with an obvious margin.\n\n| Method |  Pitts250k | Pitts30k | Tokyo24/7|\n|:---|:---:|:---:|:---:|\n|DINO-Mix   | 94.6 | 92.0 | 91.8|\n|SuperVLAD  | 97.2 | 95.0 | 95.2|\n\n**Q3. Total number of parameters and trainable parameters:** \n\nWe appreciate your attention to the details in Table 2. It's important to note that the methods compared in this table use different backbones, which may impact the direct comparability of the number of parameters. To address this concern, we provide a comparison of SALAD and SuperVLAD that both use the DINOv2-base backbone and only finetune the last four Transformer blocks of the backbone. The value in parentheses is the number of parameters in the optional across-image encoder.\n\n| Method |  Total (M)  |  Trainable (M)  |  aggregator (M)|\n|:---|:---:|:---:|:---:|\n|SALAD     |   88.0        |  29.8       |   1.4 |\nSuperVLAD  |  86.6 (+11.0)  |  28.4 (+11.0) |  0.0038|      \n\nThe number of parameters in the aggregator is the most important consideration. The number of parameters in our aggregator is much lower than that of SALAD (less than 3/1000). As far as we know, the SuperVLAD aggregator has a smaller number of parameters than any other common methods except global pooling methods (e.g., GeM).\n\n**W1. About the readability of the results:**\n\nThanks for your suggestion to improve the readability of our paper. We will improve it in the final paper. \nWe have introduced the number of clusters in the table caption of Table 3 and we will move it to the table. For the same backbone, the improvement of SuperVLAD mainly exists when there is a gap between the training data and the test data (e.g., training on Pitts30k and testing on MSLS, see the reply to **W2** for more details), which is also fits our motivation. In addition, we use fewer parameters than NetVLAD not only because of the smaller number of clusters, but also by removing the cluster centers, which reduces the parameters of NetVLAD by about half even with the same number of clusters. We will emphasize this point as you suggested.\n\n**W2. The impact of discarding cluster centers:**\n\nDiscarding cluster centers can improve robustness when there is a domain gap between the data in training and inference. As shown in Table 3, when the models trained on Pitts30k (only urban images) are tested on MSLS (including urban, suburban, and natural images), our SuperVLAD outperforms NetVLAD. More specifically, discarding cluster centers brings an absolute R@1 improvement of 6.8% (using CCT backbone) and 2.4% (using DINOv2) on MSLS-val. However, for the models trained on the GSV-Cities dataset, there is little difference in results between SuperVLAD and NetVLAD because GSV-Cities covers diverse training data (i.e., no obvious gap between training and test data). Please refer to Lines 312-324 in the paper for more details.\n\n**W3. Adding the evaluation on the SPED dataset in Tab 4:** \n\nThe results on SPED with or without (denoted as “-ng” suffix) ghost cluster are shown in the following table. There is no obvious difference between them. Considering that it can eliminate useless information and bring improvements on some datasets, we still choose to keep it. More importantly, we use it to achieve the 1-cluster VLAD.\n\n| Method |  R@1  |  R@5  |  R@10 |\n|:---|:---:|:---:|:---:|\n|CCT-SuperVLAD-ng     | 64.7 |79.1 |84.2|\n|CCT-SuperVLAD        | 64.7 |78.6 |83.9|\n|DINOv2-SuperVLAD-ng  | 89.8 |94.7 |95.9|\n|DINOv2-SuperVLAD     | 90.1 |95.6 |95.9|\n\n**W4. Evaluation on Pittsburgh-250k and Tokyo 24/7:**\nThe results on Pitts250k and Tokyo24/7 are shown in the following table. SuperVLAD also achieves SOTA results, showing good performance in day-night changes.\n\n|Method   | Tokyo24/7 | | | pitts250k | \n|:---|:---:|:---:|:---:|:---:|\n|SelaVPR    |  94.0 / 96.8 / 97.5    | | |  95.7 / 98.8 / 99.2|\n|SALAD     |  94.6 / 97.5 / 97.8  | | |  95.1 / 98.5 / 99.1|\n|SuperVLAD |  95.2 / 97.8 / 98.1   | | | 97.2 / 99.4 / 99.7|\n\n**W5. a. About MixVPR with ResNet50 and b. training data of models in Table 2:**\n\n-\ta. Please see the response to **Q2**. \n-\tb. The training data for MixVPR, CricaVPR, SALAD and our SuperVLAD is GSV-Cities (as described in Section 4.3), and for other methods is described in Appendix G.\n\nThanks again for your thoughtful review, and please let us know if you have further questions.\n\n**References**\n\n[1] Huang, Gaoshuang, et al. \"Dino-mix: Enhancing visual place recognition with foundational vision model and feature mixing.\" arXiv preprint arXiv:2311.00230 (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FzCibD3ILQ", "reviewer": "Reviewer_wUjJ", "summary": "In this paper, the authors focus on reducing the dimension of NetVLAD method for the task of visual place recognition. More specifically, the proposed method named SuperVLAD combines previous techniques including the powerful DIVOv2 as backbone, free from clusters, and ghost clusters. Experiments demonstrate that the proposed method outperforms pervious methods on Pitts30k, MSLS-val, Nordl, and SPED datasets. Moreover, the extensive ablation studies are conducted the verify the effectiveness of different components.", "questions": "NA", "limitations": "Yes", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tGood performance. Although the proposed method is mainly based on previous techniques, such as a more powerful backbone, ghost clusters, cross-image interaction, et al, the proposed method outperforms almost all previous methods on four public datasets. This sets a new benchmark and is helpful in real applications.\n\n2.\tExtensive ablation studies. In addition to the comparisons to previous approaches, the authors conduct extensive ablation studies on the backbone, training set, ghost cluster, pooling strategy, and the number of the clusters. These experiments help the readers understand which techniques contribute most to the improvements against other methods.\n\n3.\tThe paper is well written and easy to read.", "weaknesses": "My concern is about the comparison between the proposed SuperVLAD and NetVLAD. \n\n1.\tAccording to the Table 7 in the appendix. Table 7 shows that with DINOv2 as backbone, when increasing the number of clusters from 4 to 64, NetVLAD does not improve the performance. Conversely, it loses the accuracy. This is not consistent with the claim of the paper that the proposed approach reduces the dimension of features given by NetVLAD as even with very smaller number of clusters, NetVLAD also has very good performance as SuperVLAD does. I am very curious what will happen if the number of clusters for NetVLAD and SuperVLAD is set to 1, 2, 3.\n\n2.\tAccording to Table 4 and 6, the ghost cluster does not improve the performance when a the powerful DINOv2 is used as the backbone. Cross-image encoder contributes most to the improvements. It would be great to show the results of NetVLAD with 4 clusters, DINOv2, and cross-image encoder. Then we will know how many improvements come from other techniques used in SuperVLAD.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments and suggestions. We hope the following clarifications will be able to address your concerns. \n\n**W1. About the number of clusters:**\n\nSorry to confuse you. \n\n-\tFirst, since we did not specifically emphasize that as the capabilities of the backbone model improve (e.g., DINOv2) and the amount and quality of training data increase (e.g., GSV-Cities), both NetVLAD and SuperVLAD can achieve excellent performance even with a small number of clusters, this may lead readers to misunderstand that it is unique to SuperVLAD. We will improve it in the revised paper. However, when using simple models or smaller training datasets, the performance of a small number of clusters is not as good as that of a large number of clusters, and our SuperVLAD can outperform NetVLAD with a small number of clusters, as shown in Table 7. Besides, to our best knowledge, our work is the first attempt to set the number of clusters to be very small (only 4).\n\n-\tSecond, for setting the number of clusters to 1, 2, and 3, we provide the following explanations and experimental results. Setting the number of clusters to 1 will make the softmax layer for soft assignment meaningless. Our 1-cluster VLAD is achieved by adding additional ghost clusters, and there is not just one cluster during soft assignment. In fact, if there is only one cluster, all features are assigned to one category and the weight is \"1\". So SuperVLAD directly sums up all local features, which is equivalent to global average pooling (multiply by a constant), while NetVLAD subtracts a constant vector from the SuperVLAD vector (equivalent to translating the feature space), which does not affect the similarity calculation, i.e., the retrieval results are consistent. As for setting 2 or 3 clusters, the conclusion is the same as that in Table 7. That is, for using the powerful backbone (DINOv2) and the large-scale GSV-Cities dataset, the recognition performance of SuperVLAD and NetVLAD is nearly equal (but our SuperVLAD is simpler and more lightweight). However, for training on smaller datasets like Pitts30k, or/and using the simpler backbone like CCT, our SuperVLAD can outperform NetVLAD on recognition performance with an obvious margin. The detailed results (R@1/R@5/R@10) are shown in the following tables. Both NetVLAD and SuperVLAD are without ghost clusters and the cross-image encoder for fair comparison.\n\n①\t When training DINOv2-based models on GSV-Cities, both NetVLAD and SuperVLAD achieve good results without obvious differences on the R@N metric.\n|2 clusters   | Pitts30k | | | MSLS-val | \n|:---|:---:|:---:|:---:|:---:|\n|NetVLAD    | 91.8 / 96.1 / 97.3 | | | 91.5 / 96.5 / 96.9|\n|SuperVLAD  | 92.4 / 96.6 / 97.5 | | |  91.2 / 96.2 / 96.8|\n\n|3 clusters   | Pitts30k | | | MSLS-val | \n|:---|:---:|:---:|:---:|:---:|\n|NetVLAD    | 91.4 / 96.2 / 97.5  | | |  91.8 / 95.8 / 96.8| \n|SuperVLAD  | 92.1 / 96.7 / 97.8  | | |  91.9 / 95.9 / 96.2|\n\n   ② When training CCT-based models on Pitts30k, SuperVLAD outperforms NetVLAD, with an obvious margin on Pitts30k and an even larger margin on MSLS (with 12.3% absolute R@1 improvement on MSLS-val with 3 clusters). This shows that our SuperVLAD performs better than NetVLAD with a simple backbone and small training dataset, i.e., the lower limit of performance is higher than NetVLAD. Besides, it once again verifies that SuperVLAD is more robust than NetVLAD when there is a domain shift in training data and test data (i.e., training on Pitts30k and testing on MSLS). By the way, 2 and 3 clusters get obviously worse results than 8 clusters (see Table 3 in paper).\n\n|2 clusters   | Pitts30k | | | MSLS-val | \n|:---|:---:|:---:|:---:|:---:|\n|NetVLAD    | 78.3 / 90.5 / 93.5  | | |  42.4 / 53.4 / 57.2|\n|SuperVLAD  | 80.1 / 90.9 / 93.9  | | | 48.0 / 61.9 / 65.5|\n\n|3 clusters   | Pitts30k | | | MSLS-val | \n|:---|:---:|:---:|:---:|:---:|\n|NetVLAD   |  79.5 / 90.7 / 93.9  | | |  43.1 / 55.9 / 60.7|\n|SuperVLAD  |  81.8 / 91.4 / 93.8 | | |  55.4 / 67.8 / 72.2|\n\nFrom the above results, we can summarize the advantages of SuperVLAD as follows:\n\na. When using a powerful backbone and a large-scale training dataset, SuperVLAD achieves the nearly same performance as NetVLAD using fewer (about half) parameters.\n\nb. When using a simple backbone and a small training dataset, SuperVLAD can outperform NetVLAD by an obvious margin.\n\nc. When there is a domain gap in training data and test data, SuperVLAD is more robust than NetVLAD.\n\n**W2. Results of NetVLAD with 4 clusters, DINOv2, and cross-image encoder:**\n\n-\tFirst, we agree with you that the ghost cluster does not improve the performance when we use the powerful DINOv2 backbone. Considering that the ghost cluster can bring improvement (by eliminating useless information) when using a simple backbone (as shown in Table 4 of our paper), we adopted it in SuperVLAD.\n-\tSecond, we also agree with you that showing the results of NetVLAD with 4 clusters, DINOv2, and cross-image encoder can better analyze the improvements from other techniques used in SuperVLAD. The results (trained on GSV-Cities) are shown in the following table. After adding other techniques to both SuperVLAD and NetVLAD (denoted as NetVLAD*), SuperVLAD can outperform NetVLAD on recognition performance, i.e., SuperVLAD is more suitable for adding these technologies. (We have shown in our paper that the recognition performance of DINOv2-based SuperVLAD and NetVLAD is comparable without adding other technologies, in which case the advantage of SuperVLAD is simpler, more lightweight, and robust to domain gap.)\n\n|Method   | Pitts30k | | MSLS-val | | Nordland | | SPED| \n|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|NetVLAD*  |93.8 / 97.0 / 98.0 | | 91.1 / 96.8 / 97.3 | | 89.6 / 95.9 / 97.4 | | 92.6 / 96.7 / 97.5|\n|SuperVLAD  |95.0 / 97.4 / 98.2 | | 92.2 / 96.6 / 97.4 | | 91.0 / 96.4 / 97.7 | | 93.2 / 97.0 / 98.0|\n\nThanks again for your thoughtful review, and please let us know if you have further questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W7vB5RIJnv", "reviewer": "Reviewer_7MN7", "summary": "The authors proposed a new method called SuperVLAD for visual place recognition. The authors aims to fix the shortage of the previously mature NetVLAD, which is NetVLAD is not robust against domain gap and have to use high dimensional features. \nSuperVLAD reduces the number of \"clusters\" used to smaller value and also propose a low-dimensional 1-Cluster VLAD descriptor, comparable in dimension to GeM pooling but with better performance. Experimental results show that SuperVLAD, when combined with a transformer-based backbone, outperforms NetVLAD, achieving better domain generalization and surpassing state-of-the-art methods on benchmark datasets.", "questions": "There are inconsistencies in dataset usage between experiments in 4.3 (table 2) and generalization tests (table 3). Why do the two tables report accuracy from different set of datasets?", "limitations": "The performance improvement mostly comes from the DINOv2 backbone, as explained in section 4.3. Table 3 shows that if NetVLAD uses the ViT backbone, the performance gap between NetVLAD and SuperVLAD is less obvious. The authors also confirmed such shortcomes in Appendix A. The SuperVLAD no longer outperformed the NetVLAD in commonly used settings. \n\nRemaining suggestions not affecting rating:\nIt would be better to note the backbone features (e.g. DINOv2) in table 2. \nWhen explaining the difference in scene coverage between datasets in the ablation study, it would be useful to include examples of the dataset scenes so that readers can visualize the data domain.", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The work made a small modification on the formulation of NetVLAD. The intuition is based on demonstrations in figure 1. \nThe number of learnable parameters is reduced by half, because finding the centroid ck is no longer a requirement. \nThe work is compatible with GhostVLAD which uses so-called ghost clusters to absorb useless scene elements. \nThe work compares with several new methods including ones from CVPR24. Its feature dimensionality used is the relatively lower ones while having the highest recall, which verifies one of the claims. \nResource consumption is reasonable.\nLimitation and ablation analysis is comprehensive.", "weaknesses": "1. The novelty of the method is limited. It is a modification of the mature NetVLAD method and the modification appears to be small. \n2. Performance improvement is less obvious considering table 2. SuperVLAD shows minor differences between the second best method. \n3. To support the generalization claim, the authors need to show more comprehensive comparisons. How will the method perform if being used in dramatically different environments (such as training indoors but testing outdoors)? Indoor environments could be ScanNet or ETH3D, etc. So far, there isn’t an obvious domain gap between the datasets used in the generalization tests, and the generalization ability claim could be better justified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive recommendation and insightful comments. The following are Responses to the Question (denoted as **Q**) and Weaknesses (denoted as **W**).\n\n**Q. Inconsistencies in datasets between Table 2 and Table 3:** \n\nSorry to confuse you. We compare our SuperVLAD with other methods on four benchmark datasets, as shown in Table 2. However, considering that we have a large number of ablation experiments, all ablation experiments, i.e., Table 3,4,5,6,7, are only conducted on the two most commonly used datasets (Pitts30k and MSLS) to show the performance more clearly. Besides, Pitts30k and MSLS have their own training and test/val sets, so we can simply use cross-validation (training on Pitts30k and testing on MSLS, and vice versa) to demonstrate the ability of SuperVLAD to address domain shift.\n\n**W1. The modification to NetVLAD:**\n\nWe agree with you that NetVLAD is a mature method. However, the robustness against domain shift is still an important issue to be addressed. Our SuperVLAD is reasonably motivated and gets good results. Different from previous improved methods that usually complicate NetVLAD, our improvements simplify it to enhance the generalizability across different domains (the insight is novel). That is, our SuperVLAD is a simple yet effective method.\n\n**W2. The performance improvement:**\n\nAlthough the recognition performance improvement of our SuperVLAD over the previous SOTA method is not very obvious, our method is more lightweight than SOTA methods like SALAD. The number of parameters in the SuperVLAD aggregator (0.0038M) is less than 3/1000 that of the SALAD aggregator (1.4M). The output descriptor dimension of SuperVLAD (3072) is less than 1/2 that of SALAD (8448). \n\n**W3. The experiments on the indoor dataset:**\n\nThanks for your suggestion. We use the trained models (on GSV-Cities) to test on the indoor dataset Baidu Mall [1][2] to support the generalization claim. SuperVLAD achieves better results than NetVLAD (DINOv2-based), as well as other SOTA methods on this dataset (see the following two tables), which further supports our claim.\n\n| Method | R@1 / R@5 / R@10|\n|  :----:  | :----: |\n|NetVLAD |68.5 / 81.2 / 86.5|\n|SuperVLAD |69.5 / 82.7 / 87.3|\n\n| Method | R@1 / R@5 / R@10|\n|  :----:  | :----: |\n|CricaVPR |61.3 / 78.5 / 85.9|\n|SelaVPR |66.6 / 79.6 / 85.9|\n|SALAD |68.3 / 82.1 / 86.8|\n|SuperVLAD |69.5 / 82.7 / 87.3|\n\n**For the suggestion in Limitations:**\n\nThanks again for your suggestion. We provide examples of datasets to show the data domain gap between Pitts30k and MSLS in the attached PDF (Figure 1). We will add it to our final paper.\n\n**References**\n\n[1] Sun, Xun, et al. \"A dataset for benchmarking image-based localization.\" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.\n\n[2] Keetha, Nikhil, et al. \"Anyloc: Towards universal visual place recognition.\" IEEE Robotics and Automation Letters (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DMsPNnSPtP", "reviewer": "Reviewer_w8L8", "summary": "The paper addresses the problem of learning a global image descriptor suitable for visual place recognition.\nThe work builds on top of NetVLAD, i.e. performs a soft assignment of local descriptors to learnable clusters, with the main difference that the model directly aggregates the local descriptors rather than residuals per cluster. This independence of the global embedding from cluster centers is argued to increase the robustness to domain shifts. \n\nThe retrieval performance is increased by the incorporation of related works, namely ghost clusters [61] that allow to ignore ambiguous local descriptors, and a cross-image transformer within a training batch [39] to obtain viewpoint stability.\nWith DINOv2 as backend features, the approach shows SotA performance, comparable to recent works from CVPR'24.\n\nExperimental results demonstrate that only 4 clusters are sufficient, leading to a compact descriptor. The authors also propose a 1-cluster VLAD (which can be seen as a binary classification of local descriptors useful/non-useful) resulting on a 768 dim. descriptor and SotA performance compared to using GeM pooling [47] or the ViT/DINOv2 class token (both of dim. 768)", "questions": "1) In the comparison in Table 5, what do you refer to as \"class token\"? If it's the ViT class token, is a fine-tuned on the task?\n\n2) Figure 5 misses a point for CircaVPR, which has a Recall@1 of 94.9. What is its inference time?\n\n3) To be able to compare all approaches 1:1 it would have been useful to use the same feature dimensionality for all of them, e.g. via training a final FC layer. Have experiments in this direction been conducted?", "limitations": "Limitations addressed adequately in the appendix. This discussion should be moved to the main paper.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "**S1** The approach demonstrates SotA retrieval performance at smaller global descriptor size (Table 2), while maintaining a comparable inference time. \n\n**S2** The experimental evaluation of the approach is extensive and provides valuable comparisons to other SotA methods. Ablation studies document the influence of chosen architecture designs and the influence of different feature backends and training datasets well. \nThe proposed approach could have been presented more \"powerful\" be hiding some of those insights, e.g. Tab. 3 and Tab. 4. I appreciate this. \n\nNote: The good performance at small descriptor size, and thus its wider applicability, is the main reason why I lean towards acceptance. Otherwise the technical contribution and novelty is small.", "weaknesses": "**W1**: The proposed architecture is motivated by the missing capability of NetVLAD to generalize well across domains (line 51ff). Table 3 tries to make this point, and shows that SuperVLAD performs better than NetVLAD on MSLS when trained on Pitts30k. Though, the reported improvement of SuperVLAD over NetVLAD of 5pp is negligible when DINOv2 is used as backbone (0-2pp). When NetVLAD is trained on a more versatile dataset (GSV-Cities) its performance improves by ~30pp and the gap between SuperVLAD and NetVLAD becomes negligible. This demonstrates that usage of a large scale, and versatile training data set is much more important than the modified local descriptor aggregation (one of the main contributions of the paper.)\n\n[The remaining contribution is that SuperVLAD results in a more compact descriptor.]\n\n**W2** The novelty of the proposed work is limited. The main notable difference to previous work is the direct aggregation of descriptors rather than residuals per cluster. Otherwise the approach is very close to NetVLAD and borrows the ideas from [61] for \"ghost clusters\" and  [39] for cross-image interaction to obtain more robust features. To achieve SotA performance ghost clusters are not relevant (Tab. 4) but the cross-image encoder is required to achieve the performance of [39] (which is fair) and [40] \n\n**W3** The cherry picked results in Figure 4 do not provide much value, because such a matrix could be constructed in favor of any of the approaches. An illustration of typical failure cases of the proposed approach would be more helpful for the reader to understand the limitations of the approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive recommendation and constructive comments, as well as the recognition of our compact descriptor (especially for 1-cluster VLAD). We will move the limitations discussion to the main paper as you suggested. The following are Responses to the Questions (denoted as **Q**) and Weaknesses (denoted as **W**).\n\n**Q1. About “class token”:**\n\nThe “class token” is output by the DINOv2 model, which can also be called ViT class tokens (DINOv2 is a ViT model trained on the large-scale curated LVD-142M dataset). It is also finetuned on GSV-Cities (VPR dataset). That is, all methods in Table 5 have the same settings (backbone, training setting, etc.) except for the descriptor, to compare the performance of descriptors fairly. \n\n**Q2. Inference time of CricaVPR:**\n\nThe inference time of CricaVPR is 17.2ms (without PCA), which is slower than our SuperVLAD (13.5ms). We will add it to Figure 5. Besides, CricaVPR uses the optional PCA operation for dimensionality reduction, which requires more inference time if PCA is added.\n\n**Q3. Using the same feature dimensionality to compare via a final FC layer:** \n\nWe agree with you that adding the final FC layer allows different methods to output features of the same dimensionality. However, the recent DINOv2-based works (e.g., SALAD and CricaVPR) do not do this. If we directly add a final FC layer to retrain the models, it will involve the issue of optimal training parameter setting, which may make the result much worse than the original works. In this case, it is also unfair to the other works. By the way, our method aims at producing compact descriptors without additional dimensionality reduction/ transformation techniques. So, we do not add a final FC layer, which ensures that our aggregation module is lightweight.\n\n**W1: Large-scale training dataset vs. the modified aggregation method**\n\nWe agree with you that the large-scale training dataset can bring more improvements than the modified aggregation method for VPR in most cases. However, a robust aggregation algorithm is important when large training datasets are lacking. It is helpful to improve the performance floor in bad cases.\n\n**W2. The novelty of the proposed method:**\n\nIt's important to note that NetVLAD is a very successful method but its domain generalization performance is still hampered by cluster centers. Our improvement is reasonably motivated and can address this import issue (also gets good results). Unlike previous improvements that usually complicate NetVLAD, our improvements simplify it by directly aggregating local features assigned to each cluster, which no longer computes residuals and is therefore free from the cluster centers (the insight is novel). That is, SuperVLAD is a simple yet effective method (to enhance the generalizability across different domains).\n\n**W3. The illustration of typical failure cases:**\n\nThanks again for your suggestion. We provide detailed failure cases in the attached PDF (Figure 2), including cases where the retrieved image is geographically close to the query image but outside the 25-meter threshold, as well as some cases of challenging natural scenes without landmarks. We will add these failure cases to our final paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all the reviewers for their valuable time and constructive comments on our work. We reply to the concerns of each reviewer individually and will incorporate the suggestions in the revised paper. We also attach a PDF containing figures, which we refer to when answering specific questions.

Meta-review: This paper received unanimous acceptance from 5 reviewers (6,6,6,6,7) and, therefore a recommendation of acceptance.


Below are some concerns from AC, although they are **not** taken into account during the decision phase.

-------
The major contribution of this paper is a new formulation of the SuperVLAD layers, which are illustrated in Eq (4).
However, it seems that Eq (4) is just a simple case of Attention:
Given the well-known attention mechanism: $\rm{Attention}(q, k, v) = \rm{softmax}(q^T k) v$.
Equation (4) might be a special case of Attention by taking the query (q) as value (v) (q=v), that is: $\rm{softmax}(v^T k) v$, although there are some minor changes.
In this case, the major contribution, that is the formulation of the SuperVLAD layers, may not be valid anymore, thus the entire story (of removing cluster centers) in the introduction is not valid. The AC won't make a rejection recommendation based on this assumption, but the AC highly encourages the authors to (1) explain the differences between SuperVLAD and Attention in the final version or (2) withdraw the manuscript if the authors cannot provide strong justification for the difference or didn't notice this potential issue before.
Decision: Accept (poster)