SuperVLAD: Compact and Robust Image Descriptors for Visual Place Recognition

Authors: feng  lu
xinyao  zhang
canming  ye
shuting  dong
lijun  zhang
xiangyuan  lan
chun  yuan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-05 04:07:31

Abstract: Visual place recognition (VPR) is an essential task for multiple applications such as augmented reality and robot localization. Over the past decade, mainstream methods in the VPR area have been to use feature representation based on global aggregation, as exemplified by NetVLAD. These features are suitable for large-scale VPR and robust against viewpoint changes. However, the VLAD-based aggregation methods usually learn a large number of (e.g., 64) clusters and their corresponding cluster centers, which directly leads to a high dimension of the yielded global features. More importantly, when there is a domain gap between the data in training and inference, the cluster centers determined on the training set are usually improper for inference, resulting in a performance drop. To this end, we first attempt to improve NetVLAD by removing the cluster center and setting only a small number of (e.g., only 4) clusters. The proposed method not only simplifies NetVLAD but also enhances the generalizability across different domains. We name this method SuperVLAD. In addition, by introducing ghost clusters that will not be retained in the final output, we further propose a very low-dimensional 1-Cluster VLAD descriptor, which has the same dimension as the output of GeM pooling but performs notably better. Experimental results suggest that, when paired with a transformer-based backbone, our SuperVLAD shows better domain generalization performance than NetVLAD with significantly fewer parameters. The proposed method also surpasses state-of-the-art methods with lower feature dimensions on several benchmark datasets. The code is available at https://github.com/lu-feng/SuperVLAD.

Main paper: 
 introduction: 
 Visual Place Recognition (VPR) is a task that aims at quickly estimating the coarse geographical location of a place image (i.e., query) by retrieving the most similar images from a geo-tagged database [10, 41] . It has garnered significant attention in both computer vision and robotics communities, driven by its wide applications in augmented reality [42] and robot localization [59] , etc. However, VPR still faces some challenges: On the one hand, images captured at the same place may vary dramatically due to the changes in conditions (e.g., lighting and weather) and viewpoints. On the other hand, multiple different places can show high similarity, which may lead to perceptual aliasing [36] . It is quite challenging to address these issues simultaneously, especially for VPR methods that use compact global descriptors to represent the place images. Orange triangles and blue diamonds depict local descriptors from two different images. In (a) and (b), orange and blue arrows are the sum of residuals (for VLAD). With different training data distributions, the different cluster centers are yielded, causing opposite similarity results using cosine similarity (or normalized L2 distance). Compared to VLAD, our SuperVLAD, as shown in (c) and (d), simply calculates the distance between the weighted sum of local features directly, freeing from the impact of cluster centers. Thus, only minor changes will occur when dealing with two different distributions.The VPR is typically implemented with the image retrieval approaches [4, 48] . The place images are first described with global features and then the nearest neighbor search is performed over the database to get the matched place images of the query. The global descriptors are usually obtained by applying the aggregation methods, such as Bag of Words [3] and Vector of Locally Aggregated Descriptors (VLAD) [28, 35, 31, 53] , to process local (or patch) features. Although some hierarchical (i.e.two-stage) VPR methods improve robustness under challenging environments by cross-matching local features in the query and candidate images for re-ranking, they incur considerable additional computational latency and memory footprint. Taking a step back, the first stage of these hierarchical methods still requires the use of global features to retrieve candidates. As such, representing place images with compact and robust global descriptors is always the most essential and important issue in VPR.In the VPR methods based on neural networks, the NetVLAD [4] and GeM pooling [48] are the most commonly used aggregation/pooling methods. The former aggregates the feature maps extracted by neural networks with a trainable generalized VLAD layer. Its main difference from the vanilla VLAD [28] lies in its differentiable soft assignment and better performance. However, it still constructs the output vector by summing the residuals of descriptors assigned to each cluster (like VLAD), which is calculated by subtracting the learned cluster center from the local descriptors. When training data distributions differ, the learned cluster centers also vary. Even if we use a well-generalized backbone to extract the local features of two images, the differences in cluster centers can lead to completely different results in the similarity of the output global features (see Fig. 1 ). In other words, when there is a domain gap between the training and inference data, the cluster center learned on the training set is usually not suitable for inference, which can cause performance drops [5] . Besides, the global descriptor yielded by NetVLAD has a large dimension. Even with dimensionality reduction methods like PCA (or other learnable linear projections), the dimension often remains notably higher compared to the output of pooling methods. As a result, some VPR approaches requiring very low-dimensional features use GeM pooling to yield global descriptors at the cost of certain performance.To address the above issues, we propose a compact and robust global descriptor for VPR, named SuperVLAD. Our method uses the transformer-based backbone to produce initial patch features. Similar to NetVLAD, SuperVLAD calculates the weights of assigning patch features to clusters in the soft-assignment way. The key difference is that SuperVLAD does not compute cluster centers. Instead, we directly perform a weighted summation of the local features assigned to each cluster to achieve the purpose of aggregating local features. This means SuperVLAD only needs to learn the assignment of local features to clusters, without needing to learn the cluster centers, which directly improves its domain generalization. Meanwhile, we set a small number of clusters (an order of magnitude lower than vanilla NetVLAD), resulting in more compact global descriptors. Furthermore, by introducing supernumerary ghost clusters during the soft assignment and retaining one real (useful) cluster in the final output vector, we obtain a 1-cluster VLAD descriptor with extremely low dimensionality and promising performance. Our work brings the following contributions:(1) We propose a SuperVLAD aggregation method that does not require cluster centers to produce robust global descriptors. It can mitigate the performance degradation in NetVLAD caused by the training and testing data bias, while having fewer parameters (i.e., more lightweight). Besides, it can use only a small number of clusters to yield compact descriptors.(2) We also design a 1-cluster VLAD method by employing supernumerary ghost clusters during the soft assignment. It produces very low-dimensional features similar to the pooling methods. Compared to the same-dimensional GeM feature or class token, it shows notable performance advantages.(3) We conduct extensive experiments using various transformer-based backbones, demonstrating the effectiveness of our method. Furthermore, our SuperVLAD with the DINOv2 backbone outperforms state-of-the-art (SOTA) methods on several benchmark datasets with lower-dimensional descriptors.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: The abstract and introduction of our paper explicitly state our contributions, including the development of SuperVLAD and 1-Cluster VLAD and their advantages over NetVLAD (and GeM). These claims are consistent with the empirical results presented, ensuring a clear reflection of the scope and contributions of our work.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related work: 
 In the early development of VPR, the methods predominantly relied on hand-crafted local features such as SURF [6, 13] . These local descriptors were then aggregated into global features using algorithms such as Bag of Words [3] , Fisher Vector [47] , and VLAD [28, 53, 5, 32] . The resulting global feature vectors were subsequently used to perform a nearest neighbor search over the database to retrieve the most similar images. As deep learning techniques have advanced, the representation capability of deep features has been widely recognized in the VPR community [51, 4, 29, 11, 12, 44, 18, 20, 58, 60, 52, 33, 1, 2, 21, 7, 10] . Arandjelovic et al. [4] designed a pioneering architecture that combines deep neural networks with the proposed differentiable VLAD aggregation approach called NetVLAD. Likewise, other traditional aggregation algorithms were also transformed into differentiable modules as the aggregation layer of neural networks for end-to-end training [26, 45, 61] . Despite remarkable performance over global max (or average) pooling, NetVLAD-related methods [4, 29, 34, 21] tended to have high-dimensional feature representations, which can be a drawback in terms of real-time performance [9] . To address this, the Generalized Mean (GeM) pooling [48] was considered a simple alternative that obtains low-dimensional global features. This method simply extended global average pooling by using the p-norm of local features instead of the average, where p is also a trainable parameter. Additionally, Berton et al. [9] established an open-source benchmark to fairly compare these standard global-retrieval-based VPR approaches under a unified framework.Although global-retrieval-based methods have achieved reasonable performance, most of them exhibit limited robustness in challenging environments and are susceptible to perceptual aliasing. Two common methods to improve the robustness of VPR systems are leveraging temporal consistency constraints and spatial consistency constraints. The first method matches image sequences (i.e., maintains temporal continuity) [43, 16, 22, 37, 19] to achieve robust VPR with extreme condition changes. The second method usually involves a two-stage VPR process [25, 24, 8, 55, 40, 67, 38, 41] , where it first retrieves the top-k candidate images in the database with global features and then re-ranks these candidates by matching local features with spatial consistency verification. However, these methods always introduce extra consumption in computational latency or/and memory footprint.In the last two years, with the emergence of purpose-built large-scale VPR training datasets [7, 1] and pre-trained foundation models [62, 46] , some robust global-retrieval-based VPR methods have been proposed. CosPlace [7] and EigenPlaces [10] cast the training of VPR as a classification problem and trained the VPR model on the San Francisco eXtra Large (SF-XL) datasets. MixVPR [2] incorporated the deep features with the multi-layer perceptrons and trained the model with the Multi-Similarity loss [56] on the large-scale and appropriate supervised GSV-Cities [1] dataset. These works all have achieved outstanding performance using only the CNN models. Other works [30, 41, 39, 27] were based on visual foundation models and achieved better results. DINOv2 [46] has been the most widely used foundation model in VPR. It is a ViT-based [17] model trained on the large-scale curated LVD-142M dataset using a self-supervised strategy and can offer powerful visual features for downstream tasks. SelaVPR [41] proposed a hybrid global-local adaptation method to adapt the DINOv2 model for two-stage VPR. CricaVPR [39] also adapted DINOv2 as the backbone and proposed a cross-image correlation-aware representation learning method to enhance the robustness of image features. A closely related work to our SuperVLAD is the SALAD work [27] , which used DINOv2 as the backbone and presented a novel aggregation algorithm to improve NetVLAD. SALAD redefined the soft assignment of local features in NetVLAD as an optimal transport problem and employed the Sinkhorn algorithm [14] to solve it. Different from this work, our SuperVLAD maintains the soft-assignment way of NetVLAD but effectively solves the issue caused by the learned cluster centers in NetVLAD being unsuitable for various inference data distributions. Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: Our paper includes a dedicated "Limitations & Future Work" subsection where we thoroughly discuss the constraints of our approach.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate Limitations section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
methodology: 
 Fig. 2 shows the overview of SuperVLAD. We first use a backbone to extract local features (Sec.3.1), then aggregate them with SuperVLAD (Sec. 3.2). 1-Cluster VLAD is an alternative to get more compact descriptors (Sec. 3.3) and cross-image encoder is optional to boost performance (Sec. 3.4).The Vision Transformer (ViT) [17] and its variants possess superiority in capturing long-range feature dependencies and have shown remarkable performance on various computer vision tasks [66, 63, 64] , including VPR [55, 41] . In this work, we use ViT (or its variants) as the backbone for feature extraction. ViT initially divides the image into N patches and linearly projects them into D-dim patch embeddings x p ∈ R N ×D , after which it prepends a learnable class token to x p as x 0 ∈ R (N +1)×D . Following the addition of positional embeddings, x 0 is input into a sequence of transformer encoder layers to generate the feature representation. The final output of ViT includes one class token and N patch tokens. We directly discard the former and use the latter as local features to be input to the subsequent SuperVLAD aggregation layer, thereby obtaining the final global descriptor of the place image. Its robustness against domain shift can be improved to some extent. Super-VLAD completely eliminates the need for cluster centers and avoids their negative impact.SuperVLAD produces the global descriptor of an image by first associating all local features of this image with K clusters and subsequently aggregating these features into each cluster. The basic process of the SuperVLAD layer is similar to that of NetVLAD [4] , so we start by introducing our work from NetVLAD (and VLAD [28] ).For NetVLAD, given N local descriptors {x i } (x i ∈ R D ) of an image as input, and K cluster centers {c k } (c k ∈ R D ) as parameters, it computes a matrix V ∈ R K×D as the image representation. The k-th row in the matrix V accumulates the (weighted) residuals (x i -c k ) of local descriptors assigned to cluster c k . More formally, the (k, j)-th element in the matrix V is computed as follows:EQUATIONwhere x i,j and c k,j are the j-th element of the i-th local descriptor and k-th cluster center, respectively. a k (x i ) is the weight of the local descriptor x i assigned to the cluster c k . In VLAD, the assignment is hard, i.e., a k (x i ) equals 0 or 1. In contrast, NetVLAD replaces it with the soft assignment and computes a k (x i ) as:EQUATIONExpanding the squares in Eq. 2, we can readily observe the cancellation of the term e -α∥xi∥ 2 between the numerator and the denominator, yielding the following form:EQUATIONwhere vector w k = 2αc k and scalar b k = -α ∥c k ∥ 2 . In the implementation of NetVLAD, {w k }, {b k }, and {c k } are set as three independent sets of trainable parameters (while VLAD has only {c k }). That is, {w k } and {b k } are actually decoupled from {c k }. As shown in Fig. 3 , NetVLAD, with its greater flexibility than VLAD, has some potential to alleviate the performance drop caused by domain shift [4] . However, NetVLAD retains cluster centers, which would be hardly suitable for various data distributions in inference as it is learned on one training set [5] . Since the assignment of local features has been decoupled from the cluster centers in NetVLAD (in VLAD, it is based on the distance to the cluster center), we can directly remove the cluster center and aggregate the first-order statistics of local features (rather than residuals) assigned into each cluster. So, the matrix V in SuperVLAD can be formulated asEQUATIONFor each cluster, SuperVLAD has the parameters w k and b k1 , compared to NetVLAD with the parameters w k , b k , and c k , and VLAD with the parameter c k . This makes SuperVLAD not affected by the cluster center c k and provides greater flexibility compared to NetVLAD. Besides, the number of parameters in SuperVLAD is about half less than that in NetVLAD. It is worth noting that Eq. 4 is different from applying the attention mechanism to local features. Our approach involves assigning features to clusters (rather than applying attention to features). More specifically, given a feature, the sum of the weights of assigning it to all clusters is the constant 1. However, for a given cluster, the sum of the weights of assigning all features to it is not fixed (for attention, it is the constant 1). The recent work SALAD [27] also sums the local features with the soft-assignment weights to get the global descriptor. However, its soft assignment relies on clusters (the so-called optimal transport between clusters and local features), i.e., not free from clusters, and requires iterative computations to solve. Its parameters are no less than NetVLAD. These are different from our lightweight and low-computation SuperVLAD. Besides, In SuperVLAD we set only a small number of clusters, namely 4, which makes the output compact without additional dimensionality reduction techniques. As shown in Fig. 2 , the main module of SuperVLAD is a 1×1 convolution (conv) and softmax for soft-assignment, and a VLAD core for aggregation. Unlike NetVLAD, the VLAD core of SuperVLAD has no parameters, and all trainable parameters exist only in the conv layer. The matrix V is finally intra-normalized, flattened into a vector, and entirely L2-normalized as the output global descriptor. Moreover, we also absorb GhostVLAD [65] , which extends NetVLAD by introducing "ghost" clusters. Specifically, it adds additional G ghost clusters that are used for the soft assignment in the same way as the original K clusters, but the ghost clusters are disregarded in the aggregation process and do not directly contribute to the final output. Ghost clusters can be used to correspond to useless objects in VPR, such as sky, ground, dynamic objects, and so forth. In SuperVLAD, the number of ghost clusters is set to only one (i.e., G = 1).Due to the high dimensionality of the descriptors output by NetVLAD, some SOTA VPR methods use learnable linear projections or GeM pooling for low-dimensional representations, e.g., CosPlace [7] , EigenPlaces [10] , and MixVPR [2] . The GeM pooling is particularly popular because it easily produces global representations with the same dimensionality as the pooled local/patch features, although it may be lacking in performance. Here, we design a 1-cluster VLAD algorithm that can produce very low dimensional descriptors, which has the same output dimensionality as GeM but performs better. Specifically, we only need to set the number of useful clusters in SuperVLAD to 1, and the number of ghost clusters to greater than 1, (we set it to 2), i.e., K=1 and G=2, to get the 1-cluster VLAD. This means that all descriptors representing objects relevant to VPR are assigned to the same cluster, and their weighted sum (the weight is a 1 (x i )) is used as the final global descriptor. This subsection introduces an optional process for SuperVLAD (not for 1-Cluster VLAD). To further enhance the performance, we draw inspiration from the cross-image encoder proposed in CricaVPR [39] , which can use the cross-image variations as a cue to guide the representation learning and produce more robust image features through cross-image interaction in a batch. However, directly using the transformer encoder to process the entire SuperVLAD vector would incur significant memory and running time overhead, proportional to the square of the vector size. To overcome this issue, we first split the SuperVLAD descriptors by clusters and then input them into the cross-image encoder ( i.e., two stacked transformer encoders) for cluster-wise cross-image interaction. That is, this encoder will model the correlation between the aggregated features belonging to the same cluster of all images in a batch. We then concatenate the aggregated features belonging to different clusters of an image and L2 normalize again to get the final descriptor of this image.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes] Justification: Our paper only involves a small amount of theory, which has been described in the Methodology section 3 and relevant references are also provided.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
datasets and performance evaluation: 
 We conduct experiments on several VPR benchmark datasets, which cover various challenges in VPR such as viewpoint changes, condition changes, and perceptual aliasing. Table 1 provides a concise summary of them. Pitts30k [54] mainly exhibits severe viewpoint changes in the urban environment. MSLS [57] is a comprehensive dataset comprising images collected in urban, suburban, and natural scenes over 7 years, and encompasses a wide range of visual changes (viewpoint and condition changes). Nordland [10] exhibits seasonal changes in natural and suburban scenes. SPED [11] consists of low-quality and high-scene-depth images captured from diverse scenes with various condition changes. More details are in Appendix G.In our experiments, we employ the Recall@N (i.e., R@N) to assess the recognition performance of VPR methods. This metric quantifies the percentage of queries for which at least one of the N retrieved reference images falls within a predefined threshold of the ground truth. We set the threshold to 25 meters for Pitts30k and MSLS, ±10 frames for Nordland, unique counterpart for SPED, following common evaluation procedures [54, 57, 10, 11] .Here we describe the implementation details of training the DINOv2-based SuperVLAD model on the GSV-Cities [1] dataset for SOTA comparison. More details of training the other transformer-based models for the ablation study are in Appendix E. We implement our experiments on two NVIDIA GeForce RTX 3090 GPUs using PyTorch. The size of the input image is 224×224 in training (322×322 in inference) and the token dimension of the DINOv2-base backbone is 768. The descriptor dimension of SuperVLAD is 3072 and that of 1-cluster VLAD is 768. We only fine-tune the last four transformer encoder layers (freeze the previous layers) of the DINOv2 backbone. For the loss function, we utilize the multi-similarity loss [56] and set its hyperparameters as in [1] . The model training is performed using the Adam optimizer with an initial learning rate of 0.00005, halved every 3 epochs. Considering that the cross-image encoder is not initialized, we use a larger learning rate (0.0001) to train it separately. Each training batch consists of 120 places, with 4 images per place, resulting in a total of 480 images. An inference batch consists of 8 images (except for the SPED dataset where the batch size is 4). The training process is terminated when the performance on MSLS-val does not improve for three epochs. The actual number of effective training epochs is 7, and the training time is 81.6 minutes.In this section, we compare the proposed SuperVLAD with several excellent VPR methods, including seven one-stage methods using global feature retrieval (like ours): NetVLAD [4] , SFRS [21] , Method Backbone Dim Pitts30k MSLS-val Nordland SPED R@1 R@5 R@10 R@1 R@5 R@10 R@1 R@5 R@10 R@1 R@5 R@10 NetVLAD [4] VGG16 32768 81.9 91.2 93. CosPlace [7] , MixVPR [2] , Eigenplaces [10] , CricaVPR [39] , and SALAD [27] , as well as two twostage methods with re-ranking: TransVPR [55] and SelaVPR [41] . Note that MixVPR, CricaVPR, SALAD, and our SuperVLAD use the same training dataset, i.e., GSV-Cities. Meanwhile, CosPlace and EigenPlaces are trained on the purpose-built extra large-scale (SF-XL) datasets. Additionally, the latest works SelaVPR, CricaVPR, and SALAD all use the foundation model DINOv2 as the backbone (SelaVPR using DINOv2-large and others using DINOv2-base) and achieve the SOTA performance on the VPR task. So here we follow them and use the DINOv2-base backbone. The details of the above methods can be seen in Appendix H. Table 2 shows the quantitative results and our SuperVLAD achieves the best results on all these datasets.The methods based on the DINOv2 backbone, including SelaVPR, CricaVPR, SALAD, and our SuperVLAD, all achieve excellent performance and outperform the remainder methods on these datasets with diverse challenges. These methods all fine-tune DINOv2 in different ways, which shows that based on the powerful feature representation capability of DINOv2, coupled with appropriate fine-tuning, it is sufficient to cope with most challenges in the VPR task. However, our method uses a more compact feature representation (than CricaVPR and SALAD) and outperforms other methods on the four datasets. In particular, it achieves 93.2% R@1 on the SPED dataset, demonstrating significant advantages compared to other global-retrieval-based methods and two-stage methods. This indicates the high robustness of our method to handle condition variations in datasets containing low-quality and high-scene-depth images. Additionally, both CricaVPR and SuperVLAD use the cross-image encoder and thus achieve significantly better performance than other methods on Pitts30k and Nordland, which are able to provide different images of the same place in a batch and mutually improve condition invariance and viewpoint invariance [39] . However, such benefit cannot be obtained on MSLS-val and SPED (all query images from different places), our approach still outperforms all other methods (and is obviously better than CricaVPR). Inference time (ms / single image)Recall@1 SALAD MixVPR SFRS SuperVLAD Figure 5 : The comparison of some global-retrievalbased methods in Recall@1 (on Pitts30k), inference time (ms/single image), and descriptor dimensionality. The diameter of each dot is proportional to the descriptor dimension. Our SuperVLAD gets the best R@1 with the most compact descriptor.Fig. 4 qualitatively shows the superior performance of our SuperVLAD in some difficult scenes. These examples exhibit various challenges, such as drastic condition changes, viewpoint changes, and occlusions (caused by dynamic objects). Other methods mostly return similar images from different places, i.e., suffer from perceptual aliasing and fail to get the right results. However, our SuperVLAD retrieves the correct reference images, showing high robustness against these challenges.Fig. 5 simultaneously shows the R@1 on Pitts30k, the inference time of a single image, and the descriptor dimensionality. Among the four methods (SFRS, MixVPR, SALAD, and Su-perVLAD), MixVPR uses the CNN backbone and the feature mixing method to get global descriptors, which achieves the shortest inference time. The remaining three methods use VLADrelated methods to get global features, and our SuperVLAD achieves the fastest inference speed among them. Although SFRS is based on the CNN model, it uses NetVLAD with 64 clusters to obtain high-dimensional image features and then uses PCA for dimensionality reduction (time-consuming). SALAD and our SuperVLAD are based on the foundation model DINOv2. However, SALAD uses the Sinkhorn algorithm to iteratively compute optimal transport assignment between clusters and local features, which takes more inference time than our method (17.8ms vs. 13.5ms). Besides, the number of parameters in our SuperVLAD aggregator is much lower than that of SALAD (less than 3/1000), as shown in Table 3 . Our SuperVLAD uses a lightweight and low-compute aggregation layer to get the compact global descriptor (without extra dimensionality reduction), which simultaneously has fast inference speed, low-dimensional descriptors, and excellent recognition performance. In this section, we conduct a series of ablation experiments to demonstrate the effectiveness of the proposed SuperVLAD and 1-Cluster VLAD. The cross-image encoder is not used by default in ablation experiments. The SuperVLAD using the cross-image encoder is denoted as SuperVLAD † .Effect of SuperVLAD. We first validate the effectiveness of SuperVLAD by comparing it to NetVLAD. To be fair, we do not use the cross-image encoder in SuperVLAD, and apply multiple different transformer-based models as the backbone, including the ViT [17] and CCT [23] models pre-trained on ImageNet [15] , and the foundation model DINOv2 [46] (that is also a ViT model) pre-trained on the large-scale curated dataset. We conduct experiments with three training sets (MSLS, Pitts30k, and GSV-Cities) and two test sets (MSLS and Pitts30k). To validate that SuperVLAD has superior generalization ability over NetVLAD in addressing the domain shift issue mentioned above, we focus on the cross-domain inference performance of models. For example, the performance of a model trained on Pitts30k when tested on MSLS, or vice versa. The results are shown in Table 4 . Note that Pitts30k contains only urban scene images, while MSLS covers urban, suburban, and natural scenes, and GSV-Cities is a large-scale training set with accurate supervision and diverse visual variations. Thus, the performance of a model trained on Pitts30k and tested on MSLS best reflects domain generalization, while the reverse is less indicative. The model trained on GSV-Cities encounters almost no domain drift issues on these test sets (Pitts30k and MSLS-val). The experimental results largely align with these facts and the above theory. Training the SuperVLAD model (based on CCT and DINOv2) on Pitts30k and testing it on Pitts30k yields similar results to NetVLAD. However, when tested on MSLS, it shows an absolute R@1 improvement of 6.8% (with CCT) and 2.4% (with DINOv2), respectively. The SuperVLAD model trained on MSLS using ViT MSLS-val R@1 R@5 R@10 R@1 R@5 R@10 CCT-SV-ng 84.1 92.9 95. 1 Method Pitts30k MSLS-val R@1 R@5 R@10 R@1 R@5 R@10 GeM Pooling 89. exhibits similar test results to NetVLAD on MSLS-val but still has a slight advantage when tested on Pitts30k. These results demonstrate that our SuperVLAD has better domain generalization than NetVLAD. Since the GSV-Cities dataset covers the various visual variations in VPR and provides accurate supervision for fine-tuning, both models trained on it perform very well on Pitts30k and MSLS-val. However, this does not mean that our SuperVLAD has no advantage over NetVLAD with such a large-scale VPR training set. SuperVLAD achieves the same performance as NetVLAD with fewer parameters and computations, e.g., offering equivalent performance at a lower cost.Effect of ghost clusters and the performance of 1-Cluster VLAD. To investigate the impact of ghost clusters, we conduct two sets of experiments based on CCT and DINOv2 to compare the performance with and without ghost clusters. All models also do not use the cross-image encoder. The CCT-based model is trained on Pitts30k and the DINOv2-based model is trained on GSV-Cities. The results are shown in Table 5 . When based on DINOv2, whether using ghost clusters or not achieves excellent performance, with little to no difference in their performance (without using ghost clusters even slightly better). However, based on CCT and trained on Pitts30k, the use of ghost clusters resulted in a certain improvement. This indicates that the ghost cluster is still necessary. More importantly, we leverage it to achieve 1-Cluster VLAD. We compared the proposed 1-Cluster VLAD with two commonly used methods (GeM pooling and class token) that can output very compact features of the same dimension as local features (as does our 1-Cluster VLAD). The results, as shown in Table 6 , demonstrate the obvious superiority of 1-Cluster VLAD compared to the other two methods. So, our 1-Cluster VLAD can be used as a new choice for very low-dimensional descriptors.Effect of the number of clusters. To investigate the impact of the number of clusters on global descriptors, we conducted two sets of experiments using SuperVLAD with and without the cross-image encoder. All models utilize DINOv2 as the backbone and are trained on GSV-Cities. Results (see Table 7 ) indicate that even with a small number of clusters, there is no significant performance decrease. This is primarily attributed to the powerful feature representation capability of the transformer-based foundation model DINOv2, while the large-scale VPR training dataset GSV-Cities offers appropriate supervision for fine-tuning. Even with a small number of clusters, it allows for reasonable and effective classification/clustering of objects that are relevant to place recognition (e.g., buildings, vegetation). Unlike most VLAD-related methods that use dozens of clusters, our experiments show that just 4 clusters can achieve robust VPR in most cases, which provides compact global descriptors without the need for additional dimensionality reduction techniques.Table 7 : Comparison of the SuperVLAD ablated versions with different numbers of clusters, as well as with and without the cross-image encoder (the former denoted as SuperVLAD † ).No. Pitts30k MSLS-val Clusters R@1 R@5 R@10 R@1 R@5 R@10 Effect of the cross-image encoder. Table 7 also presents the comparison of Su-perVLAD with and without cross-image encoder (i.e., our cluster-wise cross-image interaction). It is evident that using a crossimage encoder consistently leads to performance improvements. However, the performance enhancement on Pitts30k is significantly greater than that on MSLS. This is because Pitts30k can provide a batch of images with different viewpoints from the same place during inference, which allows different images to directly improve the robustness of each other through the cross-image interaction, while MSLS cannot. Considering that conditions like the Pitts30k dataset might not be available in practical application, we consider the cross-image encoder as an optional component for performance enhancement. It is worth noting that, even without it, our method still achieves performance comparable to the SOTA methods.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: Our paper provides detailed formulations and descriptions of our proposed feature aggregation algorithms called SuperVLAD and 1-Cluster VLAD in the Methodology section. We also provide implementation details in the Experiments section 4.2 and Appendix E. In addition, open access to our code and model checkpoints will be provided. 
conclusions: 
 In this paper, we introduced SuperVLAD, a compact and robust global descriptor for VPR. By eliminating the need for cluster centers and utilizing a small number of clusters, SuperVLAD achieved improved domain generalization and produced more compact descriptors. We also designed a 1-cluster VLAD descriptor with extremely low dimensionality by introducing supernumerary ghost clusters during the soft assignment. Experimental results on various transformer-based backbones validated the effectiveness of SuperVLAD, producing more robust features than NetVLAD with a lighter aggregation layer. Furthermore, the DINOv2-based SuperVLAD also outperformed SOTA methods on several VPR benchmark datasets with more compact global descriptors.Limitations & Future Work. While our study presents some improvements in VPR, particularly in enhancing domain generalization ability, reducing the dimensionality of global descriptors, and reducing the number of parameters for the aggregation layer, we acknowledge two limitations of our work: Firstly, the current success of SuperVLAD relies on the use of the transformer backbone. Although our experimental results provided in Appendix C demonstrate that SuperVLAD still has certain advantages over NetVLAD when using a CNN backbone with a small number of clusters, it is not as good as NetVLAD in more commonly used settings. This can be seen as a limitation of SuperVLAD. Secondly, our method uses the cross-image encoder to further enhance performance, similar to CricaVPR [39] . It is necessary to set the inference batch size to an appropriate value.Setting it to 1 directly renders the cross-image encoder ineffective during inference, resulting in a performance drop due to the gap between training and inference. Since we can only use a single-frame query in some practical applications, we compromise to treat the cross-image encoder as an optional module that is not used (during both training and inference) when multiple-frame inference is not feasible. Even without the cross-image encoder, our SuperVLAD can achieve good performance. Some additional discussion on the limitations of SuperVLAD is in Appendix B. In future work, we will try to address the above limitations, and further assess the long-term stability and potential performance changes of the model, which are important for practical deployments. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes] Justification: Our model reached SOTA performance in the task of VPR and there is indeed a small possibility of unintended and malicious use. Nevertheless, our project is relatively not at high risk since we don't release new datasets.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: In the paper we have properly cited the original paper of the datasets. We respect the license of the referenced code and data and will properly acknowledge them in the project README.md. 
 
 Appendix: Our research on VPR, particularly the development of the SuperVLAD and 1-Cluster VLAD methods, has been motivated by the potential to significantly enhance applications in augmented reality, robot localization, and autonomous driving. The potential benefits and disadvantages depend on how people choose to use our method and how they handle failure cases.Positive Impacts on Technology Advancement. The introduction of our novel method offers a more compact and generalizable feature representation, which could lead to more efficient and robust retrieval capability for downstream applications.Potential Cases. Although our method has shown high performance on evaluation metrics, there remains a small probability of failure cases, which could potentially lead people or autopilots astray. Nevertheless, suppose our method is used in conjunction with other methods and information for autonomous driving systems, then our method mostly introduces additional safety and efficiency to the overall system.Potential Malicious Uses. Although our research on better feature representation is foundational and not tied to particular applications, the VPR system may be exploited for invasive surveillance or social media monitoring, leading to privacy concerns. It is important to avoid unethical use of VPR studies for negative purposes.We have detailed the benefits of removing cluster centers in SuperVLAD in the main paper. This section provides more comparisons and analyses between NetVLAD and SuperVLAD, mainly analyzing the possible negative effects of removing cluster centers (i.e., the advantages of using cluster centers in NetVLAD) to illustrate the limitations (or characteristics) of SuperVLAD.First, the cluster centers have the potential to play a guiding role in improving the accuracy of assigning local features to different clusters. Especially when local features are assigned to fine-grained clusters (i.e., a large number of clusters), this strength of NetVLAD over SuperVLAD will gradually become apparent. Although the DINOv2-based NetVLAD model with a small number of clusters can achieve performance comparable to a large number of clusters when training on the large-scale dataset GSV-Cities, it still makes sense to set a large number of clusters for a simpler backbone and a smaller training dataset. Here we conduct complementary experiments using the CCT backbone and Pitts30k training dataset. The results are shown in Table 8 . In this case, SuperVLAD has a clear advantage over NetVLAD with a small number of clusters, but this advantage gradually disappears with a large number of clusters. That is, NetVLAD shows a more pronounced improvement with a larger number of clusters. However, when using a large number of clusters, the memory occupied by the aggregation layer usually becomes significant in the training stage. The SuperVLAD can significantly reduce GPU memory usage because it has only half the trainable parameters of NetVLAD. Moreover, from another perspective, SuperVLAD experiences less performance degradation with fewer clusters, indicating its good ability to produce low-dimensional global features by directly reducing the number of clusters. Therefore, we do not consider this as a limitation of SuperVLAD but rather as one of its characteristics (i.e., more suitable for a small number of clusters than NetVLAD).Secondly, using the local features to subtract the cluster center to get the residual can more accurately capture the subtle differences of local features (improve the ability to distinguish details). However, when we use the transformer-based backbone (especially the foundation model) to provide powerful local features (i.e., patch tokens), this can basically be ignored. So we use the transformer backbone for our SuperVLAD, which can be considered a limitation. We will discuss the use of CNN backbone in the next section.Although we have mentioned in the main paper that our SuperVLAD requires a transformer-based backbone to extract local features (i.e., patch tokens) of images, here we also present experimental results when applied to a CNN backbone (specifically VGG16) in order to observe the limitations of our method. Considering that previous works using NetVLAD to aggregate CNN feature maps There is an obvious domain gap between these two datasets, which is the issue our method attempts to address. Besides, (d) shows an example from the Baidu Mall indoor dataset. There is another gap between indoor data and outdoor data, which is mainly caused by the fact that the objects that need to be paid attention to in the two types of data are usually different. Baidu Mall R@1 R@5 R@10 R@1 R@5 R@10 R@1 R@5 R@10 SelaVPR [41] In the main paper, we have described the implementation details of training DINOv2-SuperVLAD models on GSV-Cities for comparison with the SOTA methods. Here we introduce the details of the implementation of the ablation experiment. For the training of the models based on ViT and CCT backbones (and trained on MSLS/Pitts30k), we follow some previous work [9, 38] . The last two transformer encoder layers of the ViT backbone (ViT-Base) are truncated and the size of the input image is 224×224. The last four transformer encoder layers of the CCT backbone (CCT-14) are truncated and the layers before the third layer are frozen, using the input images with 384×384 pixels. For the training of VGG16-based models on GSV-Cities, the implementation is basically the same as the DINOv2-based model. However, the layers before the last block of the VGG16 backbone are frozen, as in [9] . ViT, CCT, and VGG16 are all pre-trained in ImageNet [15] . Following the visual geo-localization benchmark [9] , we use the triplet loss to train models on Pitts30k and MSLS.Ten hard negative images are used in each triplet. The model training is performed using the Adam optimizer with a learning rate of 0.00001. The batch size is set to 4. The training process on MSLS is terminated when the result does not improve for 6 epochs (40000 images/epoch), and that on Pitts30k are 3 epochs (5000 images/epoch). Examples from these datasets highlight challenging scenarios including severe condition changes, viewpoint variations, dynamic interference, few or no landmarks, etc. Ours SuperVLAD achieves correct results, while others fail. Moreover, we also provide failure cases of our approach as shown in Fig. 11 , which suggests that future improvements of our method may need to consider how to retrieve images that are closer to the query image and improve robustness in natural scenes. , our method (and some other methods) obtains database images that are geographically close to the query image but the radius exceeds the set threshold (25m). This phenomenon usually occurs when the camera is too close or too far away from the scene. For the third to fifth examples, all methods fail because the images are captured in natural scenes without landmarks, which seems to be a noteworthy challenge for current VPR methods. For the last challenging example (bad weather and large viewpoint changes), all global retrieval methods (including ours) fail, while SelaVPR based on local feature re-ranking returns the correct result. This shows that even though our method achieves good results using only global features, it is possible to further improve the results by local feature re-ranking.MSLS [57] . The MSLS (Mapillary Street-Level Sequences) dataset is a large-scale VPR dataset with over 1.6 million images collected in urban and suburban scenes from 30 cities on six continents, provided with GPS coordinates and compass angles. It covers many causes of environmental changes, such as illumination, weather, season, and viewpoint changes, as well as dynamic objects. The dataset contains three sets: training, public validation (MSLS-val), and withheld test (MSLS-challenge). We evaluate models on the MSLS-val set. It should be noted that the MSLS val set used in the Visual Geo-localization Benchmark [9] and CosPlace [7] contains a different number of query images as the official version2 [57] . To facilitate comparison with previous methods, we use the official version and run CosPlace on it, obtaining better results than those reported in [7] .Pitts30k [54] . The Pitts30k dataset is generated from Google Street View panoramas with GPS labels available. It provides 24 viewpoint images at each location, with large viewpoint variations as well as some dynamic objects. Pitts30k contains 10k database images in each of the training, validation, and test sets. Its test set is adopted to evaluate models in experiments.Nordland [50] . The Nordland dataset is recorded from the same viewpoint in the front of a train in four seasons. So, there are severe condition (e.g., season and weather) changes but no viewpoint changes in this dataset. It mainly contains suburban and natural place images. The ground truth of this dataset is offered by the frame-level correspondence. Following previous works [10] , we use the summer images as the database and the winter images as queries, each containing 27592 images.SPED [11] . The SPED dataset is made up of low-quality and high-scene-depth images taken from CCTV cameras around the globe. The images in this dataset show various condition variations, such as lighting, weather, and seasonal changes. This dataset covers a diverse range of outdoor scenes, including forest landscapes, country roads, and urban environments. We use the SPED-test set for experiments, with 607 images in both its database and query set.NetVLAD [4] 3 . NetVLAD is a classic VPR method with a learnable VLAD layer that is pluggable into any CNN architecture. We use the PyTorch implementation with the VGG16 backbone trained on the Pitts30k dataset.SFRS [21] 4 . This work mines hard positive samples using self-supervised image-to-region similarities for training a more robust NetVLAD model. We use the official implementation with the model trained on Pitts30k in the comparison experiment.TransVPR [55] 5 . This work combines attentions from multiple levels of vision Transformer to yield global features for the candidate retrieval, then uses an attention mask to filter feature maps to get key-patch descriptors for cross-matching in re-ranking. We use the official implementation with the model trained on Pitts30k for the testing on the Pitts30k test set and the model trained on MSLS for others.CosPlace [7] 6 . This work casts the training of VPR as a classification problem and trains the VPR model on the San Francisco eXtra Large (SF-XL) datasets. We use the official VGG16 model (with the 512-dim output feature) for testing.MixVPR [2] 7 . This work introduces an innovative holistic feature aggregation approach, which utilizes feature maps extracted from the pre-trained backbone as global features. A stack of Feature-Mixer is employed to integrate global relationships into each feature map to yield global descriptors of place images. MixVPR is trained on the GSV-Cities [1] dataset and we conduct comparisons using the best configuration (ResNet50 with 4096-dim output features).EigenPlaces CricaVPR [39] 10 . This work presents a cross-image correlation-aware representation learning method for VPR, which employs the attention mechanism to establish the correlation among multiple images within a batch. This method can leverage the cross-image variations as a cue to guide representation learning, and when producing the feature of each image, it can obtain useful information from other image features to improve its robustness against viewpoint variations, condition variations, and perceptual aliasing. To further enhance robustness, a multi-scale convolution-enhanced adaptation method is proposed in CricaVPR, adapting the pre-trained foundation model DINOv2 to the VPR task. We used the official implementation and configuration for comparison.SALAD [27] 11 . This method utilizes the pre-trained foundation model DINOv2 as the backbone, and treats the soft assignment in NetVLAD as an optimal transport problem, solving it using the Sinkhorn algorithm. This soft assignment takes into account both the relationships between features and clusters, as well as between clusters and features. Additionally, it introduces a "dustbin" cluster, which is aimed at selectively discarding features identified as non-informative, thereby improving the overall quality of descriptors. We used the official pre-trained DINOv2-SALAD model for the evaluation.NeurIPS Paper Checklist• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.In implementation, b k can also be removed (i.e., b k =0), as in the different implementation versions of NetVLAD (refer to https://github.com/Relja/netvlad/blob/master/README_more.md#netvlad-versions).https://github.com/mapillary/mapillary_slshttps://github.com/Nanne/pytorch-NetVladhttps://github.com/yxgeee/OpenIBLhttps://github.com/RuotongWANG/TransVPR-model-implementationhttps://github.com/gmberton/CosPlacehttps://github.com/amaralibey/MixVPRhttps://github.com/gmberton/EigenPlaceshttps://github.com/Lu-Feng/SelaVPRhttps://github.com/Lu-Feng/CricaVPRhttps://github.com/serizba/salad 


Decision: This paper received unanimous acceptance from 5 reviewers (6,6,6,6,7) and, therefore a recommendation of acceptance.


Below are some concerns from AC, although they are **not** taken into account during the decision phase.

-------
The major contribution of this paper is a new formulation of the SuperVLAD layers, which are illustrated in Eq (4).
However, it seems that Eq (4) is just a simple case of Attention:
Given the well-known attention mechanism: $\rm{Attention}(q, k, v) = \rm{softmax}(q^T k) v$.
Equation (4) might be a special case of Attention by taking the query (q) as value (v) (q=v), that is: $\rm{softmax}(v^T k) v$, although there are some minor changes.
In this case, the major contribution, that is the formulation of the SuperVLAD layers, may not be valid anymore, thus the entire story (of removing cluster centers) in the introduction is not valid. The AC won't make a rejection recommendation based on this assumption, but the AC highly encourages the authors to (1) explain the differences between SuperVLAD and Attention in the final version or (2) withdraw the manuscript if the authors cannot provide strong justification for the difference or didn't notice this potential issue before.

Reviews: review: paper_summary: paper_summary: This work introduces SuperVLAD, a novel image descriptor for Visual Place Recognition. It eliminates the need for cluster centers, addressing NetVLAD's performance degradation due to data bias while being more lightweight. SuperVLAD generates compact descriptors with fewer clusters. Additionally, a 1-cluster VLAD method employs supernumerary ghost clusters for soft assignment, producing low-dimensional features that outperform same-dimensional GeM features or class tokens. Extensive experiments with transformer-based backbones demonstrate the effectiveness of this approach.
strength_weakness: strength_weakness: Provide more technical and experimental details. Please see my questions below The paper could benefit from more technical and experimental details. Please see my questions below.
questions: questions: 1.	How do these new image descriptors integrate with visual localization and SLAM systems? Specifically, it would be beneficial to understand how SuperVLAD can improve the accuracy and robustness of these systems in real-world applications.
2.	Please provide some efficiency results on SuperVLAD, such as inference time and training time. Additionally, information on the training data used would be helpful. Understanding the computational demands and data requirements is crucial for assessing the practical applicability of the method.
3.	To attract readers in a machine learning conference, it would be valuable to offer general insights into how deep learning can contribute to spatial scene perception. Discussing the importance of this task to the machine learning community and the broader implications of improving visual place recognition through advanced descriptors like SuperVLAD would enhance the paper's appeal.
limitations: limitations: please see my questions. Since this work is within a subarea in CV, please provide more insights into how ML can impact this area and how this application is important to ML.
paper_summary: paper_summary: The paper introduces SuperVLAD for addressing Visual Place Recognition (VPR) task. The proposed SuperVLAD descriptor is an improvement to NetVLAD descriptor, by reducing the dimensionality, by using fewer clusters and
strength_weakness: strength_weakness: 1. It is a bit difficult to understand what contributes to the results; there is a table (Tab 6) comparing the influence of number of clusters, and comparison on backbones (e.g. Tab 3). It would be nice to have a number of clusters column in Tab3 - to show the influence of number of clusters for a given backbone, i.e. help understand where the improvements come from.
From Tab 3 -- the results, on the same backbone, seem on par with NetVLAD -- it might be worth emphasizing that this is achieved with fewer parameters, due to the smaller number of cluster centers.
2. What is the impact of discarding cluster centers? It is not clear from the text / tables if discarding the cluster centers influences the results or not.
3. Evaluation on SPED dataset (lower quality images) might benefit even more from adding ghost cluster (Tab 4).
4. Missing evaluation on Pittsburgh-250k and Tokyo 24/7 -- How does SuperVLAD perform for changing environment conditions, such as day-night?
5. MixVPR achieves comparable results on Pitts30k -- with a much smaller architecture (ResNet-50). In Tab2 -- no mention on what data are the models trained.
questions: questions: 1. Results in Table 2 (NetVLAD) don't match the results in SALAD paper ([27] -- also wrongly cited as [40] in Tab 2). Which are the correct ones?
2. How would SuperVLAD perform if the backbone was ResNet-50? ( to make it easier to compare with existing methods, like MixVPR[2])

(minor) remark: adding Venue column in Tab 2 doesn't add any value to the evaluation -- might be worth using the space to add the backbone for the specific method, to make comparison easier.

3. What are the total number of parameters and number of trainable parameters of the methods compared in Tab2?
limitations: limitations: Somewhat unclear ablation -- information is present, but not aggregated, making it difficult to understand the contribution of individual components (e.g. removing cluster centers, reducing the number of clusters -- this could be a nice plot / individual rows in a table), adding ghost clusters, changing backbone -- i.e. highlighting the contribution of each change individually.
paper_summary: paper_summary: In this paper, the authors focus on reducing the dimension of NetVLAD method for the task of visual place recognition. More specifically, the proposed method named SuperVLAD combines previous techniques including the powerful DIVOv2 as backbone, free from clusters, and ghost clusters. Experiments demonstrate that the proposed method outperforms pervious methods on Pitts30k, MSLS-val, Nordl, and SPED datasets. Moreover, the extensive ablation studies are conducted the verify the effectiveness of different components.
strength_weakness: strength_weakness: My concern is about the comparison between the proposed SuperVLAD and NetVLAD. 

1.	According to the Table 7 in the appendix. Table 7 shows that with DINOv2 as backbone, when increasing the number of clusters from 4 to 64, NetVLAD does not improve the performance. Conversely, it loses the accuracy. This is not consistent with the claim of the paper that the proposed approach reduces the dimension of features given by NetVLAD as even with very smaller number of clusters, NetVLAD also has very good performance as SuperVLAD does. I am very curious what will happen if the number of clusters for NetVLAD and SuperVLAD is set to 1, 2, 3.

2.	According to Table 4 and 6, the ghost cluster does not improve the performance when a the powerful DINOv2 is used as the backbone. Cross-image encoder contributes most to the improvements. It would be great to show the results of NetVLAD with 4 clusters, DINOv2, and cross-image encoder. Then we will know how many improvements come from other techniques used in SuperVLAD.
questions: questions: NA
limitations: limitations: Yes
paper_summary: paper_summary: The authors proposed a new method called SuperVLAD for visual place recognition. The authors aims to fix the shortage of the previously mature NetVLAD, which is NetVLAD is not robust against domain gap and have to use high dimensional features. 
SuperVLAD reduces the number of "clusters" used to smaller value and also propose a low-dimensional 1-Cluster VLAD descriptor, comparable in dimension to GeM pooling but with better performance. Experimental results show that SuperVLAD, when combined with a transformer-based backbone, outperforms NetVLAD, achieving better domain generalization and surpassing state-of-the-art methods on benchmark datasets.
strength_weakness: strength_weakness: 1. The novelty of the method is limited. It is a modification of the mature NetVLAD method and the modification appears to be small. 
2. Performance improvement is less obvious considering table 2. SuperVLAD shows minor differences between the second best method. 
3. To support the generalization claim, the authors need to show more comprehensive comparisons. How will the method perform if being used in dramatically different environments (such as training indoors but testing outdoors)? Indoor environments could be ScanNet or ETH3D, etc. So far, there isn’t an obvious domain gap between the datasets used in the generalization tests, and the generalization ability claim could be better justified.
questions: questions: There are inconsistencies in dataset usage between experiments in 4.3 (table 2) and generalization tests (table 3). Why do the two tables report accuracy from different set of datasets?
limitations: limitations: The performance improvement mostly comes from the DINOv2 backbone, as explained in section 4.3. Table 3 shows that if NetVLAD uses the ViT backbone, the performance gap between NetVLAD and SuperVLAD is less obvious. The authors also confirmed such shortcomes in Appendix A. The SuperVLAD no longer outperformed the NetVLAD in commonly used settings. 

Remaining suggestions not affecting rating:
It would be better to note the backbone features (e.g. DINOv2) in table 2. 
When explaining the difference in scene coverage between datasets in the ablation study, it would be useful to include examples of the dataset scenes so that readers can visualize the data domain.
paper_summary: paper_summary: The paper addresses the problem of learning a global image descriptor suitable for visual place recognition.
The work builds on top of NetVLAD, i.e. performs a soft assignment of local descriptors to learnable clusters, with the main difference that the model directly aggregates the local descriptors rather than residuals per cluster. This independence of the global embedding from cluster centers is argued to increase the robustness to domain shifts. 

The retrieval performance is increased by the incorporation of related works, namely ghost clusters [61] that allow to ignore ambiguous local descriptors, and a cross-image transformer within a training batch [39] to obtain viewpoint stability.
With DINOv2 as backend features, the approach shows SotA performance, comparable to recent works from CVPR'24.

Experimental results demonstrate that only 4 clusters are sufficient, leading to a compact descriptor. The authors also propose a 1-cluster VLAD (which can be seen as a binary classification of local descriptors useful/non-useful) resulting on a 768 dim. descriptor and SotA performance compared to using GeM pooling [47] or the ViT/DINOv2 class token (both of dim. 768)
strength_weakness: strength_weakness: **W1**: The proposed architecture is motivated by the missing capability of NetVLAD to generalize well across domains (line 51ff). Table 3 tries to make this point, and shows that SuperVLAD performs better than NetVLAD on MSLS when trained on Pitts30k. Though, the reported improvement of SuperVLAD over NetVLAD of 5pp is negligible when DINOv2 is used as backbone (0-2pp). When NetVLAD is trained on a more versatile dataset (GSV-Cities) its performance improves by ~30pp and the gap between SuperVLAD and NetVLAD becomes negligible. This demonstrates that usage of a large scale, and versatile training data set is much more important than the modified local descriptor aggregation (one of the main contributions of the paper.)

[The remaining contribution is that SuperVLAD results in a more compact descriptor.]

**W2** The novelty of the proposed work is limited. The main notable difference to previous work is the direct aggregation of descriptors rather than residuals per cluster. Otherwise the approach is very close to NetVLAD and borrows the ideas from [61] for "ghost clusters" and  [39] for cross-image interaction to obtain more robust features. To achieve SotA performance ghost clusters are not relevant (Tab. 4) but the cross-image encoder is required to achieve the performance of [39] (which is fair) and [40] 

**W3** The cherry picked results in Figure 4 do not provide much value, because such a matrix could be constructed in favor of any of the approaches. An illustration of typical failure cases of the proposed approach would be more helpful for the reader to understand the limitations of the approach.
questions: questions: 1) In the comparison in Table 5, what do you refer to as "class token"? If it's the ViT class token, is a fine-tuned on the task?

2) Figure 5 misses a point for CircaVPR, which has a Recall@1 of 94.9. What is its inference time?

3) To be able to compare all approaches 1:1 it would have been useful to use the same feature dimensionality for all of them, e.g. via training a final FC layer. Have experiments in this direction been conducted?
limitations: limitations: Limitations addressed adequately in the appendix. This discussion should be moved to the main paper.

Comments: title: Thanks for your reply
post rebuttal
Official Comment by Authors
Thank you for your reply
comment: We are pleased that our response has addressed your concerns. Many thanks for increasing the score.
Thanks for the rebuttal. I have no other concerns and upgrade my rating.
Dear Reviewer wUjJ,

Thanks again for your helpful comments and suggestions. We hope our responses could address your concerns. As the discussion period nears its end, please let us know if you have further questions or concerns. We'd be very glad to address them. 

Best regards,

Authors
Thanks a lot for your positive feedback and increasing the score! We'll be sure to incorporate your suggestions into our final paper.
Many thanks to the authors for the detailed and thorough answers to all the questions and concerns raised by myself and the other reviewers.

It would be worth including the answer to Q3 in the main paper, as it highlights a strength over SALAD ( the much smaller number of parameters in the aggregator ).

After reading through the other reviews and the answers, I am editing my initial review to increase the rating to 6.
Thanks Reviewer 7MN7 for asking about domain gap on indoor environments, and authors for running the experiment. While not familiar with BaiduMall of Sun et al (thanks for mentioning it!) - it seems suitable for the task of localization, while ScanNet was not designed for localization tasks (but could probably be split into query and db images).
Potential candidates could also be (maybe for future work)
* InLoc: Indoor Visual Localization with Dense Matching and View Synthesis, of Taira et al
* Matterport3D: Learning from RGB-D Data in Indoor Environments of Chang et al  (might need a split for query images).