GS-CPR: Efficient Camera Pose Refinement via 3D Gaussian Splatting
Authors: Changkun Liu
Shuai Chen
Yash Sanjay Bhalgat
Siyan HU
Ming Cheng
Zirui Wang
Victor Adrian Prisacariu
Tristan Braud
Venue: ICLR 2025 Poster (2025)
Keywords: Visual Localization
Camera Pose Estimation
3D Gaussian Splatting
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: we present GS-CPR, a novel test-time camera pose refinement framework leveraging 3DGS for scene representation.

Abstract: We leverage 3D Gaussian Splatting (3DGS) as a scene representation and propose a novel test-time camera pose refinement (CPR) framework, GS-CPR. This framework enhances the localization accuracy of state-of-the-art absolute pose regression and scene coordinate regression methods. The 3DGS model renders high-quality synthetic images and depth maps to facilitate the establishment of 2D-3D correspondences. GS-CPR obviates the need for training feature extractors or descriptors by operating directly on RGB images, utilizing the 3D foundation model, MASt3R, for precise 2D matching. To improve the robustness of our model in challenging outdoor environments, we incorporate an exposure-adaptive module within the 3DGS framework. Consequently, GS-CPR enables efficient one-shot pose refinement given a single RGB query and a coarse initial pose estimation. Our proposed approach surpasses leading NeRF-based optimization methods in both accuracy and runtime across indoor and outdoor visual localization benchmarks, achieving new state-of-the-art accuracy on two indoor datasets.

Published as a conference paper at ICLR 2025
GS-CPR: EFFICIENT CAMERA POSE REFINEMENT VIA 3D GAUSSIAN SPLATTING
Siyan Hul Zirui Wang? Victor Adrian Prisacariu Tristan Braud
Changkun Liu
Shuai Chen? Yash Bhalgat
Ming Cheng?
HKKUS
University of Oxford
Dartmouth College
Iterative Neural Refinement
One-shot GS-CPR (ours)
(ours)
Marepo
Marepo GS-CPR (ours)
Figure 1: GS-CPR refines pose predictions of state-of-the-ard APR and SCR models a one-shot manner, achieving greater accuracy compared the iterative neural refinement method, such as NeFeS (Chen et al., 2024a). Each subfigure is divided by a diagonal line, with the bottom left part rendered using the estimated/refined pose and the top right part displaying the ground truth image.
ABSTRACT
We leverage 3D Gaussian Splatting (3DGS) scene representation and propose a novel test-time camera pose refinement (CPR) framework, GS-CPR. This frame- work enhances the localization accuracy of fate-of-tthean absolute pose regres- sion and scene coordinate regression methods. The 3DGS model renders high- quality synthetic images and depth maps facilitate the establishment of 2D-3D correspondences. GS-CPR obviates the need for training feature extractors or de- scriptors by operating directly on RGB images, utilizing the 3D foundation model, MASt3R, for precise 2D matching. To improve the robustness of our model in challenging outdoor environments, we incorporate an exposure-adaptive module within the 3DGS framework. Consequently, GS-CPR enables efficient one-shot pose refinement given single RGB query and a coarse initial pose estimation. Our proposed approach surpasses leading NeRF-based optimization methods in both accuracy and runtime across indoor and outdoor visual localization bench- marks, achieving new state-of-the-ee art accuracy two indoor datasets. The project page is available at: https:/ / xrim- lab. qithub. Co/CS-CPR/
INTRODUCTION
Camera relocalization, the task of determining the 6-DoF camera pose within given environment based on a query image, critical for numerous applications, including robotics, autonomous vehi- cles. augmented reality, and virtual reality. Current methods for camera pose estimation primarily fall into the categories of structure-based approaches and absolute pose regression (APR) techniques. Classic structure- based pipelines (Dusmanu et al. 2019 Sarlin et al. 2019 Taira et al.) 2018 Noh et al. 2017: Sattler et al. 2016;; [Sarlin et al. 20200 Lindenberger et al. 2023) rely on 2D-3D cor- respondences between a point cloud and the reference image. Another class of structure-based
"cliudg coandd@@oonnet.cco hk, research conducted during visit Active Vision Lab, University of Oxford.

---

Published as a conference paper at ICLR 2025
2023 Wang et al. 2024a methods Scene Coordinate Regression (SCR) (Brachmann et al. 2017
Brachmann & Rother 2021) uses neural networks for direct regression of 2D-3D 3D correspondences.
These 2D-3D correspondences
fed into Perspective-n-Poing (PnP) (Gao et al., 2003) for pose
estimation. APR methods (Kendall et al. 2015 Wang et al. 2019 Chen et al. 2021 Shavit et al.) (02T] employ neural networks infer camera poses from query images directly. While APR ap- proaches offer fast inference times, they often struggle with accuracy and generalization (Sattler al. 2019; Liu al. 2024a). SCR methods generally achieve higher accuracy but the cost of increased computational complexity.
Pose Estimator
Pose Prior P
Rendered Depth T
Pose Rafined p
PNP + RANSAC
Query Image 4
Exposure Histogram
3D Gaussian Splat
2D-3D matches
+ ACT л E
Rendered Image T,
Matcher M
2D-2D matches
Figure 2: Overview of GS-CPR. We assume the availability of pre-trained pose estimator F and a pre-trained 3DGS model H of the scene. For a query image la, we first obtain an initial estimated pose ş from the pose estimator F. Our goal is to output refined pose p'.
Given the above limitations, there has been growing interest in pose refinement methods to hance the accuracy of the initial pose estimates of an underlying pose-estimation method. Recent approaches have leveraged Neural Radiance Fields (NeRF) (Mildenhall al.! 2020) for this pur- pose. For instance, NeFeS (Chen et al. 2024a) proposes test- time refinement pipeline. However, offers limited improvements in accuracy and suffers from slow convergence due the computa- tional demands of NeRF rendering and the requirement for backpropagation through the pose esti- mation model. Furthermore, recent NeRF-based localization method CrossFire (Moreau et al.) 2023 establishes explicit 2D-3D matches using features rendered from NeRF. However, training a customized scene model together with the scene-specific localization descriptor is required, and exhibits lower accuracy compared to classic structure-based methods.
To address the challenges of slow convergence, limited accuracy, and the need for training cus- tomized feature descriptors, propose novel test-time pose refinement framework, termed GS-
CPR. as illustrated in Figure land Figure 2 GS-CPE employs 3D Gaussian Splatting (3DGS) (Kerbl al. 2023 for scene representation and leverages its high-quality, fast novel view synthesis (NVS) capabilities to render images and depth maps. This facilitates the efficient establishment of 2D)-3D) correspondences between the query image and the rendered image, based on the initial pose estimate from the underlying pose estimator (e.g., APR, SCR). We incorporate an exposure-adaptive module into the 3DGS model to improve its robustness to the domain shift between the query image and the rendered image. Secondly, our method operates directly on RGB images, utilizing the 3D vision foundation model MASt3R (Leroy et al., 2024) for precise matching, eliminating the need for train- ing scene-specific feature extractors descriptors (Chen et al., 2024a Moreau et al. 2023). This significantly accelerates our method compared to iterative NeRF-based refinement methods (Chen
al.) 2024a) and makes framework casier to deploy than CrossFire (Moreau et al. 2023 and its variants (Zhou et al. 2024 Liu et al. 2023 Zhao et al. (0024).
Lastly, we conduct comprehensive quantitative evaluations and ablation studies the 7Scenes (Glocker et al. [0133] Shotton et 2013). 12Scenes (Valentin et al.) 2016). and Cambridge Landmarks (Kendall et al. 2015) benchmarks. GS-CPR significantly enhances the pose estimation accuracy of both APR and SCR methods across these benchmarks, achieving new sate-oof-the-a
accuracy the indoor datasets. Unlike previous NeRF-based methods (Chen al. 2024a)
2

---

Published as a conference paper at ICLR 2025
which fail to improve SCR methods, such ACE (Brachmann al. 2023
method offers sub-
stantial improvements and outperforms other leading NeRF- based methods
(Germain et al. 2022
Moreau et al. 2023 Zhou et al. 2024 Liu et al. 2023 Zhao et al. 2024).
RELATED WORK
Pose Estimation without 3D Representation.
straightforward approach for coarse pose esti-
mation is using image retrieval (Arandjelovic al. 2016 Ge et al. 2020 Gordo et al.
2017 to
average poses from top-retrieved images, but this lacks precision, Absolute Pose Regression (APR)
methods (Kendall et al., 2015 Kendall & Cipolla 2016
2017
Wang et al. 2019 Chen al. 2021
2022 Shavit et al. 2021 Chen et al. 2024b] Lin et al.
directly regress a pose from query image using trained models, bypassing 3D representations and geometric relationships. Despite be-
2024
ing fast, APR methods suffer in accuracy and generalization (Sattler et al. 2019: Liu et al. 2024a)
compared to structure-basee techniques. LENS (Moreau et al. (022) enhances APR by augmenting views with NeRF, but matching the accuracy of 3D structure- based methods remains challenging. To improve APR methods' accuracy, we used 3DGS as a 3D representation and utilized its geometry information to optimize the initial prediction.
Structure-based Pose Estimation. Classical 3D structure-bass -baue methods, like the hierarchical lo- calization pipeline (HLoc) (Dusmanu et al. 2019; [Sarlin et al., 2019 Taira et al. 2018 Noh et al.)
2017 Sattler et al. 2016 Sarlin et al. [220 Lindenberger et al. 2023), ) predict camera poses using a
point cloud and database of reference images, requiring descriptor storage and 2D-3D correspon-
dence through image retrieval.
contrast, Scene Coordinate Regression (SCR) methods (Brach-
mann et al., 2017 2023 Wang et al. 2024a Brachmann & Rother 2021) directly regress 2D>33D correspondences using neural networks and apply PnP (Gao et al.] (003) and RANSAC (Fischler & Bolles [981] for pose estimation. Our GS-CPR eliminates the need for reference images and de-
scriptor databases by using a 3DGS model for scene representation, further optimizing SCR outputs
like ACE (Brachmann et al. 2023).
NeRF-based Pose Estimation. NeRF-based pose estimation methods (Chen et al., 2024a Yen-
Chen et al. 2021| Lin et al. 20023) rely on iterative rendering and pose updates, leading to slow con- vergence and limited accuracy. While NeFeS (Chen et al.) (024a) improves APR pose estimation, it faces difficulties in enhancing SCR results and suffers from long refinement runtime. HR-APR (Liu et al. 2024a) speeds up optimization by 30%, but the average runtime of each query still takes sev- eral seconds on hiigh-performance GPU. Other bese- methods like FQN (Germain et al., 2022). CrossFire (Moreau et al., 2023). NeRFLoc (Liu et al.) 2023), and NeRFMatch (Zhou et al.,
2024 improve positioning by establishing 2D-3D matches but require specialized feature extractors and suffer from slow rendering and quality issues.
3DGS-based Pose Estimation. With the NVS field transitioning from NeRF to 3DGS, methods pro- posed by [Sun al. (2023) and [Botashev et al. (2024) refine camera poses in an inefficient iterative manner by inverting 3DGS, following iNeRF (Yen-Chen et al.] 2021). Ib. In contrast, 6DGS (Bortolon)
al. 2024) achieves one-shot estimate by projecting rays from an ellipsoid surface, avoiding it- eration. Although both methods use 3DGS for visual localization, neither has been tested on large benchmarks (Kendall et al. 2015 Valentin al. (016) or compared with mainstream methods like SCR and APR. We propose an approach using 3DGS for 2D-3D correspondences, similar to Cross- Fire (Moreau et al., 2023), but without requiring training feature extractors or feature matchers. Our method generates high-quality synthetic images and employs direct 2D-2D matching, making it faster and easier to deploy than previous NeRF-based methods such as NeFeS, CrossFire, and other
variants (Germain et al. 2022 Zhou et al. 2024
Liu al.
2023
2024a
Zhao al.. 2024).
PROPOSED METHOD
GS-CPR is a test-time camera pose refinement framework. We assume the availability of pre- trained pose estimator and a 3DGS model of the scene. For query image, we first obtain an initial estimated pose from the pose estimator. Our goal to output refined pose.
Given query image I E #xx33 with camera intrinsics K E R333, pose estimator F (typically
an APR or SCR model) predicts initial 6-DoF pose ý [ Rt where f. E R and R E R3x3
3

---

Published as a conference paper at ICLR 2025
represent the estimated translation and rotation respectively. Subsequently, for the viewpoint pretrained 3DGS model 'H renders image i, E #xxx3 and depth map la E "xx We use an exposure-adaptice affine color transformation (ACT) module & during this rendering process to enhance the robustness of our model to challenging outdoor environments (see Section 3.T)- A matcher M then establishes dense 2D-2D correspondences between l4 and f,. Then we can establish the 2D-3D matches based on Lg and L (see Section 3.2). Finally, we obtain the refined pose p' from these 2D-3D matches (see Section 3.2). An overview of our framework is depicted in Figure We also explore a faster pose refinement framework without 2D-3D matches depicted in Figure (see Section 3.3
3.1 3DGS TEST- TIME EXPOSURE ADAPTATION
Existing literature (Kerbl et al. 2023 Lu et al. 2024) shows that 3DGS achieves high-quality novel view renderings but assumes training and testing without significant photometric distortions. In visual relocalization, mapping and query sequences often differ in lighting due to varying times, weather, and exposure. This creates a significant appearance gap between 3DGS renderings and query images, negatively impacting 2D-2D matching performance.
To address this issue, we apply an exposure-adaptivy affine color transformation module (Chen al. 2022 2024a) to 3DGS, allowing the 3DGS adaptively render appearances during testing and accurately reflect the exposure of 7 Specifically, we use a 4-layer MLP that takes the luminance histogram of the query image as input and produces 3x3 matrix Q along with 3-dimensional bias vector b. These outputs are then directly applied to the rendered pixels of the 3DGS as shown in Equation| ensuring a closer match the exposure of the query image.
C(r) QCrend (r) b, where C(r) is the final per-pixel color and Crena(r) is the rendered per-pixel color obtained from 3DGS model H.
3.2 POSE REFINEMENT WITH 2D-3D CORRESPONDENCES
GS-CPR estimates the camera pose by establishing 2D-3D correspondences between the query im- age Ig and the scene representation. This process involves the following steps:
2D-2D Matching. First, image I, is rendered from the initial estimated viewpoint p. A Matcher M then used to establish 2D-2D pixel correspondences Cg.r between the query image Ig and the rendered image f.. In our implementation, the matcher M is a recently released 3D vision foundation model, MASt3R (Leroy al., 2024). MASt3R demonstrates strong robustness for 2D- 2D matching across image pairs with the sim-to-real domain gap.
3D Coordinate Map Generation. Simultaneously, we use our trained 3DGS model H. render depth map La from the viewpoint p. We modify the rasterization engine of 3DGS to render the depth map as follows:
i, x d,a VEN
(1- 1=1
where d; is the z-depth of each Gaussian in the viewspace and a; the learned opacity multiplied by the projected 2D covariance of the 2th Gaussian. In our framework, ground truth depth maps are not required for supervision during training of the 3DGS model H. Using the rendered depth map la. camera intrinsics K, and pose p. we obtain the 3D coordinate map X e H x3 for the rendered image f,.
Establishing 2D-3D Correspondences. By combining the 2D-2D correspondences Cg.r with the 3D coordinate map X... establish 2D-3D correspondences between I, and the scene. For each matched pixel in I we obtain its corresponding 3D coordinate from x
Pose Refinement. Finally, we obtain the refined pose & by feeding these 2D-3D correspondences into PnP Gao et al. 2003) solver with RANSAC (Fischler & Bolles 1981) loop. This process
4

---

Published as a conference paper at ICLR 2025
does not require backpropagation through the pose estimator FF or the 3DGS model H. ensuring efficient computation and enabling its usage with any black-box pose estimator model.
Using 2D-3D correspondences, coupled with PnP RANSAC, provides robust pose refinement that is much faster and more accurate than methods relying solely on rendering and compari-
method eliminates the requirement training specialized feature descriptors that previous approaches (Chen et al.
(Yen-Chen et al. 2021 Lin al. 2023 Sun al. 2023). Furthermore,
2024a Moreau et al., 2023 Chen al. 2022
Zhao et al., 2024 rely for robustness.
3.3
FASTER ALTERNATIVE WITH RELATIVE POST ESTIMATION
Pase Rror
Essimator
>F
Rerdend Depti
Rea fedt
Refuad
Query image 4
Exposure Histog am
Rendered
Matcher
M
Figure 3: Overview of GS-CPR Different from GS-CPR in Figure 2 (highlight with the red box), use La to recover the scale of trel. Then we calculate the refined pose p' based on Ryel and strel without matching.
While GS-CPR provides high accuracy through 2D-3D correspondences, we also explore an alter- native approach that prioritizes computational efficiency. This variant, which we call GS ES-CPRyet- utilizes MASt3R's point map registration capabilities to estimate relative pose without matching. Figure shows overview of the GS- GS-CPR,d approach.
Specifically, MASt3R generates point maps a and P, for both the query image I, and the rendered image 7, and predicts the relative rotation R and translation tze between the two images. However, this relative pose predicted by MASt3R needs be aligned to the scene's scale s. We recover the scale by aligning the point map P, with the depth map la rendered from the 3DGS model H. The final refined pose is computed as:
p' |R R R + st,el,
where R. é are the initial rotation and translation estimates. As shown in Table offers trade-off between speed and accuracy, making it ideal for rapid refinement of APR methods like DFNet (Chen al. 2022).
and GS-CPRgal
EXPERIMENTS
4.1 EVALUATION SETUP
Datasets.
We evaluate the performance of GS-CPR across three widely used public visual local- ization datasets. The 7Scenes dataset (Glocker et al. 2013 Shotton al. (013) comprises seven indoor scenes with volumes ranging from 1-18 m The 12Scenes dataset (Valentin et al. 2016) fea- tures 12 larger indoor scenes, with volumes spanning from 14-79m². The Cambridge Landmarks dataset (Kendall et al.) 2015) represents large-scale outdoor scenarios, characterized by challenges such as moving objects and varying lighting conditions between query and training images.
Evaluation Metrics. We report types of metrics to compare the performance of different meth- ods. The first metric is the median translation and rotation error. The second metric is the recall rate, which measures the percentage of test images localized within em and D
Baselines. In our experiment, to demonstrate the improvement capabilities of our framework, we the initial estimates of APR and SCR methods our baseline. We employ method top of

---

Published as a conference paper at ICLR 2025
Table 1: Comparisons on 7Scenes dataset. The median translation and rotation errors (cm/)) of different methods. The best results are in bold (lower is better). Second best results are indicated with an underline. NRP stands for Neural Rendering-bart -basd Pose Estimation.
Methods
Chess
Fire
Hleuds
Office
Pumpkin
Redliitchn
Stairs
Avg. 4 [em/"]
PoseNet (Kendall et 2015
1044,22
27/10.0
18/13.0
17/5.97
19/4.67
22/5.91
35/10.5
21/7.74
APR
MS Transforinet Siavvi et ml 2021
11/6.38
23/11.5
13/13.0
18/8.14
1778.42
16/8.92
29/10.3
18/9.51
DFNNet Chelt
3/1.12
6/2.30
6/1.54
7/1.92
7/1.74
12/2.63
6/1.93
Marepo Chen et al
2
1,90883
2.300.92
2.1/1.24
2.900.93
2.5/0.88
2.9/0.98
5,91148
2.9/1.44
DSAC*
Brachmann &
2021
0.5/0.17
0.840.28
0,5/0
1.2/0.34 34
1.2/0.28
0.7/0.21
2.7/0.78
1
SCR
ACB Braclimann ef m 21763
0.5
0.8/0.53
0.5/0.53
1.0/0.29
1.00.22
0.8/0.2
2.9/0.81
1. 34
GLACE (aang,
0.6/0.18
0,9/0.34
0,6/0.3 34
1/1/0.29
0.9/0.23
0.8/0.20
3.2/0.93
1.2/0.36
FQN-MN (Grmain et al. 2022
4,1/1.31
10.5/2.97
9.2/2.45
3.6/2. 36
4.6/1 76
16, 1/4.42
[39. 5/34.67
28/7.3
CrossFire Moreait
1/0.4
5/1.9
3/233
5/1.6
3/.8
2/0.8
12/1.9
4.4/1.38
PNeRFLoc Zhao Ei 2012
2/18
2/L,88
1/0.83
3/1.05
6/1.51
5/1.54
32/5.73
7.3/1.76
DFNet + NeFeSm Clva
2024a
210.57
2/0.74
2/1.28
2/0.56
2/0.55
2/0.57
5/1.28
2.4/0.79
NRP
McREMatch Zhou &t al. 2024
HR-APR [iil di PAZ241
0.9/0.3
2/0.55
1,1/0.4
2/0.75
1,5/L0
2/1.45
3,0/0.8
2/0.64
2.2/0.6
2/0.62
1,0/0.3
2/0.67
10.1/1.7
5/1.30
2.4/0.89
2.8/0.7
MCLoe Trivigno
2/0.8
3/1.4
5/16
6/1.
6/2.0
4.1/1.43
DFNer GS-CPK (ouns))
0.7/0.20
0.9/0.32
0.60 36
1,2/0.32
1.3/0.11
0.9/D.25
2.2/0.61
1.1/0.34
Marepo GS-CPR (ours)
0.640.18
0.740.28
0.5/0.32
1.1/0.29
L.00.26
0.8/0.21
1.5/0.4
0.9/0.28
ACE + GS-CPR (ours)
0.5/0.15
0.6/0.25
0.4/0.28
0.900.26
1.00.23
0.7/0.17
1.4/0,42
0.8/0.25
the prevailing APR methods, DFNett (Chen et al. 2022) and Marepo (Chen et al. 2024b), as well as well-known SCR method, ACE (Brachmann et al. 2023),, the pose estimator F. We follow the default settings of these pose estimators to obtain the initial pose prior for each query imag The
term APR/SCR + GS-CPR denotes the one-shot refinement. A similar naming convention applies
to APR/SCR + GS-CGP RS-CCP We also include a comparison here with the state-of-the-are NeRF-based
methods (Chen al.) 2024a Moreau al. 2023 Zhou al. 2024 Liu et al.] 2024a Germain
itt al., [0222 Zhao et al. 2024: Liu et al. 2023) and MCLoc (Trivigno et al. (024), which is
pose refinement framework agnostic scene representation. MCLoc provides results using 3DGS
models as scene representations for the 7Scenes and Cambridge datasets.
Implementation Details. GT Poses: For both the 7Scenes and 12Scenes datasets, we adopt the SIM ground truth (GT) provided by BBachhaan et al.) (2021). As demonstrated in NeFeS (Chen et al.) 2024a), SfM GT can render superior geometric details compared to dSLAM GT for the 7Scenes dataset. Gaussian Splatting: For the training of the 3DGS model of each scene, we utilize the sparse point cloud of training frames generated by COLMAP (Schonberger & Frahm] 2016) as the initial input. We select Scaffold-GS (Lu et al. 2024) as our 3DGS representation, incorporat- ing modifications detailed in Sections 3. and 3.2 to adapt exposure and enable depth rendering. Scaffold-GS reduces redundant Gaussians while delivering high-quality rendering compared to the vanilla 3DGS (Kerbl et al.] 2023). For the exposure- exposure-adaptive ACT module, we follow the default setting in Chen et al. (2024a), computing the query image's 's histogram in the YUV color space and binning the luminance channel into 10 bins. In addition, we apply temporal object filtering to filter out moving objects in the dynamic scene using an off-the-shelf method (Cheng et al. 2022), lead- ing to better accuracy in scene reconstruction quality and pixel-matching performance. Training Details: We employ the official pre-trained MASt3R (Leroy al. 2024) model without fine-tuning for 2D-2D matching and resize all images to 512 pixels on their largest dimension. The modified Scaffold-Ge GS model is trained for each scene with 30,000 iterations on an NVIDIA A6000 GPU. We implement our framework with Py PyTocch (Paszke et al. 2019). Additional details can be found in the Appendix A. and A.2
4.2 LOCALIZATION ACCURACY
We conduct quantitative experiments on three datasets evaluate the improved localization accu- racy of our framework compared to the APR and SCR methods.
7Scnnes Dataset. Using the 7Scenes dataset, we evaluate the performance of DFNet, Marepo, and ACE with GS-CPR. Table demonstrates that GS-CPR significantly reduces pose estimation errors for DFNet, Marepo, and ACE with one-shot refinement. Table 2 shows that GS-CPR significantly improves the proportion of query images below 5cm, 5" and 2cm, 2 pose error. It is worth noting
Note that the original paper of Marepo reports results on 7cenne using dSLAM GT; we retrained the ACE head of Marepo using SfM GT.

---

Published as a conference paper at ICLR 2025
Table We report the average percentage (%) of frames below a (5cm, 5%) and (2cm, 29) pose error across 7Scenes. IR denotes image retrieval.
Methods
Avg. [6em, 5"]
Avg. 1 [2em, 2
DFNet
43.1
8.4
APR
Marepo
84.0
33.7
oe (SP- + SCi (Sarlin et 2020 2019
95.7
84.5
IR+SIM points
DVLAD+R2D2 Turri al. 2015 Kevaud el al 2019
95.7
87.2
DSAC* ACE
97.8
80.7
SCR
97.1
83.3
GLACE
95.6
82.2
DFNet NeFeSpo
78.3
45.9
HR-APP
76.4
40.2
NeRFMatch
78.4
NRP
NeRFLoc (Liu ct al. 2023
89.5
DFNet GS-UPK (ours)
94.2
76.5
Marepo GS-CPR (ours)
99.4
89.6
ACE GS-CPR (ours)
100
93.1
Table 3: Comparisons on Cambridge Landmarks dataset. We report the median translation and rotation errors (cm/") of different methods. Best results are in bold (lower is better) among the NRP approaches.
Methods
Kings Hospital
Shop
Church
Avg. [cm//
11 (SP+SG) (k=1)
13/0.22 18/0.38
6/0.25
9/0.28
12/0,28
SFM points
oLoc (SP+SG) (k=10)
11/0.2 15/0.31
4/0.21
7/0.22
9/0.24
PoseNat
93/2.73
147/6.62
62
237/5.94
175/5.79
MS Transfommer
85/1.45 175/2.43
88/3.20
166/4.12
129/2.80
APR
LENS (Mreau al. 2022
330.5 44/0.9
27/1.6
53/1.6
39/1.
DFNer
73/2.37
200/2.90 2899
67/2.21
137/4.02
119/2.90
PMNet (Lin 2024
68/L97 103/1.31
58/2.10
133/3.73
90/2.27
ACE GLACE!
29/0.38
31/0.61
5/0.3
19/0.6
21/0.47
SCR
20/0332 20/0.41
5/0.22
9/0.3
14/0.32
FQN-MN
28/0.4
54/088
13/0.6
58/2
38/1
CrussFire
47/0.7
43/0.7
20/1.2
39/1.4
37/1
DFNet + NeFeSqo
37/064
98/1.61
17/0.60
42/1.38
49/1.00 0.66
NRP
DFNet NeFeSSo
37/0.54
52/0.88
15/0.53
37/1.14
35/0.77
HR-APR
36/058
53/0.89
13/0.51
38/1.16
35/0.78
MCLoc
31/0.42 39/0.73
12/0.45
26/0.8
27/0.6
DFNet GS-CPR (ours)
23/0.32
42/0.74
10/0.36
27/0.62
26/0.51
ACE GS-CPR (ours)
20/0.29 21/0.40 5/0.24
13/0.40
15/0.33
We report the accuracy based on official open-source models (Wang et al. 2024a).
Results of DFNet + NcFeS3o taken from Liu et al. (2024a).
that ACE + (G-CPP outperforms HLoc (Superpoint (De'Tone et al. 2018 Superglue (Sarlin al. 2.20), indicating that 3DGS has the potential to replace traditional point-clouds in visual localiza- tion pipelines. Figure (a) shows that after refinement using GS-CPR, the rendered image of the estimated pose better matches the real image.
Cambridge Landmarks Dataset. We conduct a quantitative evaluation by deploying DFNet and ACE with GS-CPR. Marepo is not included in this comparison due to the absence of an official model for this dataset. Table 3 demonstrates that GS-CPR significantly reduces pose estimation errors for both DFNet and ACE. Specifically, the accuracy of DFNet GS-CPP GS-CPR with one-shot optimization significantly surpasses that of CrossFire and DFNet NeFeS with 30 and even 50 steps of optimization (see Table 3. This result fully demonstrates the efficiency of our GS-CPR On the Kings College scene, DFNet GS-CPR outperforms ACE after our refinement. ACE GS-CPR consistently improves ACE accuracy across all four scenes. Refining the pose using method results in a rendered image that aligns more accurately with the ground truth image as illustrated in Figure (c).
12Scenes Dataset. We conduct the guantitative evaluation using Marepo and ACE with GS-CPR. The former works (Brachmann et al. 2023) Wang al. 2024a) report the percentage of frames below a 5cm, 5 pose error. Since SCR methods have already achieved good results with this metric, stringent standard (2cm, 2%) and report the median translation and
in this paper

---

Published as a conference paper at ICLR 2025
Table We report the average accuracy (%) of frames meeting a [5em, 5%), [2em, 29) pose error threshold, and the median translation and rotation emror (cm/") across 12Scenes.
Methods
Avg. Err lem/ /1 Avg. [5cm, 5
Avg. 1 [2em, 2
Marepo
2.1/1.04
95
50.4
DSAC*
0.5/0.25
99.8
96.7
ACE
0.7/0.26
GLACE
0.7/0.25
100
97.5
Marepo GS-CPR (ours)
0.7/0.28
98.9
90.9
ACE + GS-CPR (ours)
0.5/0.21
100
98.7
Table 5: We report the average accuracy (%) of frames meeting [5cm, 5%] pose error threshold, and the median translation and rotation errors (cm/").
Datasets
7Scenes
Cambridge
Methods
Avg. Ace f [5em, 5
Avg. Err [cm/1
Avg. Err [em/
DFNet
43.1
6/1.93
119/2.9 $5/0.57
DFNet + GS-CPR,, (ours)
80.5
2.7/0.38
DFNet GS-CPR (ours)
94.2
1.1/0.34
26/0.51
ACE
97.1
1.1/0.34
21/0.47
ACE GS-CPR,a (ours)
79.9
2.8/0.43
47/0.54
ACE GS-CPR (ours)
100
0.8/0.25
15/0.33
rotation errors (cm/"). Table shows that GS-CPR significantly improves the percentage of query images below 2cm, pose error and median pose error for Marepo and ACE. Figure 4 (b) shows that after refinement using our GS-CPR, the rendered image with our pose estimation aligns better with the real image.
GS-CPR vs. GS-CPR We compare GS-CPR, a pose refinement framework that uses 2D-3D correspondence, with GS-CPR a faster alternative that uses relative pose from MASt3R, Both frameworks evaluated 7Scenes and Cambridge Landmarks datasets using DFNet and ACE predictions. Table 5 shows that GS-CG CS-- achieves notable accuracy improvement with DFNet on both indoor and outdoor datasets, though it is less effective than GS-CPR. However, GS-CPR is significantly faster than GS-CPR and other NeRF-based methods, as discussed Section 4.33 While GS-CPR improves coarse pose estimates from APR methods like DFNet, struggles with accurate pose estimates from SCR methods. For ACE, GS-CPR,el results in performance degradation because our pose refinement relies on the relative pose estimator MASt3R, which struggles to provide more accurate relative pose estimates when the ACE- aCE-predieted pose is sufficiently close to the GT pose. Higher median rotation and translation errors in Table compared GS-CPP indicate that scale recovery is not the only challenge for GS-CPR as rotation is caleeinndependet.
ACE
ACE GS-CPR (Ours)
ACE
ACE (S-CPR(Oum)
ACE (G-GPR(DDuR)
Marapo
GS-OPR
Marapo
Marapo GS-CPR (Ours)
DFNoc
DFNer 08-CPR
(b)
Cambridge
Figure 4: Our GS-CPR enhances pose predictions for Marepo, DFNet, and ACE. Each subfigure divided by diagonal line, with the bottom left part rendered using the estimated/refined pose and the top right part displaying the ground truth image. Patches highlighting visual differences are emphasized with green insets for enhanced visibility.
8

---

Published as a conference paper at ICLR 2025
Table 6: Runtime Analysis (test on Cambridge Landmarks).
Mothods
CrossFire
DFNett NeFeSon
HR-APE,
MCLee
DFNet GB,CPR,a (ours) DFNet GS-CPR (ours) ACE + GS-CPR (ours)
Avg. [em/"] 37/1.
380.8
3M0.8
2780.6
55X0.6
26A15
15/03
Avg. time (9
0.3
2,4
Table 7: Results of different matchers (LoFTR (Sun al.. 2021), DUS$3R Wang al. 2024b)
and MASt3R (Leroy et al., 2024)) on the 7Scenes dataset. GS-CPR denotes using LoFTR as the matcher M. GS-CPR denotes using DUSt3R as M, and GS-CPR M denotes using MAST3R as
M. The table presents median translation and rotation errors (cm/") of the different methods
Methods
Marepo
GS-CPR L GS-CPR GS-CPR M
ACE
GS-CPR
GS-CPR D
GS-CPR
Avg. [em/]
2.9/1.04 1.5/0.40 2.1/0.7
0.9/0.28
1.100.34
1.000.31
1.5/0.6
0.8/0.25
4.3 RUNTIME ANALYSIS
We evaluate the runtime of the proposed framework using an NVIDIA GeForce RTX 4090 GPU. On average, 3DGS rendering takes 3.7 ms on the 7Scenes dataset and 12 ms on the Cambridge Land- marks dataset (due to higher scene complexity and image resolution), MASt3R relative pose estima- tion takes 71 ms. MASt3R matching takes additional 42 ms, and PnP+RANSAC takes another 52 ms. As a result, our GS-CPR only adds ms to the inference time of the pose estimator F, and our GS-CPR ESsCPR adds less than 180 overhead. All time measurements We compare the runtime and accuracy with other methods Table 6 On the Cambridge Landmarks dataset, MCLoc requires average of 2.4 per query with 80 iterations (Trivigno al. 2024). In contrast, our ACE+GS-CAE CPR with one-shot optimization only takes 0.19 per query. Therefore, in terms of efficiency and improvement, our GS-CPR is better than MCLoc when using 3DGS as scene representation. Although GS-CPR is less accurate than GS-CPR, it is more efficient. GS-CPR,al provides a feasible solution to pose refinement when the time budget is important.
averaged over 1,000 runs.
4.4 ABLATION STUDY
In this section, we first demonstrate the rationale behind selecting MASt3R as the matcher M in GS-CPR. Subsequently, we show that ACT effectively reduces the domain gap between the query image and the rendered image, thereby enhancing the refinement accuracy.
Different Matchers. We compare three matching methods: LoFTR (Sun et al.) 2021
DUS(3R (Wang et al. (024b). and MASt3R - within GS-CPR in the 7Scenes dataset. For DUSt3R and MASI3R, we resize all images to 512 pixels in their largest dimension. For LoFTR, pre-trained model for indoor scenes and maintain the frames in the 7Scenes dataset at 640 x 480. As shown in Table 7 Marepo GS-CPR and ACE GS-CPR using MASt3R M achieve the highest improvement. Conversely, ACE + GS-CPR using DUSt3R does not yield any improvement. Marepo GS-CPR using DUSt3R and Marepo/ACE GS-CPR using LoFTR show lower improvement compared to MASt3R. These results validate our choice of design to use MASt3R as a matcher M.
the
Affine Color Transformation. To enhance the robustness of the 3DGS model in image rendering and to reduce the domain gap between the rendered image and the query image, we incorporated an ACT module into the Scaffold-GS model, as described in Section 3.1 Figure 5 illustrates the improvement in image rendering quality with the ACT module applied. The performance enhance- ment of ES-CPR from the ACT module is demonstrated Table 8 On the Cambridge Landmarks dataset, employing the ACT module in the DFNet GS-CPR setup reduces both the average median translation and rotation errors.
4.5 DISCUSSION
In this section, we provide additional insights and discussions of our design choices.
Replace Feature Descriptors. Given that 3DGS can render high-quality synthetic images f, in real-time, show that using a pre-trained 3D foundation model, MASt3R, can directly establish accurate 2D-2D correspondences Cz. between I and f, with sim-to-real domain gap. As demon-
9

---

Published as a conference paper at ICLR 2025
Table 8: Ablation study for ACT module Cambridge Landmarks dataset. We report the median
translation and rotation errors (cm/").
Methods
Kings Hospital
Shop Church|Avg. J [cm/"]
DFNet + GS-CPR (w/o. ACT) 54/0.84 12/0.34 34/0.72
34/0.59
DFNet + GS-CPR (w. ACT) 23/0.32 42/0.74 10/0.36 27/0.62
26/0.51
(a) Ground Truth
(b) Scaffold-GS/PSSNR.6.55 dB
(c) Ours/PSNR:18.44c2
Figure 5: Benefit of the ACT module. A regular 3DGS model tends to render images based on the lighting conditions and the appearance of its training frames, as demonstrated by the synthetic view of Scaffold-GS in (b). However, in challenging visual localization datasets, such as ShopFacade in the Cambridge Landmarks, some query frames may have different exposures compared to the training frames. (c) Our proposed Scaffold-GS + ACT can adaptively adjust the exposure based on the query's 's histogram.
strated in Section 4.2 GS-CPR achieves significantly higher accuracy than NeRF-based refinement pipelines that rely on feature rendering. Direct RGB matching makes our framework more compact, reduces runtime, eliminates the need for training additional neural radiance features, and simplifies both deployment and usage.
Efficient and Effective Pose Refinement, As pose estimator, DFNet provides less accurate pre- dictions than Marepo and ACE, but NeFeS reports the best results over DFNet. To ensure a fair comparison with NeFeS, we present examples in Figure iillsstratnn that GS-CPR outperforms NeFeS in both efficiency and effectiveness. With only one-shot optimization, our GS-CPR achieves higher accuracy than NeFeS with 50 optimization iterations when combined with DFNet both the indoor 7Scenes and outdoor Cambridge Landmarks datasets. This superior performance is due our method's leverage of 3D geometry (depth rendering) of the representation, unlike previous NeRF-based refinement methods (Chen et al. 2024a Yen-Chen et al. (021) that use only 2D fea- ture/photometric information in an iterative process, rendering candidate poses and comparing them with the target image. Additional discussion can be found in the Appendix A.3
DFNetsNorss
DFNcc GE-CPRIS
DFNC-NoFRS
DFN 08-CPR
(a) 7Scenes
(b) Cambridge
Figure 6: A comparison between DFNet GS-CPR and DFNet + NeFeS50-
5 CONCLUSION
We present GS-CPR, novel test-time camera pose refinement framework leveraging 3DGS for scene representation to improve the localization accuracy of state-of-the-art APR and SCR methods. GS-CPR enables one-shot pose refinement using only single RGB query and coarse initial pose estimate from APR and SCR methods. Our approach outperforms existing NeRF-basee optimization methods in both accuracy and runtime across various indoor and outdoor visual localization bench- marks, achieving new tatte-of-theatt accuracy on two indoor datasets. These results demonstrate the effectiveness and efficiency of our proposed framework.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Relja Arandjelovic, Petr Gronat, Akihiko Torii, Tomas Pajdla, and Josef Sivic. Netvlad: Con ar- chitecture for weakly supervised place recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5297-5307, 2016.
Matteo Bortolon, Theodore Tsesmelis, Stuart James, Fabio Poiesi, and Alessio Del Bue. 6dgs: 6d pose estimation from single image and a 3d gaussian splatting model. arXiv preprint (rXiv:2407.15484 2024.
Kazii Botashev, Vladislav Pyatov, Gonzalo Ferrer, and Stamatios Lefkimmiatis. Gsloc: Visual localization with gaussian splatting. 2024 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.
IEEE, 2024.
Eric Brachmann and Carsten Rother. Visual camera re-localization from rgb and rgb-d images using dsac. IEEE transactions pattern analysis and machine intelligence, 44(9))5847-558) 2021.
Eric Brachmann, Alexander Krull, Sebastian Nowozin, Jamie Shotton, Frank Michel, Stefan Gumhold, and Carsten Rother. Dsac bsac-differentiable ransac for camera localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2017.
Eric Brachmann, Martin Humenberger, Carsten Rother, and Torsten Sattler. On the limits of pseudo ground truth in visual camera re-localisation. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6218-62122 022822 2021.
Eric Brachmann, Tommaso Cavallari, and Victor Adrian Prisacariu. Accelerated coordinate encod- ing: Learning relocalize in minutes using rgb and poses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 5044-5053, 2023.
Shuai Chen, Zirui Wang, and Victor Prisacariu. Direct-posenet: absolute pose regression with pho- tometric consistency. In 2021 International Conference on 3D Vision (3DV), pp. 1175-1185. IEEE, 2021.
Shuai Chen, Xinghui Li, Zirui Wang, and Victor A Prisacariu. Dfnet: Enhance absolute pose regres- sion with direct feature matching. In Computer Vision-ECCo 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part X, pp. 1-17. Springer, 2022.
Shuai Chen, Yash Bhalgat, Xinghui Li, Jia-Wang Bian, Kejie Li, Zirui Wang, and Victor Adrian Prisacariu. Neural refinement for absolute pose regression with feature synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20987-20996, 2024a.
Shuai Chen, Tommaso Cavallari, Victor Adrian Prisacariu, and Eric Brachmann. Map-relative pose regression for visual re-localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20665-206644 2024b.
Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked- attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1290-1299, 2022.
Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich, Superpoint: Self-supervised interest point detection and description. In Proceedings of the IEEE conference computer vision and pattern recognition workshops, pp. 224-236, 2018
Mihai Dusmanu, Ignacio Rocco, Tomas Pajdla, Mare Pollefeys, Josef Sivic, Akihiko Torii, and Torsten Sattler. D2-net: A trainable enn for joint description and detection of local features. In Proceedings of the ieee/cvf conference computer vision and pattern recognition, pp. 8092- 8101, 2019.
2019.
Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24 (6):381-395, 1981.
11

---

Published as a conference paper at ICLR 2025
Xiao-Shan Gao, Xiao-Rong Hou, Jianliang Tang, and Hang-Fei Cheng. Complete solution clas- sification for the gesppective-thre-poor problem. IEEE transactions on pattern analysis and machine intelligence, 25(8):930-99) 943, 2003.
Yixiao Ge, Haibo Wang, Feng Zhu, Rui Zhao, and Hongsheng Li. Self-supervising fine-grained re- gion similarities for large-scale image localization. In European Conference on Computer Vision, 2020.
Hugo Germain, Daniel DeTone, Geoffrey Pascoe, Tanner Schmidt, David Novotny, Richard New- combe, Chris Sweeney, Richard Szeliski, and Vasileios Balntas. Feature query networks: Neural surface description for camera pose refinement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5071-5081, 2022.
Ben Glocker, Shahram Izadi, Jamie Shotton, and Antonio Criminisi. Real-time rgh-d camera relo- calization. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). pp. 173-179. IEEE, 2013.
A. Gordo, J. Almazan, J. Revand, and D. Larlus. End-to-end learning of deep visual representations for image retrieval. IJCV, 2017.
Martin Humenberger, Yohann Cabon, Noć Pion, Philippe Weinzaepfel, Donghwan Lee, Nicolas Guérin, Torsten Sattler, and Gabriela Csurka. Investigating the role of image retrieval for visual localization: An exhaustive benchmark. International Journal of Computer Vision, 130(7):1811- 1836, 2022.
Journal
Jianhao Jiao, Jinhao He, Changkun Liu, Sebastian Aegidius, Xiangcheng Hu, Tristan Braud, and Dimitrios Kanoulas. Litevloc: Map-lite visual localization for image goal navigation. arXiv preprint arXiv:2410.044111 2024.
Alex Kendall and Roberto Cipolla. Modelling uncertainty in deep learning for camera relocalization. In 2016 IEEE international conference on Robotics and Automation (ICRA), pp. 4762 4769. IEEE, 2016.
Alex Kendall and Roberto Cipolla. Geometric loss functions for camera pose regression with deep learning. Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 2017.
Alex Kendall, Matthew Grimes, and Roberto Cipolla. Posenet: A convolutional network for real- time 6-dof camera relocalization. In Proceedings of the IEEE international conference on com- puter vision, pp. 2938 2946, 2015.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Trans. Graph., 42((): 139-11 2023.
Vincent Leroy, Yohann Cabon, and Jérôme Revaud, Grounding image matching in 3d with mast3r. In European Conference Computer Vision, pp. 71-91. Springer, 2024.
Jingyu Lin, Jiaqi Gu, Bojian Wu, Lubin Fan, Renjie Chen, Ligang Liu, and Jieping Ye. Learning neural volumetric pose features for camera localization. In European Conference on Computer Vision, pp. 198-214. Springer, 2024.
Yunzhi Lin, Thomas Müller, Jonathan Tremblay, Bowen Wen, Stephen Tyree, Alex Evans, Patri- cio A Vela, and Stan Birchfield. Parallel inversion of neural radiance fields for robust pose estimation. In 2023 IEEE International Conference Robotics and Automation (ICRA), pp. 9377-9384. IEEE, 2023.
Philipp Lindenberger, Haul-Edouard Sarlin, and Marc Pollefeys. Lightglue: Local feature matching at light speed. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17627-17638, 2023.
Changkun Liu, Shuai Chen, Yukun Zhao, Huajian Huang, Victor Prisacariu, and Tristan Braud. Hr- apr: Apr-agnostic framework with uncertainty estimation and hierarchical refinement for camera relocalisation. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp- 6544-85500 IEEE, 2024a.
12

---

Published as a conference paper at ICLR 2025
Changkun Liu, Jianhao Jiao, Huajian Huang, Zhengyang Ma, Dimitrios Kanoulas, and Tristan Braud. Air-hloc: Adaptive retrieved images selection for efficient visual localisation. arXiv preprint arXhv:2403.18281, 2024b.
Jianlin Liu, Qiang Nie, Yong Liu, and Chengjie Wang. Nerf-loc: Visual localization with condi- tional neural radiance field. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 9385-9392. IEFE, 2023.
Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold-gs: Structured 3d gaussians for view-adaptinn rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20654-20664, 2024.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In European Conference Computer Vision, pp. 405-421. Springer, 2020.
Arthur Moreau, Nathan Piasco, Dzmitry Tsishkou, Bogdan Stanciulescu, and Arnaud de La Fortelle. Lens: Localization enhanced by nerf synthesis. In Conference on Robot Learning, pp. 1347-1356. PMLR, 2022.
Arthur Moreau, Nathan Piasco, Moussab Bennehar, Dzmitry Tsishkou, Bogdan Stanciulescu, and Arnaud de La Fortelle. Crossfire: Camera relocalization on self-supervised features from an implicit representation. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 252-262, 2023.
Hyconwoo Noh, Andre Araujo, Jack Sim, Tobias Weyand, and Bohyung Han. Large-scale image retrieval with attentive deep local features. In Proceedings of the IEEE international conference on computer vision, pp. 3456-3465, 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
Jerome Revaud, Cesar De Souza, Martin Humenberger, and Philippe Weinzaepfel. R2d2: Reliable and repeatable detector and descriptor. Advances in neural information processing systems, 32, 2019.
Paul-Edouard Sarlin, Cesar Cadena, Roland Siegwart, and Marcin Dymczyk. From coarse fine: Robust hierarchical localization at large scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2019.
Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 4938 4947, 2020.
Paul-Edouard Sarlin, Mihai Dusmanu, Johannes L Schönberger, Pablo Speciale, Lukas Gruber, Vik- tor Larsson, Ondrej Miksik, and Mare Pollefeys. Lamar: Benchmarking localization and mapping for augmented reality. In European Conference on Computer Vision, pp. 686-704. Springer, 2022.
Torsten Sattler, Bastian Leibe, and Leif Kobbelt. Efficient & effective prioritized matching for large- scale image-based localization. IEEE transactions on pattern analysis and machine intelligence, 1909):1744-17555 2016.
Torsten Sattler, Qunjic Zhou, Mare Pollefeys, and Laura Leal-Taixe. Understanding the limitations of cnn-based absolute camera pose regression. Proceedings of the IEEE/VV conference on computer vision and pattern recognition, pp. 3302-3312, 2019.
Johannes L Schonberger and Jan-Michael Frahm. Structure- from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4104-4113, 2016.
Yoli Shavit, Ron Ferens, and Yosi Keller. Learning multi-scens absolute pose regression with trans- formers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2733-2742. 2021.
13

---

Published as a conference paper at ICLR 2025
Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene coordinate regression forests for camera relocalization in rgb-d images. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2930-2937, 2013.
Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. Loftr: Detector-free local feature matching with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8922-8931, 2021.
Yuan Sun, Xuan Wang, Yunfan Zhang, Jie Zhang, Caigui Jiang, Yu Guo, and Fei Wang. icomma: Inverting 3d gaussians splatting for camera pose estimation via comparing and matching. arXiv preprint wrwiv:2312.00000000 2023.
Hajime Taira, Masatoshi Okutomi, Torsten Sattler, Mircea Cimpoi, Marc Pollefeys, Josef Sivic, Tomas Pajdla, and Akihiko Torii. Inloc: Indoor visual localization with dense matching and view synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7199-7209, 2018.
Akihiko Torii, Relja Arandjelovic, Josef Sivic, Masatoshi Okutomi, and Tomas Pajdla. 24/7 place recognition by view synthesis. Pruceedings of the IEEE conference on computer vision and pultern recognition, pp. 1808-1817, 2015.
Gabriele Trivigno, Carlo Masone, Barbara Caputo, and Torsten Sattler. The unreasonable effec- tiveness of pre-trained features for camera pose refinement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12786-12798, 2024.
Julien Valentin, Angela Dai, Matthias Nießner, Pushmeet Kohli, Philip Torr, Shahram Izadi, and Cem Keskin. Learning to navigate the energy landscape. In 2016 Fourth International Conference on 3D Vision (3DV), pp. 323 332. IEEE, 2016.
Bing Wang, Changhao Chen, Chris Xiaoxuan Lu, Peijun Zhao, Niki Trigoni, and Andrew Markham. Atloc: Attention guided camera localization. arXiv preprint arXiv:1909.03555, 2019.
Fangjinhua Wang, Xudong Jiang, Silvano Galliani, Christoph Vogel, and Mare Pollefeys. Glace: Global local accelerated coordinate encoding. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 21562-21571, 2024a.
Shuzhe Wang. Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Ge- ometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20697-20709 2024b.
Lin Yen-Chen, Pete Florence, Jonathan T Barron, Alberto Rodriguez, Phillip Isola, and Tsung- Yi Lin. inerf: Inverting neural radiance fields for pose estimation. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1323-1330. IEEE, 2021.
Boming Zhao, Luwei Yang. Mao Mao, Hujun Bao, and Zhaopeng Cui. Pnerfloc: Visual localization with based neural radiance fields. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 7450-7459, 2024.
Qunjie Zhou, Maxim Maximov, Or Litany, and Laura Taiie The nerfect match: Exploring nerf features for visual localization. European Conference on Computer Vision, pp. 108-127. Springer, 2024.
14

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 GT POSES DETAILS
In Section 4.2 report evaluation results based the SfM ground truth (GT) poses for the 7Scenes dataset, as these poses can render higher quality images (Chen et al.
2024a Since NeFeS (Chen
et al. 2024a) demonstrates the superior accuracy of SfM poses using NeRF the scene representa-
tion, we provide quantitative comparison and illustrative rendering examples of 3DGS in Figure 7 These results affirm that SIM poses are more accurate, leading to higher quality rendered images and depth maps when using 3DGS. We utilize pre-built COLMAP models from Brachmann al. (2021) for 7Scenes and 12Scenes datasets, and the models from HLoc toolbox (Sarlin al. 2019 for the Cambridge landmarks dataset. For the 7Scenes dataset, we enhance the accuracy of the sparse point cloud by utilizing dense depth maps provided by the dataset, combined with the Hloc toolbox and rendered depth maps (Brachmann & Rother 2021).
Table 9: Quatitative comparison between the 3DGS models implemented in Section trained by dSLAM GT poses and SfM GT poses. We report the average PSNR (dB) for the test frames in each scene. The best results in bold (higher is better).
dSLAM GT
SIM GT
Scenes
avg. PSNR
avg. PSNR t
chess
19.6
23.1
fire
21.2
heads
18.4
19.7
office
19.4
21.7
pumpkin
20.3
23.2
redkitchen
18.5
21.4
stairs
19.7
20.1
avg.
19.4
21.5
(5) dSLAM GT
GIM
Render performance example (dSLAM GT vs. SfM GT). The 3DGS model trained with SfM GT poses (b) renders superior geometric details compared the dSLAM 3DGS (a) for the same query image, particularly in the chessboard and pieces area.
Figure 7:
A.2 SEMANTIC SEGMENTATION WHEN BUILDING 3DGS
To handle challenges outdoor datasets, we apply temporal object filtering to filter out moving objects in the dynamic scene using off-the-shelf method (Cheng al. 2022), leading to better accurate scene reconstruction quality and pixel-matching performance. We show examples of se- mantic segmentation in Figure 8 and its effect novel view synthesis (NVS) results in Figure 9 This approach, together with ACT, allows our 3DGS models to provide more robust and better ren- dering results.
15

---

Published as a conference paper at ICLR 2025
Figure 8: Example of masking on the ShopFacade scene. sponding semantic masks.
Top: original images; Bottom: corre-
Gl
(ol/w, Seg Meok wv ACT
(b) wib. Sog Masi w. ACT
Image
Seg Meak ACT
Masi
Figure 9: Rendering performance comparison. The 3DGS model trained with segmentation masks renders superior geometric details and fewer artifacts compared to the model trained without masks.
A.3 THE ADVANTAGES GS-CPR OVER OTHER APPROACHES
Advantages render and comapre methods: Methods Yen-Chen et al. 2021 Lin al. 2023
Chen et al.] 2024a Sun et al. 2023 Trivigno et al.) 2024 leverage only the geometric information of the representation for rendering but do not use it for 2D-3D matching. Consequently, they offer limited accuracy gains and are hindered by slow convergence and high computational costs due to iterative rendering. While NeFeS (Chen et al. 2024a) reduces rendering time and cost by using feature maps and feature loss rather than photometric loss, its accuracy potential remains lower than methods employing 2D-3D matches from original RGB images due the loss of information in feature maps.
Advantages over structure-based methods: Classical 3D structure-based methods, such as Hloe (Dusmanu et al.) 2019) Sarlin et al.) 2019 Taira et al. 2018 Noh et al. 2017: Sattler et al.) 2016 Sarlin al.] 2020; Lindenberger et al. 2023), estimate camera poses using 3D SIM point cloud and a reference image database. HLoc requires storing descriptor database and retrieving the top-k most similar images for 2D-3D correspondences, typically requiring k= 5 to 40 images for robust localization (Humenberger et al. 2022 Sarlin et al.) 2022: Leroy et al. 2024). Our approach offers two key advantages: (I) While HLoc requires matching operations, our GS-CPE only re- quires one, and its single-shot pose optimization surpasses the accuracy of traditional HLoc. (2) For
16

---

Published as a conference paper at ICLR 2025
challenging queries, even the top-1 retrieved image may have limited overlap with the query (Lu jet al. 2024b). However, since GS-CPR performs NVS based on APR and SCR predictions, the ren- dered images exhibit greater overlapping region with the query, leading to more accurate matches. We provide examples in Figure TO The key insight is that both image retrieval and ACE pose- based retrieval are restricted to identifying queries within limited reference pool. In contrast, our approach theoretically allows for an unlimited reference pool. (3) Using 3DGS instead of sparse point clouds for scene representation enables the domain shift of the rendered image according to the query's s exposure through learning approach, offering greater flexibility.
System design analysis: Our approach goes beyond simply combining 3DGS and MAS13R As outlined in Section 3.2 our method leverages the matching components of MASt3R to eliminate the need for training extra features to match image pairs with sim-to-real domain gap- -a common limitation of other NeRF-hased pose estimation techniques. However, relying solely on MASt3R with reference images fails to deliver accurate metric translation due the lack of scale informa- tion and cannot build 2D-3D matches for absolute pose estimation. This limitation arises because MASt3R is unable to generate metric 3D points within the pre-built global coordinate system. For instance. Diao et al. (2024) addresses this problem in robotics tasks by incorporating a depth camera. To resolve this challenge, 3DGS in our framework serves a critical function by rendering metric depth, enabling accurate 2D-3D matching. Besides, the rendered view generated by 3DGS from SCR and APR poses aligns much better than normal image retrieval from fixed reference images. This integration is important in recovering precise scale and achieving robust and accurate pose es- timation with sufficient matches. By combining the strengths of these components, our framework addresses current limitations.
A.4 SUPPLEMENTARY VISUALIZATION
To complement our quantitative analysis, we present additional results in Figure [ that provide qualitative perspective on pixel-wise alignment using NVS based on 3DGS across three datasets. A video is also included in the supplementary material.
A.5 FAILURE CASES AND LIMITATION
One limitation of our method lies in its dependency on the accuracy of the initial pose estimates provided by the pose estimator. When the initial pose is highly inaccurate, the overlap between the rendered images and the query image is insufficient to establish reliable 2D-3D correspondences for accurate pose estimation. As shown in Figure 12] GS-CPR cannot refine the DFNet's initial pose in this case because it is too far away from the GT pose.
Following Section 4.5 of NeFeS (Chen et al., 2024a), we conduct quantitative experiments to eval- uate the limitations of GS-CPR. Specifically, we introduce random perturbations to the ground truth poses of test frames on the ShopFacade scene, applying fixed magnitudes of rotational and trans- lational errors independently. The results after pose refinement using GS-CPR are presented in Table I0 and Table II Our framework can improve the accuracy when rotation error < 50% and translation error < 8 meters, respectively. In contrast, NeFeS achieves accuracy improvements only for rotational errors under 35' and translational errors below meters. These findings highlight that our method significantly expands the optimization range, enhancing its robustness larger pose perturbations.
Table 10: Average rotation error after refinement by GS-CPR.
Jitter-magniudee ())
5
10
20
30
40
50
55
60
Avg. Rot. Error (
0.23
0.23
0.27
0.35
0.6
7
26
83
Table 11: Average translation error after refinement by GS GS-CPR.
Jitter-magnitude (m)
2
3
4
5
6
8
10
Avg. Trans. Error (m)
0.19
0.38
0.51
0.88
1.13
2.0
3.1
10.7
17

---

Published as a conference paper at ICLR 2025
seq www-01/frreeeeee 868138
220-8//rrane-00655
scq-03/trame- 880869
query
top-1 retrieved image by NetVLAD
top-1 retrieved image ACE's pose
The rendered image pose
query retrieved
query retrieved image (ACE): 1258 (a) Stairs
query rendered image (ACE): 2968
image (NetVLAD): 1623
sec-99/ranee88441
seq-03/ 550-033/rane-00088 seq-88 beq-08/frane- htt/teennn00022222
query
top-1 retrieved image by NetVLAD
top-1 retrieved image based on ACE's pose
The rendered image
based on ACE's pose
query and the retrieved image NetVLAD): 2387
query and the retrieved image (ACE): 1640
query and the rendered
image (ACE): 2921
(b) Office seq-05/frea
seq-84/
scq-13/ far-n3/f 000788
query
top-1 retrieved image by NetVLAD
top-1 retrieved image based ACE's pose
The rendared Image
based ACE's pose
query and the retrieved image (NetVLAD): 2255
query and the retrieved image (ACE);
and the rendered image (ACE): 2970
(c) Redkitchen
The image rendered from the pose estimator's predictions exhibits a greater overlapping region with the query image than the one retrieved by NetVLAD (Arandjelovic et al. 2016) and the one retrieved by ACE's pose. We use MASt3R as the matcher. Since the matches are very dense, show the number of matches but only visualize 20% of the matches.
Figure 10:
18

---

Published as a conference paper at ICLR 2025
DFNet
DFNet GS-CPR
Marepo
Marepo+ GS-CPR (Ours)
ACE
ACE GS-CPR (Ours)
ACE
ACE GS-CPR (Ours)
DFNet NeFeS
DFNet GS-CPR (Ours)
DFNet NeFeS
DFNet GS-CPR (Ours)
Figure 11: Each subfigure is divided by diagonal line, with the bottom left part rendered using the estimated/refined pose and the top right part displaying the ground truth image. Patches highlighting visual differences are emphasized with green insets for enhanced visibility.
DFNet
DFNet +GS-CPR
Figure 12: Failure example. Each subfigure is divided by diagonal line, with the bottom left part rendered using the estimated/refined pose and the top right part displaying the ground truth image.
19

---

Published as a conference paper at ICLR 2025
This paper demonstrates the effectiveness of our framework commonly used datasets and bench- marks. However, reconstructing high-quality 3DGS models for large scenes remains a significant challenge. Exploring the application of this framework to large-scale scenes for accurate visual camera relocalization is promising avenue for future work.
We report the average accuracy (%) of frames meeting a [5em, 5%), [2cm, 29] pose error threshold, and the median translation and rotation errors (cm/") across 7Scenes and 12Scenes.
Table 12:
Datasets
Methods
Avg. Err + [cm/"] Avg. f [5em, 5 Avg. 1 [2em, 2")
GLACE
1.2/0.36
95.6
82.2
7Scenes
GLACE GS-CPR (ours)
0.8/0.27
99.5
90.7
GLACE GLACE GS-CPR (ours)
0.7/0.25
100
97.5
12Scenes
0.5/0.21
100
98.9
Table 13:
Comparisons on Cambridge Landmarks dataset. We report the median translation and rotation errors (cm/") of different methods.
Methods
Kings Hospital Shop Church Avg. & [cm/
GLACE!
20/0.32 20/0.41
5/0.22 9/0.3
14/0.32
GLACE²
19/0.3
17/0.4
4/0.2
9/0.3
12/0.3
GLACE GS-CPR (ours)
17/0.28
18/0.34
5/0.21 8/0.28
12/0.28
Accuracy based on official open-source models (Wang et al. Accuracy reported the paper (Wang et al. 2024a).
2024a).
A.6 SUPPLEMENTARY EXPERIMENTS
GLACE (Wang 2024a) an enhanced version of ACE tailored for large-scale outdoor scenes, while exhibiting nearly identical accuracy in indoor environments compared to ACE. We present the results of GLACE + GS-CPR in Tables 12 and 13 to provide supplementary results for evaluating the performance of our approach. GS- CPR significantly improves GLACE accuracies in two of three datasets (7scenes and 12scenes), demonstrating the effectiveness of our method. the Cambridge Landmarks dataset, we achieve comparable results, with slight edge in rotational error.
20


Reviews:
[{"review_id": "YVxJiG6TT3", "reviewer": "Reviewer_BjDV", "summary": "This paper presents a test-time refinement method, GSLoc, for improving camera localization accuracy. With a single RGB query, the method leverages ACT (exposure-adaptive affine color transformation) enhanced 3DGS and MASt3R to establish 2D-3D correspondences and then uses PnP + RANSAC for pose estimation. On two indoor and one outdoor datasets, the method achieved SOTA results.", "questions": "What are the failure cases and potential limitations of the work? Please provide some quantitative and qualitative results.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "++ The paper shows that, with introducing any new techniques, simply combining existing techniques can achieve state-of-the-art results for camera localization.", "weaknesses": "-- It appears that the authors misused \\citep and \\citet. This creates some difficulty when reading the paper.\n\n-- Formulas should be followed by punctuation marks.\n\n-- Please provide the detailed derivation of Eq. (3). It seems the translation part is incorrect.\n\n-- Please provide analyses on the failure cases and potential limitations of the work.\n\n-- The title is GSLoc, where GS stands for Gaussian Splatting, but the meaning of Loc is not mentioned in the paper. Since the focus of this work is on pose/localization refinement, Loc may not be the most suitable term.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KoDeAWtURZ", "reviewer": "Reviewer_rRXL", "summary": "This method applies 3DGS to address the test-time camera pose refinement task. The manuscript first renders RGB and depth images from 3DGS, then matches the query image with the rendered images. To bridge the domain gap between the query image and 3DGS, an exposure-adaptive module is incorporated. Additionally, MASt3R is used as a training-free feature extractor for accurate matching. Extensive experiments demonstrate the effectiveness of the proposed GSLoc method.", "questions": "Q1 - The sota comparision such as GLACE + GSLoc.\n\nQ2 - The quality of the rendered depth image.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1 - The task appears meaningful, and the results presented in the manuscript are quite promising.\n\nS2 - The Affine Color Transformation module is a noteworthy component, effectively addressing the RGB domain gap between the query image and the rendered images.\n\nS3 - The experiments are well-designed and effectively demonstrate the superiority of the proposed GSLoc method over various baseline approaches.", "weaknesses": "W1 - The method appears to lack comparisons with state-of-the-art approaches, such as GLACE + GSLoc and ACE/GLACE + NeFes. Including these comparisons would provide a clearer benchmark for evaluating its performance.\n\nW2 - I’m curious about the quality of the depth images rendered by 3DGS and whether they are consistent enough to serve as a reliable information source.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MRShqrd2Gy", "reviewer": "Reviewer_HMwQ", "summary": "This paper proposes a new approach to optimize the initial camera pose in visual localization. Using 3DGS as the scene representation, RGB images and depth maps are rendered based on the initial camera pose of the query image estimated by other methods. Then, 2D-2D matches between the query image and the rendered results are computed via MASt3R and elevated to 2D-3D matches using the depth maps. Finally, the relative pose is solved through PnP to optimize the initial camera pose. Compared to previous camera pose optimization methods based on cost functions and optimization, the GSLoc pipeline achieves one-shot pose optimization, offering advantages in both speed and accuracy. The effectiveness of GSLoc has been validated on indoor and outdoor visual localization datasets.", "questions": "- What are the feature detector and matcher used by HLoc in the comparison shown in Table 2?\n- In the runtime analysis, there seem to be some issues with the time calculation for GSLoc and GSLoc_rel.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- The GSLoc pipeline combines the advantages of speed and accuracy. Through high-quality 3DGS reconstruction and rendering, it directly calculates optimized camera poses using image matching, which enhances the robustness and usability of the system compared to optimization methods based on specific cost functions.\n- The writing of the paper is easy to follow.\n- Comprehensive experiments have been conducted, demonstrating advantages over other camera pose optimization methods and traditional visual localization approaches.", "weaknesses": "- The paper lacks certain experiments to demonstrate the necessity of the proposed method:\n    - Table 2 shows the effectiveness of GSLoc compared to HLoc. However, it does not pair HLoc with an improved initial pose estimation method, such as ACE, as done for GSLoc. A reasonable baseline would be to use ACE to estimate the initial camera pose, retrieve the top-K images with the closest poses from the database, and then compute the localization result using HLoc and optionally refinet it with PixLoc.\n- The novelty of the method is limited, as the effectiveness of the pipeline primarily relies on improvements in reconstruction and rendering quality provided by Scaffold-GS as the scene representation.\n- There is a lack of analysis on the method’s applicability:\n    - Feasibility of high-quality 3DGS reconstruction: Reconstructing high-quality 3DGS to render the images needed for GSLoc requires densely sampled viewpoints. Moreover, modeling requires images with viewpoints very close to that of the query. How robust is GSLoc when there are no views close to the query viewpoint (i.e., in cases of wide baselines)?\n    - Impact of significant appearance changes on the method (e.g., on datasets like Aachen-day-night or similar ones of smaller scenes).\n    - Feasibility of the method for large-scale visual localization (e.g., Aachen-day-night).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0Y0WYOVMau", "reviewer": "Reviewer_tSXc", "summary": "The paper pays attention on camera pose refinement with 3D Gaussian Splatting as a scene representation, which aims to enhance the localization accuracy of state-of-the-art absolute pose regression and scene coordinate regression methods. A refinement framework, called GSLoc, is proposed to establish 2D-3D correspondences by an exposure-adaptive module. Experiments demonstrate that GSLoc improves the localization accuracy over different relocaliztion methods.", "questions": "The question and suggestions are listed in the part of Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1 The paper proposes a novel test-time pose refinement framework, termed GSLoc, which employs 3D Gaussian Splatting for scene representation and utilizes the 3D vision foundation model MASt3R for precise matching.\n\n2 GSLoc enhances the pose estimation accuracy of both APR and SCR methods across these benchmarks.", "weaknesses": "1 The main contribution is incremental. \n\n(1)Firstly, the motivation of GSLoc is less interesting. From my viewpoint, the usage of feature matching for pose refinement is not novel. Meanwhile, the paper just uses 3D Gaussian Splatting for scene representation instead of traditional hand-crafted point cloud. Compared with Gaussian Splatting SLAM systems (like Gaussian SLAM [1, 2]), the idea of this paper is less innovative.\n\n(2)Secondly, the innovative contributions are few. The paper seems a combination of current hot technologies, like 3D Gaussian Splatting and the 3D vision foundation model MASt3R. \n\n[1]Gaussian Splatting SLAM, CVPR 2024\n\n[2]GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting\n\n[3]LoFTR: Detector-Free Local Feature Matching with Transformers, CVPR 2021\n\n[4]DKM: Dense Kernelized Feature Matching for Geometry Estimation\n\n2 The localization accuracy comparisons with feature matching based methods are missing. The core idea behind GSLoc is feature matching with 3D Gaussian Splatting. It is curious that whether GSLoc is also effective for pose refinement of leaned feature matching based methods, like LoFRT[3], DKM[4] or others. It is suggested to add these results. \n\n\n3 The overlap of Figure 2 and 3 is somwhat high. It is suggested to modify Figure 3 to express the difference of GSLocrel in comparison with GSLoc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "guzkPZ2xT9", "reviewer": "Reviewer_UwQs", "summary": "This proposes a novel visual localization method that utilizes 3D Gaussian Splatting. This approach replaces the point cloud in the pipeline of visual localization with a 3DGS model as the 3D representation. The camera pose of a query image is computed in a coarse-to-fine manner. The coarse pose is predicted by using a pretrained pose estimator. The pose refinement is carried out by neural rendering and image matching. The method is evaluated across indoor and outdoor datasets, outperforming state-of-the-art NeRF-based methods and traditional localization methods.", "questions": "What is the major difference compared with 3DGS SLAM methods since they can also optimize the camera poses?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea of using 3DGS for visual localization is simple yet effective. It replaces the key components in the localization pipeline with advanced alternatives. For instance, the image retrieval module is replaced with a pose estimator for a coarse pose prediction. The 2D-3D matching is performed based on a prebuilt 3DGS model instead of the point cloud. The authors take the challenges, such as the appearance gap, into account and introduce an exposure adaption module, which enhances the appearance consistencies. The proposed method achieves more accurate camera pose estimates than the existing localization approaches on several datasets. The paper is well-written and easy to follow.", "weaknesses": "-\tThe superiority of 3DGS in the framework of visual localization is a bit unclear. The scalability of 3DGS seems questionable. For instance, in a large-scale and unbounded scenario, the 3D map might be built from thousands of images in the traditional localization pipeline. The 3DGS appears to be inapplicable in this case. If this is the case, the requirement of the 3DGS model might be a limitation in real applications.\n\n-\tThe authors argue that in HLoc, the top-1 retrieved image may have limited overlap with the query. However, this is the limitation of the current image retrieval techniques rather than a shortcoming of HLoc itself. In the introduced pipeline, the coarse pose is computed using an off-the-shelf approach, which might be more powerful than image retrieval. Therefore, this approach could also benefit HLoc. For example, the reference can be selected based on the predicted camera pose. The advantage of 3DGS is not convincing enough.\n\n-\tThe evaluation metrics are inconsistent across different datasets. The authors report accuracy in some cases and pose error in others.\n\n-\tThe comparison with traditional localization methods like HLoc on Cambridge Landmarks is missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)