3D Gaussian Splatting based Scene-independent Relocalization with Unidirectional and Bidirectional Feature Fusion
Authors: Junyi Wang
Yuze Wang
Wantong Duan
Meng Wang
Yue Qi
Venue: NeurIPS 2025 poster ()
Keywords: Camera relocalization
3D Gaussian Splatting
Feature Fusion
Primary area: applications

Abstract: Visual localization is a critical component across various domains.
The recent emergence of novel scene representations, such as 3D Gaussian Splatting (3D GS), introduces new opportunities for advancing localization pipelines.
In this paper, we propose a novel 3D GS-based framework for RGB based, scene-independent camera relocalization, with three main contributions.
First, we design a two-stage pipeline with fully exploiting 3D GS.
The pipeline consists of an initial stage, which utilizes 2D-3D correspondences between image pixels and 3D Gaussians,
followed by pose refinement using the rendered image by 3D GS.
Second, we introduce a 3D GS based Relocalization Network, termed GS-RelocNet, to establish correspondences for initial camera pose estimation.
Additionally, we present a refinement network that further optimizes the camera pose.
Third, we propose a unidirectional 2D-3D feature fusion module and a bidirectional image feature fusion module, integrated into GS-RelocNet and the refinement network, respectively, to enhance feature sharing across the two stages.
Experimental results on public 7 Scenes, Cambridge Landmarks, TUM RGB-D and Bonn demonstrate state-of-the-art performance.
Furthermore, the beneficial effects of the two feature fusion modules and pose refinement are also highlighted.
In summary, we believe that the proposed framework can be a novel universal localization pipeline for further research.

3D Gaussian Splatting based Scene-independent Camera Relocalization with Unidirectional and Bidirectional Feature Fusion
Junyi Wang
Yuze Wang" vangyyuee19807090163.. com
Wantong Duan
junyiwang@sch edu.
wantongddbuaa, edu.
Meng Wang www.:ouuaaaaauu.uuaaa..eee
Yue Qi
wangm05@buaa edu. cn
qy@buaa cerobbaaaeed....om edu. cn
Abstract
Visual localization is a critical component across various domains. The recent emergence of novel scene representations, such as 3D Gaussian Splatting (3D GS), introduces new opportunities for advancing localization pipelines. In this paper, we propose novel 3D GS-based framework for RGB based, scene-independent camera relocalization, with three main contributions. First, we design a two-stage pipeline with fully exploiting 3D GS. The pipeline consists of an initial stage, which utilizes 2D-3D correspondences between image pixels and 3D Gaussians, followed by pose refinement using the rendered image by 3D GS. Second, we introduce 3D GS based Relocalization Network, termed GS- RelocNet, to establish correspondences for initial camera pose estimation. Additionally, we present a refinement network that further optimizes the camera pose. Third, propose a unidirectional 2D-3D feature fusion module and bidirectional image feature fusion module, integrated into GS-RelocNet and the refinement network, respectively, to enhance feature sharing across the two stages. Experimental results on public 7 Scenes, Cambridge Landmarks, TUM RGB-D and Bonn demonstrate state-of- the-art performance. Furthermore, the beneficial effects of the two feature fusion modules and pose refinement are also highlighted. In summary, we believe that the proposed framework can be a novel universal localization pipeline for further research.
Introduction
Visual localization is considered fundamental research problem in computer vision and is applied across a variety of scenarios, including Augmented Reality (AR), Mixed Reality (MR), robotics, and autonomous driving Wang et al. (2024c); Uia et al. (2024); Zhu et al. (2024). The primary function of a localization algorithm is to estimate the 6-DoF (Degrees Freedom) camera pose within a target environment.
As current works cover, two main types of methods are investigated achieve robust localization performance, including feature matching and geometry regression. Specially, feature matching methods employ either hand-crafted Liu et al. (2017) or learned features (SuureesSSunnr et al. (202I) to establish pixel correspondences for localization, while geometry regression approaches train the deep network
II. School of Computer Science and Technology, Shandong University, Qingdao, Shandong, China.
2. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China. Qingdao Research Institute of Beihang University, Qingdao, Shandong. China. Corresponding Author: Yue Qi.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Mesism Localization on Scames
GS-RelocNet with Unidirection Feature Fusion
Initial Camera Pesc
20
Pose Refinement with Bidirectiona: Feature Fusion
80
DFNwl
Monocular RGB
Reloc3R
3D Gaustians
Confidence Score
GROSSFIRE
Refined Camera Pose
100
CS-CPR
STDLoC
120
Combrodig Landmarks
SER
Neet
G5
D Scene- indeeendet
Figure 1: (a) Localization pipeline. The framework predicts the initial pose by establishing 2D-3D correspondences between pixels and 3D Gaussians, followed by a pose refinement process to optimize pose by rendered views using 3D GS. (b) Localization performance comparison. Our method obtains the least localization error on the two datasets, while supporting scene-independent localization.
to solve the camera pose, by Absolute Pose Regression (APR) Chen et al. (2022) or Scene Coordinate Regression (SCR) Wang Qil (203bb). However, due to the representation, these methods always ignore texture and illuminatoon information, which limits their capacity to fully represent the scene.
Recent neural and geometric 3D structures for Novel View Synthesis (NVS) have gained significant popularity for scene representatin Mildenhall et al. (202T); Kerbl et al. (2023); Hu et al. (2023). Specially, 3D Gaussian Splatting (3D GS) offers
approach to training and rendering performance for NVS, presenting new opportunities for the localization pipeline. However, how to leverage 3D GS for robust and accurate localization remains significant challenge.
Current methods predominantly utilize 3D GS for pose refinement Keetha et al. (2024); Yan et al. (2024); Liu et al. (2025), which heavily relies the accuracy of initial camera pose estimation. When the initial pose estimation is inaccurate or fails, the refinement process becomes ineffective. To address this, our motivation lies in the employment of 3D GS for both pose initialization and refinement, while achieving scene-independent relocalization to enhance robustness.
As illustrated in Fig. [(a) we introduce novel relocalization framework that first establishes 2D- correspondences between image pixels and 3D Gaussians, followed by refinement stage that predicts the relative pose between real and rendered views using 3D GS. A feature fusion module incorporated in both stages enhance correspondence regression. To the best of our knowledge, this is the first 3D GS based, scene-independent relocalization framework, offering robust solution for challenging localization tasks. Specially, the term "eene-independemn indicates that framework can achieve robust relocalization in target scene without requiring scene-specific pre-training, in contrast to most "scene-dependent" methods that necessitate prior training on the target scene. Our contributions summarized follows.
We propose a innovative framework for rene-inndeendent camera relocalization. The framework comprises initial pose estimation by establishing correspondences between pixels of the input image and the scene expressed by the 3D Gaussians, and pose refinement by predicting the relative pose between the input image and rendered view using 3D GS.
We design GS GS-RelocNet for predicting the correspondences obtain the initial camera pose. Within GS-RelocNet, we introduce a unidirectional feature fusion module to merge geometry and texture features for learning confidence scores between each pixel and 3D Gaussian.
We propose pose refinement network based on 3D GS. In the refinement network, we present a bidirectional feature fusion module to combine features from rendered and real images.
To validate the performance Landmarks. As illustrated in Fig. [(b) the results demonstrate the state-of-the-art localization perfor- mance on the two datasets. Meanwhile, our framework can support indeeednnee relocalization, denoting that it can perform relocalization in unseen scenes.
framework, we conduct experiments on Scenes and Cambridge

---

2 Related Works
2.1 Localization with Feature Matching
Traditional hand-crafted feature matching based methods typically follow pipeline consisting of feature extraction, matching, global map construction, and optimization Liu et al. (017)); Sattler et al.
(2017). Moreover, semantic SLAM systems Yang & Scherer (2019) You et al. (2023) ) Lin al. (022bb
Xi et al. (2025); Zhang et al. (2025) incorporate semantic information derived from learned features to enhance the robustness and accuracy of traditional hand-erafted feature based processes. Alternatively, learned feature matching methods aim to estimation pixel correspondences for pose estimation Wang et al. (2022) 2024d). Based on LoFTR [Sun et al. (202T). Efficient LoFTR Wang et al. (2024d) performed the transformer with aggregated attention mechanism using adaptive token selection for efficiency.
2.2 Localization with Geometry Regression
The geometry regression methods can be broadly categorized into two approaches, including APR and SCR methods. APR methods train the deep network to learn the relationship between 2D images and 6- DoF' camera poses et al. (2022 2024b). While APR methods offer high computational efficiency, they are often limited by accuracy and generalization issues Liu al. (2024b). Alternatively, SCR techniques calibrate the pose by using the Kabsch or Perspective-n-Poins (PnP) algorithm to the known source and evaluated target coordinate, which achieve considerable localization performance Wang & Qil (2021) 2023b). Recent hot SCR researches are DUSt3R [Wang et al. (2024b) and its subsequent extension works Leroy et al. 2024). By using large training samples, the methods achieve outstanding localization accuracy and generalization ability. These methods focus on geometry regression, bug can not fully exploit texture features.
2.3 Localization with Neural Radiance Field (NeRF) and 3D GS
Due to the outstanding NVS performance, NeRF is applied to the localization pipeline with iterative rendering and pose updates Germain et al. (2022) Moreau et al. (2023); Chen et al. (2024a); Wang jet al. 2023);; Xu et al. (2024). NeRFect Match Zhou et al. (2024b) explored the potential of NeRF internal features in establishing precise 2D-3D matches for localization. With the shift in the NVS field from NeRF to 3D GS. STDLoc Huang et al. (2025) introduced a matching-oriem sampling strategy and scene-specific detector to achieve efficient and robust pose estimation.
Gaussian
3 Method
3.1 Overview
Given target image and 3D scene model expressed through 3D GS, our method predicts the 6-DoF camera pose of the target image within the scene. The overall localization process is composed of two stages, both utilizing 3D GS in distinct ways. the initial pose estimation stage, we establish the correspondences between each image pixel and 3D Gaussian, followed by PnP algorithm with RANSAC to solve the initial camera pose. Based on the predicted pose, we proceed to the refinement stage, where we first render the view using 3D GS, and then predict the relative pose between the target and rendered images to perform pose optimization.
The advantages of the proposed pipeline are twofold. First, we use 3D Gaussians to represent the scene in the initial pose estimation stage. Compared the point cloud representation in SCR methods, 3D Gaussians retain more detailed geometric and texture information. Additionally, by establishing correspondences between image pixels and 3D Gaussians, GS- ESS-RelocNet enables scene-independent relocalization. Second, we employ the rendered view generated by 3D GS for pose refinement, which reduces the domain gap between the rendered and real views. Through this refinement process, the localization accuracy is further enhanced.
3

---

3.2 Initial Pose Estimation by GS-RelocNet
Pose estimation process. In initial stage, we regress the confidence matrix between image pixels and selected Gaussians by GS-RelocNet. Based the confidence threshold, establish the correspondences between pixels and Gausaians, followed by a PnP method with RANSAC to solve the initial pose.
GS-RelocNet
DINO
Frozen
Encoder:
Embredding
Confidence Matrik
P
Self Asiention
64
128
256
512
512
256
128
Target image
Uroes Siit Thack
Camera
Sonalization
Tinheddinn
Goussion
256
512
256
128
Upsample
Point Cloud
Merging
Ruz
Sdf Arteriot
Pouling
RS18
Comvii
Redage Swrn
Transform
Point cloud
Acd
Position Embedding
Feature
Blocks
Unpooling
Concatenate
XN
Convld
Aureaiic
Feature Fusion
Unidirectional
Layer Norm
Serialized Attention
Layer
Norm
Figure 2: Architecture of OS--RelcNNet. The GS-RelocNet framework integrates RGB feature
encoder, a 3D Gaussian feature encoder, an RGB descriptor decoder, a point descriptor decoder, and confidence metric regression module. In the figure, arrows indicate the process flow, and numbers adjacent to each block denote the corresponding filter size.
Inputs and outputs of GS-RelocNet. The JS-RelocNet receives a monocular RGB image and a 3D model expressed by 3D Gaussians inputs, and outputs the confidence scores that quantify the correspondence between pixels and 3D Gaussians. These confidence scores are then used to establish 2D-3D correspondences for camera pose prediction.
Input processing. For image processing, GS-RelocNet employs patch partition and embedding to segment the images into multiple parts. For the 3D Gaussians, the point cloud serialization and embedding are exploited to transform unstructured 3D Gaussians into structured format. Specifically, the position, alpha, covariance matrix and spherical harmonic function of 3D Gaussian are processed independently, with the features from all four components concatenated. Additionally, we incorporate features from DINO V2 Oquab et al. (2023), along with depth estimation head, supplementary input to enhance geometry feature learning. Notably, the parameters of the DINO V2 model are kept frozen during this process.
Architecture GS-RelocNet. The detailed architecture of GS-RelocNet presented Fig. It consists of a spatial image encoder, 3D Gaussian encoder, image descriptor decoder, a 3D Gaussian descriptor decoder, and a confidence matrix decoder. Within both encoders and decoders, we incorporate unidirectional feature fusion module to facilitate effective feature sharing from the model to the RGB image. The image branch employs a Swin Transformer Liu et al. (202T) architecture, consisting of multiple Swin Blocks, while the point cloud branch utilizes Point Transformer V3 Wu et al. (024) for 3D Gaussian feature learning.
Unidirectional feature fusion module. Between consecutive image and 3D Gaussian learning blocks, we propose unidirectional feature fusion module to combine and features, shown at the lower left of Fig. 2 The module takes image features and geometry features inputs, and outputs the fused features of the two parts. Let the input RGB feature have dimensions H, x W., x Da,. and the model feature have dimensions N x D m- The whole fusion process is as follows.

---

Step 1, alignment of features. The module aligns the image and 3D model features. Specifically, the RGB feature is reshaped to Nm. x (H, x W,, D, followed by ID convolution to transform it to N.. x Dm, aligning with the model feature dimensions.
Step fusion with self-attention. In this step, the features from the image and 31) Gaussian first added. Subsequently, multi-head self-attention is applied to added features and original model features respectively. Finally, the both features are again added achieve feature fusion.
Step 3. feature transformation. The combined feature is reshaped to H, x W x (Nm x Dm/ A Swin Transformer block is then applied to restore the feature dimensions to H. W. Du. matching the input RGB feature. The input RGB feature is added to this output to produce the merged feature.
Step 4, iterative fusion. Steps through 3 are iteratively applied. Specifically, the fused features from the current iteration are used the RGB input for the next iteration.
Through these steps, GS-RelocNet can perform one-way fusion of 2D image and 3D Gaussian features. Specifically, the model features can influence the image descriptor learning, but not the other way around. On one hand, this feature sharing enhances RGB feature descriptor learning. On the other hand, the independence of model descriptor learning allows for pre- pre-prediction of model descriptors before the inference stage, significantly accelerating the overall process. In summary, we argue that this fusion mechanism contributes to the localization task, and its effectiveness will be validated in the subsequent experiments.
Confidence matrix regression. After regressing the N; image descriptors and Ng 31) Gaussian descriptors, GS-RelocNet performs regression of confidence matrix with N; * Ng scores, where each score represents the confidence between its corresponding image and 3D Gaussian. The regression process proceeds as follows. First, for the image features, GS-RelocNet applies a positional encoding operation followed by self-attention operation. Similarly, for the 3D Gaussian features, GS-RelocNet also applies a self-attention operation. Next, cross- cooss-attention applied to the two sets of features. Finally, a dual softmax operation is employed to predict the final confidence matrix.
Loss of GS-RelocNet. To train GS-R Reloclee we need to construct the ground truth confidence between each 3D Gaussian and image pixel. Given a 3D Gaussian with 2D covariance matrix Σ under the current view, then the ground truth confidence is calculated the following formula,
1
C
(a - -1)]
2T Σ
where y denotes the 2D Gaussian center, and x is the pixel position.
4 Pose Refinement
Refinement process. After processing with GS- Reloceee,,,, we obtain the initial camera pose of the target image. Subsequently, we rendered the current view using 3D GS with the predicted camera pose. The refinement process is to predict the relative pose ( RRe TF) between the input real image and rendered view. Finally, the refined pose (R", TF) is obtained by the following formula.
- R2
TØ T-= R2 T. +
To predict R., TP, we propose refinement network to regress residual coordinate map proposed in the work Wang & Qi (203a)) between the rendered and real views.
Refinement network. As illustrated in Fig. 3 the network takes real and rendered views as inputs, and outputs the residual coordinate map, followed by a PnP method with RANSAC to predict the relative camera pose (RP, TF). The architecture of the refinement network consists of several Swin blocks to learn features of both real and rendered images.
Bidirectional feature fusion. Similar to GS- Res-Relo,eet, we design feature fusion module to combine the features of real and rendered images. The key distinction is that the feature fusion process is bidirectional, following these steps. First, the two sets of features are added together, and self- attention operation applied to fuse the real and rendered features. Second, Swin block is used to further process and learn the fused features. Finally, the resulting features of both parts are obtained by adding the original features to the fused output. Notably, the three steps outlined above can be repeated multiple times to refine the feature fusion process.
5

---

Refinement Network
Bidirectional Feature Fusion
Patch Emhedding
Ch
Maraed Image Festures
128
256
512
Roal View
Resiuaal Loordinete Map
Sel Aneraion
Embedding
C
C
Rised Rendered
Image Fastures
128
256
512
Randered
Figure 3: Architecture of the refinement network. The network takes real and rendered views as inputs, and outputs the residual coordinate map to predict the relative camera pose, which is composed of real image feature encoder, rendered image feature encoder and bidirectional fusion module.
Loss function of refinement network. For the loss of the residual coordinate (Lossmap). we use L loss to train the refinement network. In addition to the coordinate map loss, the refinement network also exploits the auxiliary loss to facilitate feature learning. Specifically, the pose loss is expressed using the following formula,
Lossguz = exp(-T)) IT - - I exp( 11 Q +
where T T represent the prediction and ground truth of camera position, Q", Q" mean the predic- tion and ground truth of camera orientation, and T, Q are variables learned by the refinement network to balance the three terms.
Although the auxiliary loss directly outputs the 6-DoF relative camera pose, not used as the final result in our framework. This decision is based on the observation that direct learning based methods typically yield less accurate localization results Wang et al. (2020). Hence the refinement network uses the map to predict the camera pose, with the auxiliary loss as additional guidance.
Loss = Lossmap + a * Lossauz.
4.1 Datasets and Implementation Details
Datasets and train-test split. We conduct experiments on indoor 7 Scenes Shotton et al. 2013).
TUM RGB-D Sturm al. (2012. Bonn Palazzolo et al. (2019). ScanNet Dai al. (201)), outdoor MegaDepth Li & Snavely (2018) and Cambridge Landmarks Kendall et al. (2015). The implemen- tation divided into scene scene-dependent and scene-independent settings. In scene- indeeeednddn setting, we train GS- RelocNet on ScanNet and test it 7 Scenes, TUM RGB-D and Bonn. For outdoor scene-independent setting, GS-RelocNet trained on MegaDepth Li Snavely (2088) and tested on Cambridge Landmarks. When performing the relocalization task on in a scene-dependent manner, GS- RelocNet is trained on 7 Scenes and Cambridge Landmarks respectively.
Gaussian selection. In training and inference stages, 4096 Gaussians are selected by spatially uniform sampling. Specifically, we partition the 3D space into S, S, x S² grids with each grid resolution of O. 1m. Let N₂ denote the total number of Gaussians. For each grid cell containing N Gaussians, we randomly sample N₂ # 4096/N, Gaussians. If the number of sampled Gaussians less than 4096, we randomly duplicate some samples to reach the desired count for training.
Ge-RelocNet details. The RGB input of framework is resized 256 256 pixels, while 4096 3D Gaussians spatially uniform sampled from all 3D Gaussians, are processed by the Point Transformer network. In Fig. 2] the module setting and output filter size are indicated near the corresponding modules. Additionally, GS-RelocNet leverages an ADAM W optimizer with learning rates 2 x 10
Refinement network training details. After obtaining the initial poses, use 3D GS to render the view. Both the real and rendered images are resized to 128 x 128, while the output size of the relative structure is 64 64. In Fig. 3 the module setting and output filter size also annotated. The loss coefficient a in Eq. 4 is set to 0.3, the variables T, Q are initially set to 0.0.
Inference details. After training GS-RelocNet, we use the selected Gaussians and the test image to obtain the confidence map. Then we initially apply a fixed threshold (set to 0.7 our experiments) to eliminate correspondences with similarity scores below this value. Subsequently, for each pixel

---

associated with multiple Gaussian correspondences, the pixel's coordinate is computed as weighted average of the selected Gaussians, weighted by their confidence values. Additionally, if the number of correspondences falls below 100, we re-run GS ESS-ReloNNet by selecting an alternative set of 4096 Gaussians from the grids that contain Gaussians with confidence higher than 0.7. After determining the pixel and its weighted Gaussian coordinate, we use PnP with RANSAC to solve the initial pose.
Based on the initial pose, we render the view by trained 3D GS. Given the input image and rendered view, we predict the residual coordinate map by the refinement network, followed by a PnP method with RANSAC solve the relative pose between them. Finally, the refined pose is obtained by Eq.
4.2 Static 7 Scenes
Table 1: Median position (cm) and rotation (9) errors on 7 Scenes. The sign F means the result with SfM pseudo ground truth, while others leverage the original KinectFusion ground truth. In each scene, the red and blue marks represent the first and second.
Method
Chess
Fire
Heads
Office
Pumplan
Kischen
Stairs
Mean
Scene-lependenl
MS. Transtommer Shavis ot
M22TUCCVV 2021)
1188.58
231115
13713.0
18/8.14
1788,42
1678.92
29/10.3
189.51
APR
DFNer Chan et on gitee (LEC 20222
ChererrCCe 2u2e (CVPR 2004)
3/1.12
2.1/1.24
2:311 39
1.8/2.03
4/.29
6/1.54
2.8/1.26
7/1.92
3.5/1 48
4.2/1.11
711.74
12/2.65
5.6/167
6/1.5
3.2/1.54
MS Hyp cieits & Keller 2025
(CVPR 2025)
1144.34
2349.9
13/10.7
1786.05
16/5.24
1776.66
27/6.00
18/7.00
(Bachhulln
2023)
0.55/011
0330.33
0.53/033
1.10022
2.890811
1,10034
SCR
(Glany crala
CVPR 2024)
2/0.78
20.74
3/082
4/100
X1.19
4/1.2
2.71.10
Gl GLACE Warn 2f al:
21124 (CVPR 2024)
8.60.18
0.00 34
0.6034
1,10029
0.900.33
0.80200
3,200.93
1.20.36
CROSSPIRE (Sbrrent of 212 (CVPR 2023)
1814
3719
32.3
37L.
3008
270.3
1271.9
4/1100
NeRF
2izz4a TECCV 2024)
0.940.30
1.10.40
1.5/1000
30.80
2:200.00
1-00.30
10,0/1,00
2.800700
PMNat[ drate puzeu L 2024)
4/1.70
104.1
7/4.23
7/196
14/3,33
1443.66
16/3.2
103.24
DFNec 1 CPKTLi all 21925
(ICLK 3025)
0.
0.90332
0660366
120032
13003
0.5022
2.20161
L.1034
GS-CPP Li 20155
2025)
0.50115
066225
0.40.25
1/00.23
1,400.42
STDLoc [uang at al. 2025 (U25))2025)
0.46m.15
0.57/0.24
0.459026
0.8600344
0.93/0.21
0.630
1,1200
0.76-0.24
Ours! (No Ketinement) Ours!
0.44/0.17
0.61/1.44
0.390030
0.89/0.34
0.95/0.28
0660022
1.3600.9
0.750126
0,41/1115
0.55M.21
0.370826
0.850034
0.9200.25
0.588.18
1.300.35
0.71A1.25
Scene-indepemmeet
Hond-crillel
Active Scarch al MUT6S (TPANI)
40
30.35
27.45
Whtel
803.00
75:37
32222
3172.46
RPR
KclocNciBahlin of 2ITK (FOCY 2018)
Relative RoggNeLaassa
2017 (ICCV 2017)
31/15,00
21770.90
40/19.00
33211 30
24/22.20
191322
38/14,10
31710.91
44/18.20
41/16.50
30
3311220
35/23.60
36.1/18.57
29. 29.
InLoc Taira et al. 2013
2013)
3/1055
31.07
2/1.16
3/1.05
5/1.55
9/2.47
5.1/1.38
Strtin orale 2001 (CVPR
2/080
2/0.73
1/0.2
3/0.82
4/L.11
3/1.20
5/1.30
2.9/0.98
SCR
Wang àt 3. [Wny & Oi 20233a (ISMAR 2023)
2,40099
2,090.9
16/1.27
2.4/1.01
3,3/1.20
2.5/1.14
3.1/1.22
2,611111
DUSER Waig orain X1ge0 (CVP 2024)
340.96
4F1.02
1/L.00
4/1.04
S/1.26
4/1.36
21/4.06
6/1.53
Relockk
20255 CCPPR 2025)
340.99
4/1.13
2/1.23
510.88
7/114
5/1.25
12/1 25
5.4/112
Ours Kelinement)
1,20.655
171.1
2.5/1.10
1.70US
Ours
1,00.72
1 0.64
0.600.0
1,40822
2.30.93
2,0/1,02
1,90992
1.40822
In Table D we provide the experimental localization results on 7 Scenes. with both scene-dependent and scene-independent categories using the median position and rotation errors.
Scene-dependent method comparison. In scene-independent setting, our results gain best accuracy on both mean position and orientation metries. Among 7 scenes, our method obtains the best position on all 7 scenes and orientation on scenes, demonstrating the state-of-the-ar performance under scene-dependent situation. Additionally, our method achieves 1.21cm/OO.. /0.61 with the original ground truth, which also demonstrates superior accuracy compared to other methods evaluated.
Scene-independed method comparison. With scene-independent approaches, our method achieves the lowest mean position and orientation errors on all scenes. Moreover, our method significantly re- duces the position error from other best 2. 2.6cm /0.98 to 1. -4cmm/0.82 46.2% 1..2% 16.%%. Besides the localization accuracy, to our knowledge, our framework is the first scene-independent relocalization method with 3D GS.
Result visualization. Fig. 4 provides visualization results of the camera trajectories with green poses denoting the ground truth and blue ones representing our results. The results show minimal differences between the predicted and ground truth poses, demonstrating the suitability of our method.
4.3 Dynamic TUM RGB-D and Bonn
Challenges on TUM RGB-D and Bonn. The TUM RGB-D test sequences involve two individuals walking around a table increase the complexity of localization. Similarly, the Bonn dataset features highly dynamic sequences, such as individuals manipulating boxes or interacting with balloons.
7

---

Office Seq.02
Kitchen Seq.06
Stairs Seq.01
Figure 4:
The visualization results of camera pose 7 Scenes. In each scene, the green and blue poses denote the ground truth and prediction respectively.
Table 2:
RMSE of ATE [cm] results in four dynamic scenes of TUM RGB-D. In each scene, the red and blue marks represent the first and second respectively.
It3 walwaiing......
xyz
myy
crafed
ORB-S SLAM Mar-Artal & Tardós 2017
45.9
9.3
05.8
32.8
38.5
VINS Song
DynaSLAM Bescus et 20188
1.5
0.6
3.5
2.5
2.0
Hand-crafied Semanties
DS-SLAA
2.5
44.4
3.0
12.7
LC-CRF DU 2020
1.6
1.1
2.8
2.5
3D
Onrs
1.1
0.4
2.2
2.0
1.4
Root Mean Square Error (RMSE) of Absolute Trajectory Error (ATE) on dynamic TUM RGB-
D. Table 2 presents the RMSE of ATE for four dynamic scenes, compared against DRB-SLAM2. DynaSLAM, DS-SLAM, LC-CRF SLAM, and DGM-VINS. Our approach consistently outperforms these state-of-the-art SLAM systems in the RMSE of ATE metric, demonstrating superior localization accuracy in dynamic environments.
RMSE ATE on dynamic Bonn. evaluated our framework on the Bonn dataset across 20 test scenes, consistent with LC-CRF SLAM, and compared it with ReFusion Palazzolo al. (2019), MaskFusion Runz et al. (2018), and LC-CRF SLAM Du et al. (2020). The mean RMSE of ATE results are 23. 8cm (ReFusion), 25.1em (MaskFusion), 6.8cm (LC-CRF SLAM), and 4.3cm (Ours). Our framework achieves the lowest mean RMSE of ATE, highlighting its exceptional localization accuracy in highly dynamic scenes.
4.4 Cambridge Landmarks
Table 3: Median localization results Cambridge Landmarks compared with other methods. Units of position and orientation are centimeter (cm) and means the result with scene-independent setting. In each scene, the red and blue marks represent the first and second.
Method
College
Hospital
Shop
StMary
Mcan
Trnnffeneer Shavit 2021 (ICCV 2021)
85/1.45
175/2.43
88/3.20
166/4.12
APR
DhNee el al. 2022 (ECC
20222)
73/2.37
200/2.98
67/2.21
137/4.02
119/2,90
Iarra al. 2010 [CVP
2018)
3170.73
DSACY Rother 2001
(TPAMI)
15/0.3
21/0.4
5/0.3
13/0.4
14/0.35
ACFBBrachmann
20243
2023)
29/0.38
DUSERR2244 Wang of 20244
(CVPR 2024)
20/0.32
26/0.46
9/0.38
11/0.38
17/0.39
Reloc3R. 224 Exong
T 2025)
8770.6
1800,53
41/073
NeuMap Tanget 2025 (CVPR 2023)
14/022
19/0.4
6/0.3
17/0.5
14X1.35
CROSSFIKE Crral (2023) (ICCV 2023)
47/0.7
43/0.7
20/12
29/14
37/1.00
NeRt
NeRFMaich MarMMachh/oooug Zhou
24/240 (ECCV 2024)
12.7/0.2
20.7/0.4
8.7004
11.3/0.4
0.13/3
PMNeeLLi draii 20240 (ECCV 2024)
68/1.97
103/1.31
58/2.10
133/3.73
91/2.28
DFNer + GS- PR Lu (2025) (ICLR 2025)
26/0.34
48/0.72
10/0.66
2770.62
28/0.51
ACE + GS. [P 2023 (ICLR 2025)
25/0.29
26/0.38
5X0.23
13/0.41
17/33
Gaussian
STDLoc Hunng
TCVP 2025)
1570.17
11.9N021
3/0.13
4.7/0.14
9N0.16
Ours (No Kelmement
11/0.19
13/0
4/0.18
7/0.15
9/0.20
Ours
12/0.18
13/0.25
5/0.19
7/0.20
9/0.21
Ours
9/0.15
10/0.19
3/0.15
5/0.13
7A0.16
In Table 3 we provide the experimental localization results on Cambridge Landmarks in comparison with APR, SCR, NeRF and 3D GS methods. The results on Cambridge Landmarks demonstrate that method gains the best performance on both mean position and orientation metrics. Compared

---

with the recent PMNet Lin et al. (2024)) the position error is significantly reduced from 91cm to 7cm, further validating the effectiveness of our framework.
Generalization on Cambridge Landmarks. To assess generalization, we trained GS-ReloeNet on the MegaDepth dataset [Li & Snavely (2018) and tested it the Cambridge Landmarks dataset in a Secee-independent setting (marked as !). The mean pose error across four scenes is 9cm/0.21 surpassing the performance of DUS13 (17cm/0.399) and Reloc3R (48cm/0.58"), These results underscore the robustness and generalization capability of GS-RelocNet in diverse, unseen environ- ments.
Discussion of pose refinement. In Tables and 3 we also present the localization performance without pose refinement. Two notable observations emerge from the results. First, pose refinement leads to improvements in localization accuracy across all scenes, demonstrating the positive impact of the refinement process using 3D Gaussians. Second, even without pose refinement, method remains competitive with other state-of-the-art approaches. On 7 Scenes, the scene-dependent result comparable with STDLoc, while the inneeinndeeeddent performance is obviously more accurate than others. On Cambridge, the results are also comparable to those of STDLoc.
Discussion of running time. Our framework efficiency comprises initial pose estimation with confidence map regression and PnP, and pose refinement with view rendering, residual map regression and PnP. On average, it processes testing images at 65 ms (15.4 FPS) Scenes and Cambridge Landmarks. The average running times per frame are as follows, 39 ms for confidence regression, 4 for initial PnP with RANSAC, 9 ms for view rendering, 8 ms for residual coordinate regression, and ms for PnP in pose refinement. This outperforms 3D GS based methods like ACE Brachmann al. (2023) GS-CPR Liu et al. (2025) (190ms, 5.3 FPS) and STDLoc Huang ct al. (2025) (143ms, 7 FPS), highlighting our computational efficiency.
Nvidia 4090 GPU across
Discussion of comparison with 3G GS based methods. From Tables 1 and 3 we can that localization performance of 3D GS based STDLoc and ACE + GS-CPR slightly less accurate than ours, but also is competitive. Compared to these methods, the additional superiorities of our framework lies in two aspects. First, our method supports scene- -edeeidndeendent relocalization, while STDLoc and GS-CPR requests training before localization in the target scene. Second, our inference speed is faster than the other two methods, achieving more than twice speed.
4.5
Detailed Studies
Table 4: Scene-independent localization results on 7 Scenes with original ground truth and Cambridge I Landmarks with different settings.
Fusion encoder
Fusion decoder
Fusion Refinement
Scenes
Cambridge Landmarks
ST
2.1/1.17
1470.35
$2
1.9/1.14
13/0.32
53
1.911111
12/0.36
$4
X
2.0/1.03
13/0.35
V
1.67.97
11/0.28
S6
1.70..22
10/0.26
1.500.4
9/0.23
S8
14/0,82
9/0.21
Irerations of fusion model in GS-RelocNet
1.70.92
12/0.29
S10
1.40.82
9/0,21
S11
14/0.86
10/0.22
812
1.5/0.87
11/0.24
Discussion of outstanding performance. The results of the aforementioned experiments demonstrate state-of-the-art performance of our framework. We attribute our outstanding performance to three main factors. First, our framework leverages the full potential of 3D GS for both initial pose estimation and pose refinement. Second, GS-RelocNet is specifically designed to establish accurate correspondences between pixels and 3D Gaussians. Third, we propose a refinement network that predicts the relative pose between real and rendered images using bidirectional feature fusion module. In the following sections, we present ablation experiments to demonstrate the effectiveness of these fusion modules.
S8 of Table we conduct ablation experiments of the three fusion modules, including the unidirectional feature fusion module in encoder, decoder of
A Ablation studies of fusion modules.

---

GS- US--ReloNeet and the bidirectional fusion module in the refinement network. In S5, S6, and S7, each module is removed individually. With this setting, the two features are reshaped and concatenated directly. It is evident that accuracy across all three datasets decreases in comparison to setting S8. In S2. S3, and S4, two modules are removed individually, and the results are less accurate than those in settings S5, S6, and S7. In setting $1, where all three modules are removed, the accuracy decreases most significantly, further confirming the positive impact of the three fusion modules.
Detailed studies of the iterations in the unidirectional feature fusion module. In S9 - S12 of Table 4 we explore the effect of the number of iterations in the unidirectional feature fusion module by setting the iteration values to 1, 2, 4, and 8, respectively. When the iteration is set to 1, the result demonstrates that the estimation performance is less accurate than the other settings. Moreover, the results for iterations set to 2, 4, and 8 are comparable. This discrepancy can be explained by the fact that with only one iteration, the RGB and point cloud features are not sufficiently fused, resulting in less accurate estimates. In contrast, with 2, 4, or 8 iterations, the fusion between the two feature types sufficiently accomplished for RGB based pose estimation, leading to comparable performance.
Limitation discussion. A primary limitation of our framework is its reliance on a high-quality 3D GS model of the target scene. When the 3D GS model is of suboptimal quality, localization performance may degrade, leading to failures or significant errors.
4.6 AR Application
Rendering a walking person and a bottle
Rendering a static person and two bollles
Figure 5: AR effect on the Chess and Office scene of 7 Scenes. We render a virtual walking person and a bottle onto the Chess scene, and two virtual bottles and a static person onto the Office scene based on the predicted camera pose of our framework.
To demonstrate the performance in real-world AR applications, we present the virtual-real fusion results for two scenes using the predicted pose by our framework, the Chess and Office scenes from render virtual walking person and bottle
the 7 Scenes dataset, shown in Fig. [ Specifically, onto the Chess scene, and two virtual bottles and a static person onto the Office scene.
Conclusion
In this paper, we propose a novel 3D Gaussian based camera relocalization framework, composed of of two stages, an initial pose estimation stage, which predicts 2D-3D correspondences between image pixels and 3D Gaussians, and a refinement stage, which estimates the residual pose between the target and rendered views. To estimate the 2D-3D correspondences, we introduce descriptor matching network called GS-RelocNet. Within GS- GS-RelocNet, we design a unidirectional feature fusion model to combine RGB features with Gaussian features. After obtaining the initial camera pose, we proceed with the pose refinement network. In this refinement network, we propose a bidirectional feature fusion model to merge the features from the real and rendered images. To validate the performance of our framework, we conduct experiments on both indoor Scenes, TUM RGB-D, Bonn and outdoor Cambridge Landmarks datasets. The results demonstrate sfateoofthh-aa localization accuracy both datasets. Additionally, we provide detailed studies on the feature fusion modules and the refinement stage, further highlighting the effectiveness of our approach.
In summary, this paper makes a significant contribution by introducing a scene-independent localiza- tion framework through the full utilization of 3D Gaussians. By leveraging 3D Gaussians in both the initial and refinement stages, our method is able to deliver more accurate localization results across a variety of scenes. We hope that the proposed framework could be universal localization pipeline.
10

---

Acknowledgments
The paper supported by Shandong Provincial Natural Science Foundation (No. ZR2024QF215), Key Research and Development Program of Rizhao (No. 20242DYF0100533) National Natural Science Foundation China (No. 62072020) and the Open Project Program State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (No. VRLAB2024A**)
The authors thank the Zhiyang Innovation Technology Co., Ltd. for computing power and data support.
References
Balntas, V., Li, S., and Prisacariu, Relocnet: Continuous metric learning relocalisation using neural nets. In European Conference on Computer Vision, pp. 751-767, 2018.
Bescos, B., Fácil, J. M.. Civera, J., and Neira, J. Dynaslam: Tracking, mapping, and inpainting dynamic scenes. IEEE Robotics and Automation Letters, 3(4):4076-40833 2018.
Brachmann, E. and Rother, C. Learning less is more-6d camera localization via 3d surface regression. In JEEE Conference on Computer Vision and Pottern Recognition, pp. 4654-4662, 2018.
Brachmann, F. and Rother, C. Visual camera re-localization from rgb and rgh-d images using dsac. IEFE Transactions Pattern Analysis and Machine Intelligence, 44(9):5847-5855, 2021.
Brachmann, E., Cavallari, T., and Prisacariu, V. Accelerated coordinate encoding: Learning to relocalize in minutes using rgb and poses. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 5044-5053, 2023.
Chen, S., Li, X., Wang, 7., and Prisacariu, V. A. Dfnet: Enhance absolute pose regression with direct feature matching. In European Conference Computer Vision, pp. 1-17. Springer, 2022.
Chen, S., Bhalgat, Y., Li, X., Bian, J.,W., Li, K., Wang, Z., and Prisacariu, V. A. Neural refinement for absolute pose regression with feature synthesis. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20987-20996, 2024a.
Chen, S., Cavallari, T., Prisacariu, A., and Brachmann, Map-relative pose regression for visual re- localization. In IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 20665-20674, 2024b.
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., and Nießner, M. Scannet: Richly- annottte 3d reconstructions of indoor scenes. In IEEE Conference Computer Vision and Pattern Recognition, pp. 5828-5839, 2017.
Dong, S., Wang, S., Liu, S., Cai, I.., Fan, Q., Kannala, J., and Yang, Y. Relac3r: Large-scale training of relative camera pose regression for generalizahle, fast, and accurate visual localization. In IEEE Computer Vision and Pattern Recognition Conference, pp. 16739-16752, 2025.
Du, Z.-., Huang, S-S.,, Mu, T.-J., Zhao, Q. Martin, R. R., and Xu, K. Accurate dynamic slam using erf- bra-b long-term consistency, IEEE Transactions on Visualization and Computer Graphics, 28(4):1745-175) 2020.
Ferens, and Keller, Y. Hyperpose: Hypernetwor--infussed camera pose localization and extended cambridge landmarks dataset. arXiv preprint 20X:v:2303.02610, 2025.
Germain, H., DeTone, D., Pascoe, G., Schmidt, T., Novotny, D., Newcombe, R., Sweeney, C., Szeliski, R., and Balntas, V. Feature query networks: Neural surface description for camera pose refinement, In /EEE/CVF Conference Computer Vision and Pottern Recognition, pp. 5071-5081. 2022.
Giang, K. T., Song. S., and Jo. S. Learning produce semi-dense correspondences for visual localization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19468-19478, 2024.
Hu, J., Mao, Bao, H., Zhang, G., and Cui, Cp-slam: Collaborative neural point-based slam system. Advances Neural Information Processing Systems, 36:39429-39444 2023.
Huang. Z., Yu, H., Shentu, Y. Yuan, J., and Zhang. G. From sparse to dense: Camera relocalization with scene-specific detector from feature gaussian splatting, In IEEE Computer Vision and Pattern Recognition Conference, pp. 77059-27060 2025.

---

Jia, P.. Liu, Y., X.. Zhao, X., Wang, Y., Du, Y., Han, X., Wei, X.. Wang. S., and Yin, D. G3: effective and adaptive framework for worldwide geolocalization using large multi-modallity models. Advances Neural 2024.
Information Processing Systems,
Keetha, N., Karhade, J., Jatavallabhula, K. M., Yang, G., Scherer, S., Ramanan, D., and Luiten, Splatam: Splat track & map gaussians for dense rgh-d slam. /EEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21357-21366, 2024.
Kendall, A. and Cipolla, R. Geometric loss functions for camera pose regression with deep learning. In IEEE Conference on Compuder Vision and Pottern Recognition, pp. 5974-5983, 2017.
Kendall, A.. Grimes, M., and Cipolla, R. Posenet: A convolutional network for real-time 6-dof camera relocalization. In IEEE International Conference Computer Vision, pp. 2938 2946, 2015.
Kerbl, B., Kopanas, G., Leimkühler, T., and Drettakis, G. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions Graphicx, 42(()) 2023.
Kulhanek, J.. Peng. S., Kukelova, Z., Pollefeys, M., and Sattler, T. Wildgaussians: 3d gaussian splatting in the wild, arXiv preprint arXiv:2407.08447, 2024.
Laskar, Z., Melekhov, Kalia, S., and Kannala, Camera relocalization by computing pairwise relative poses using convolutional neural network. IEEE International Conference on Computer Vision, pp. 929-938, 2017.
Leroy. V., Cabon, Y., and Revaud, J. Grounding image matching in 3d with mast3r. In European Conference on Computer Vision, pp. 71-91, 2024.
Li, Z. and Snavely, N. Megadepth: Learning single-view depth prediction from internet photos. In IEEE Conference on Computer Vision and Pottern Recognition, pp. 10400, 2018
Lin, Gu, J., Wu, B., Fan, L.,, Chen, R., Liu, L.,, and Ye, J. Learning neural volumetric pose features for camera localization. In European Conference Computer Vision, pp. 198-214. Springer, 2024a.
Lin, X., Ruan, J., Yang. Y., He, L., Guan, Y., and Zhang, H. Robust data association against detection deficiency semantic slam, IEEE Transactions on Automation Science and Engineering, 21(1):868 880, 2024b.
Liu, C., Chen, S., Bhalgat, Y., Hu, S., Cheng, M., Wang, Z., Prisacariu, V. A., and Braud, T. Gsloc: Efficient camera pose refinement via 3d gaussian splatting. arXiv preprint arXiv:2408. 1085, 2024a.
Liu, C., Chen, S., Zhao, Y., Huang, H., Prisacariu, V., and Braud, T. Hr-apr: Apr-agnostic framework with uncertainty estimation and hierarchical refinement for camera relocalisation. arXiv preprint arXiv:2402.14371, 2024b.
Liu, C., Chen, S., Bhalgat, Y. S., Hu, S., Cheng, M., Wang, Z., Prisacariu, V. A., and Braud, T. Gs-epr: Efficient camera pose refinement via 3d gaussian splatting. In International Conference on Learning Representations, 2025.
Liu, L., Li, H., and Dai, Y. Efficient global 2d-3d matching for camera localization in a large-scale 3d map. In IEEE International Conference on Computer Vision, pp. 2372-2381. 2017.
Liu, Z. Lin, Y., Cao, Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In /EEE/CVF International Conference on Computer Vision, pp- 20122-10222 2021.
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, T., Ramamoorthi, R., and Ng, R. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106 2021.
Moreau, A., Piasco, N., Bemnehar, M., Tsishkou, D., Stanciulescu, B., and de La Fortelle, A. Crossfire: Camera relocalization self supervised features from implicit representation. In IEEE International Conference on Computer Vision, pp. 252 262, 2023.
Mur-Artal, and Tardós, D. Orb-slam2: An open-source slam system for monocular, stereo, and rgh-d cameras. IEEE Transactions on Robotics, 33(5):1255-1262 2017.
Oquab, M., Darcet, T., Moutakanni, T., Vo, V. Szafraniec, M., Khalidov, V., Fernandez. P., Haziza, D., Massa, F. El-Nouby, A., Howes, R., Huang, P., Xu, H., Sharma, V., Li, S. Galuba, W., Rabbat, M., Assran, M., Ballas, N., Synnaeve, G., Misra, 1., Jegou, H.. Mairal, J., Labatut, P., Joulin, A., and Bojanowski, P. Dinov2: Learning robust visual features without supervision, 2023.
12

---

Palazzolo, E., Behley, J., Lottes, P., Giguere, P., and Stachniss, C. Refusion: 3d reconstruction in dynamic environments for rgb-d cameras exploiting residuals. In IEEE/RSI International Conference on Intelligent Robots and Syxtems. pp. 7855-7862, 2019.
Runz, M., Buffier, M., and Agapito, I.. Maskfusion: Real-time recognition, tracking and reconstruction of multiple moving objects. In IEEE International Symposium on Mixed and Augmented Reality, pp. 10-20. IEEE, 2018.
Sarlin, P.-E.. Unagar, A., Larsson, M., Germain, H., Toft, C., Larsson, V., Pollefeys, M.. Lepetit, V., Ham- marstrand, L., Kahl, F., et al. Back to the feature: Learning robust camera localization from pixels to pose, In IFEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3247-3257, 2021.
Sattler, T., Leibe, B., and Kobbelt, L. Efficient & effective prioritized matching for large-scale image-based localization, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(9):1744 1756, 2016.
Sattler, T., Leibe, B., and Kobbelt, L. Efficient & effective prioritized matching for large-scale mmage-based localization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(9):1744-1))) 2017.
Shavit, Y., Ferens, R., and Keller, Y. Learning multi-scene absolute pose regression with transformers. In IEEE/CVF International Conference on Computer Vision. pp.
2021.
Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., and Fitzgibbon, A. Scene coordinate regression forests for camera relocalization in rgh-d images. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2930-2937, 2013.
Song. B., Yuan, X., Ying, Z., Yang. B., Song, Y., and Zhou, F. Dgm-vins: Visual-inerte slam for complex dynamic environments with joint geometry feature extraction and multiple object tracking. IEEE Transactions on Instrumentation and Measurement, 72:1-11. 2023.
Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. A benchmark for the evaluation of rgb-d slam systems. In JEEE/RSJ International Conference Intelligent Robots and Syxtems, pp. 573-580, 2012.
Sun, J., Shen, Z., Wang, Y., Bao, H., and Zhou, X. Loftr: Detector-free local feature matching with transformers.
In IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 8922-8931, 2021.
Taira, H., Okutomi, M., Sattler, T., Cimpoi, M., Pollefeys, M., Sivic, J., Pajdla, T., and Torii, A. Inloc: Indoor visual localization with dense matching and view synthesis. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 7 7209, 2018.
Tang, S., Tang, S., Tagliasacchi, A., Tan, P., and Furukawa, Y. Neumap: Neural coordinate mapping by auto- transdecoder for camera localization. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 929-939, 2023.
Valentin, J., Dai, A., Nießner, M., Kohli, P., Torr, P., Izadi, S., and Keskin, C. Learning to navigate the energy landscape. In International Conference 3D Vision, pp. 323-332, 2016.
Wang. B., Chen, C., Lu, C. X., Zhao, P., Trigoni, N., and Markham, A. Atloc: Attention guided camera localization. In AAAI Conference on Artificial Intelligence. volume 34, pp. 1899311401, 2020.
Wang. F., Jiang, X., Galliani, S., Vogel, C., and Pollefeys, M. Glace: Global local accelerated coordinate encoding. In IEEF/CVF Conference on Computer Vision and Puttern Recognition, pp. 21562-21571, 2024a.
Wang, J. and Qi, Y. Camera relocalization using deep point cloud generation and hand-crafted feature refinement. In IEEE International Conference on Robotics and Automation, pp. 5891-5897, 2021.
Wang. J. and Qi, Y. Scene-independent localization by learning residual coordinate map with cascaded localizers. In IEEE International Symposium on Mixed and Augmented Reality. pp. 79-88. 2023a.
Wang, J. and Qi. Y. Simultancous -nneeenndeenndntt camera localization and category-level object pose estimation via multi-level feature fusion. In IEEE Conference Virtual Reality and 3D User Interfaces, pp- 254-264. IEEE, 2023b.
Wang, Q. Zhang, J.. Yang, K., Peng, K., and Stiefelhagen, R. Matchformer: Interleaving attention in transformers for feature matching. In Asian Conference on Computer Vision, pp. 2746 22662, 2022.
Wang, S., Leroy, V., Cabon, Y., Chidlovskii, B., and Revaud, J. Dust3r: Geometric 3d vision made casy. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20697-20709, 2024b.
Wang, T., Sheng, H.. Chen, R. Yang. D., Cui, Z., Wang. S., Cong, R. and Zhao, M. Light field depth estimation: A comprehensive survey from principles future. Confidenne Computing, 4(1):100187, 2024c.
13

---

Wang. Y. Yan, Y., Shi, D.. Zhu, W., Xia, J.. Jeff. T., Jin, S., Gao, K., Li, and Yang, X. Nerf-ibvs: visual servo based nerf for visual localization and navigation. Advances in Neural Information Processing Systems, 36: 8292-8304, 2023.
Wang. Y.. He. X., Peng, S.. Tan, D., and Zhou, X. Efficient loftr: Semi-dense local feature matching with sparse-like speed. arXiv preprint arXiv:2403.0476. 2024d.
Wu, X., Jiang, L.,, Wang, P.-.., Liu, Z.., Liu, X., Qiao, Y., Ouyang, W., He. T., and Zhao, H. Point transformer Simpler faster stronger. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4840 480004881,22000 2024.
Xi, J., Zhang, W. Xu, Zhu, S.. Tang, L., Zhao. Three-dimensional dynamic gesture recognition method based on convolutional neural network. Hrigh-Confidence Computing, 5(1):100280, 2025.
Xu, Y., Jiang, H., Xiao, Z., Feng. J., Zhang. L. Dg-slam: Robust dynamic gaussian splatting slam with hybrid pose optimization. Advances in Neural Information Processing Syxtems, 37:51577-
2024.
Yan, C., Qu, D., D., Zhao, B., Wang, Z. Wang, D., and X. Gs-slam: Dense visual slam with gaussian splatting. In EEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 1959S-19604, 2024.
Yang. S. and Scherer, Cubeslam: Monocular 3-d object slam. IEEE Transactions on Robotics, 35(4):925 938, 2019.
You, M., Luo, C., Zhou, H., and Zhu, Dynamic dense crf inference for video segmentation and semantic slam. Pattern Recognition, 133:109023, 2023.
C., Liu, Z., Liu, X.-J,, Xie, Yang, Y.. Wei, Q., Fei, Q. Ds-slam: semantic visual slam towards dynamic environments. In IEEE/RSJ International Conference on Intelligent Robots and Syxtems, pp. 1168-1176 1744, 2018.
Zhang, D., Wang, C., Wang, W., Li, P., Qin, M., and Wang, H. Gaussian in the wild: 3d gaussian splatting for unconstrained image collections. In European Conference Computer Vision, pp. 341-359, 2024.
Zhang, J., Peng. W., Xiao, A., Liu, T., Fu, J., Chen, J., and Yan, Z. Kans-detr: Enhancing detection transformer Computing, pp. 100336, 2025.
with kolmogorov-arnold networks for small object. "ligh-Confidee
Zhou, L., Luo, Z., Shen, T., Zhang, J., Zhen, M., Yao, Y., Fang, T., and Quan, L. Kfnet: Learning temporal camera relocalization using kalman filtering. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 4919-4928. 2020.
Zhou, Q. Maximov, M., Litany, O., and Leal-Taixé, L. The nerfect match: Exploring nerf features for visual localization. In European Conference Computer Vision, pp. 108 127, 2024a.
Zhou, Q. Maximov, M.. Litany, O., and Leal-Taixé, L. The nerfect match: Exploring nerf features for visual localization. In European Conference Computer Vision, pp. 108-127. Springer, 2024b.
7hu, J., Yan, S., Wang, I.., Zhang, S., Liu, Y., and Zhang, M. Lod-loc: Aerial visual localization using lod 3d map with neural wireframe alignment. Advances in Neural Information Processing Systems, 37:119063-119999 2024.
14

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: [Yes]
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: [Yes]
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: [Yes]
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: [Yes]
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppp.ccppulli//uuuii wwwiicccguie//eeeeeeeeeis CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: [Yes]
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: [Yes]
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
17

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: [Yes]
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httpp::////uriss...// http//uurrr.cccuuu/////e httpp//ssccppp..ttttttttttteeeea
Answer: [Yes]
Justification: [Yes]
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
18

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: [Yes]
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee whw.ree.wthhhe....c.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
19

---

Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
20

---

Justification: [NA]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy htteas////eerrpppoccc///o/rreers for what should or should not be described.
21

---

A Technical Appendices and Supplementary Material
A.1 Residual Coordinate Map for Pose Refinement
Given a pair of images, the residual coordinate map denotes the XYZ coordinate difference between the current and previous camera coordinate space Wang & Qi (2023a), which is used predict the relative pose. In our paper, given the real input image and rendered view with initially predicted pose R. I." we use the residual coordinate map solve the relative pose Re; 7 between them. Then the refined pose is obtained by Eq. 2
The construction process of residual coordinate map is follows. We substitute the depth information (Z-axis value) with the grayscale value Ma denotes the XYZ coordinates under the camera space of the input real view, obtained by uniformly sampling from the grayscale image. For each point P E Ma, we first transform it to the world space. Then, it is converted to the camera space of the rendered frame. Finally, the relative point is obtained by subtracting the original coordinate from the transformed one. In summary, the coordinate representation is defined by the following formula.
M, (RC E) M2 + TH
(5)
where E denotes the identity matrix. Through regressing the coordinate map by the refinement network, R2, T can be predicted by the PnP method with RANSAC.
A.2 More Dataset Details
7 Scenes Shotton (Seeton et al. 2013). This dataset includes seven indoor scenes, each containing 2 to 10 sequences. It provides depth images, color frames, and ground truth poses. scenes were recorded using a handheld Kinect RGB-D camera at resolution of 640 480 and are divided into separate training and testing sets. The ground truth poses were obtained using the KinectFusion system. This dataset has recently been used as benchmark in studies Kendall & Cipolla y0 [Brachmann & Rother 2018); Zhou et al. (2020), making convenient for comparison with other methods.
Cambridge Landmarks Kendall et al. (2015). This outdoor dataset includes several large outdoor environments. this paper, we use five scenes (College, Hospital. Shop, and Church) evaluate localization accuracy. Ground truth poses in each scene were calibrated using Visual SFM, and sparse point cloud is also provided.
ScanNet Dai et al, (017). ScanNet contains over 1,500 scans, amounting to around 2.5 million views. The dataset was captured using user-friendly and scalable RGB-D capture system. To evaluate scene-independent performance, we train GS-RelocNet on ScanNet and tested it on the 7 Scenes dataset.
TUM RGB-D Sturm et al. (012). The TUM RGB-D dataset is designed to benchmark visual odometry and SLAM systems. We select the four dynamic scenes (fr walking xyz. fr walking static, fr_ walking rpy, fr walking half) from TUM RGB-D to evaluate the localization performance in dynamic environments.
Bonn Palazzolo et al. (2019). The Bonn dataset is tailored for dynamic localization, featuring highly dynamic sequences. We select 20 sequences from the dynamic subset (same LC-CRF SLAM Du ct al. (2020)) where individuals perform various tasks such as manipulating boxes or interacting with balloons, along with 2 static sequences.
12 Scenes Valentin al. (016). The 12 Scenes dataset features 12 larger indoor environments, with volumes ranging from 14m to 79m3.
A.3 More Implementation Details
3D GS training details. To construct the 3D GS model, we first utilize COLMAP to generate an initial point cloud using ground truth poses. Subsequently, we employ the original 3D GS model with its default configuration settings.
-iterations: 30000. Total number of training iterations.
-position_ _r_jnit:r 0.00016. The initial learning rate of the Gaussian position. -position Ir final: 0.0000016. The final learning rate of the Gaussian position.
22

---

-position Ir delay _ulltymutttt/ 0.01. The delay multiplier before the learning rate decay begins. Secistion_i__mx_stteee 30000. The total number of steps for the learning rate decay.
feature_lrr 0.0025. Learning rate of spherical harmonic function coefficients.
-opacity_ir: 0.05, Learning rate of opacity.
-scaling_Irt Ir: 0.005, Scaling learning rate.
-rotation 0.001. Learning rate of rotation.
densify_from iter: 500. Densification begins from which iteration.
-densffy_until_iiee 15000. Densification ends at which iteration.
densiiiication_interval 100. Perform densification and pruning checks every few iterations.
oppacity_prune__trrshhooo 0.005. Opacity pruning threshold.
density Seonsify_gra__thresiee thresshohdd 0.0002. The gradient threshold for densifying the Gaussian sphere.
PnP details. The PnP with RANSAC uses OpenCV implementation with following parameters.
-iterationsouut: 100. The number of RANSAC iterations.
-reprojectionErro:: 8. Threshold for reprojection error.
-confidence: 0.99. Degree of confidence.
SOLg::OOLVENN ITERATIVE. PnP solver algorithm.
Pose extension in the refinement network. In the refinement network training, small pose differences between rendered and real frames require high coordinate accuracy, which increases the learning difficulty. To address this, we extend the relative pose using fixed coefficients. The position scaled directly, and the orientation is expanded through a transformation between the quaternion and Euler angles. Specifically, the extension coefficient is set to 8.0.
Initial pose estimation by PnP. For the PnP solver with RANSAC, we adapt the traditional RANSAC framework by incorporating our predicted confidence scores. Conventionally, RANSAC determines the final result based on the number of inlier points. In our modified approach, we instead use the sum of the confidence values of the inlier points to make this determination, thereby improving the reliability of the pose estimation.
A.4 More Results on Scenes
Table 5: The percentage localization error under 5cm, 5 and 2cm, 2 on indoor 7 Scenes compared with other methods. Specially, the sign means the result with SfM pseudo ground truth, while others leverage the original KinectFusion ground truth. The red and blue marks represent the first and second.
Method
5cm, 3" (f)
2em, 2" (f)
Cheet Chen et al.
2022
43.1
8,4
APR
Marepo Chen el al.
2024b
84.0
33.7
DSAC Brachmann & Rother
2021
97.8
80.7
SCR
ACE# Brachmann
2023
97.1
83.3
GLACE Wang al.
Wang
2024a
95.6
82.2
NeReS Chen 2024a
78.3
45.9
NeRF
HR-APR[LIu
20221
76.4
40.2
NeRFMaich" Zhou el al. (2024b
78.4
DFNet + GS-CPR Liu al. |202
(Accepted by ICLR 2025)
94.2
76.5
ACE GS-CPRi Liu et al.
2025
(Accepted by ICLR 2025)
100.0
93.1
Gaussian
STDLoc Huang et al.
2025
(Accepted hy CVPR 2025)
99.1
90.9
DFNet GS-CPRT Liu al. 2024a
94.2
76.5
Ours
99.8
94.9
Scm, 5" and 2cm, 2 metric on 7 Seenes. Besides the accuracy, the localization stability is also an
important metric, always expressed by the percentage of position and orientation error under 5em, 5 and 2em, 2". Table preseens comparison of our results with those of DFNet Chen et al. (2022).
23

---

Marepo Chen et al. (2024b). DSAC* Brachmann & Rother 2021). ACE Brachmann et al. (202)).
GLACE Wang al. (2024a), NeReS Chen al. (2024a). HR-APR Liu al. (2024b). NeRFMatch
7hou et al. (2024b) and GSLoc Liu et al. (222aa)
Our approach achieves accuracy of 99.8%, slightly below the state-of-the-ar performance of ACE Brachmann et al. (2023) GS-CPR Liu et al. (2025) (100%). However, our method outperforms all other competing approaches. For the more stringent 2cm, 2 metric, our framework demonstrates least a 1.7% improvement in accuracy compared to the next-best method, underscoring its robustness in challenging indoor environments. In comparison to DFNet Chen al. (2022) GS-CPR Liu et al. (2025), our method consistently achieves higher accuracy across both metrics. Notably, while GS-CPR relies on accurate initial pose estimates, our approach excels independently, demonstrating superior generalization without requiring such priors.
A.5 Results on 12 Scenes
Table 6: The percentage of localization error under 2cm, 2 12 Scenes compared with other methods. The red and blue marks represent the first and second. The results are reported in Liu et al. 2024a).
Method
2cm, 2 (1)
APR
Marepo Chen al.
2024b
50.4
DSAC Brachmann & Rother 2021
96.7
SCR
ACE Brachmann ct al. 2025
97.2
GLACE Wang al.
2024a
97.5
Marepo Chen et al.
20246
GS CP-CPRI
Liu et 2025
90.9
3D Gaussian
Ours
98.7
2cm, 2 metric 12 Scenes. To further evaluate localization performance, conduct experiments
on the 12 Scenes dataset using the 2cm, 2 metric. Table 6 presents the results comparison with other approaches. Our method achieves the highest accuracy in localization for the 2cm, 2 metric. Compared to the 3D Gaussian based refinement method, GS-CPR Liu et al. (2025). our approach demonstrates achieves 7.6% improvement.
A.6 More Detailed Studies of GS-RelocNet
Table eeeeeeindeeendent localization results on 7 Scenes with original ground truth and Cambridge Landmarks with different settings.
GS-RelocNet
Scenes
Cambridge Landmarks
Gaussian Number
SI
512
2.4/1. 14
14/0.36
S2
1024
2.0/0.94
12/0.29
S3
2048
1.7/0.89
10/0,22
54 4096
1.4/0.82
9/0.21
8192
1.5/0.88
9/0.21
16384
1.9/0.97
12/0.28
S7
Using Point Cloud instead of 3D Gaussians
2,5/1.43
16/0.45
Refinement Network
Rendering Method
S8
3D model
1.8/0.92
12/0.28
S9
NcRF
1.6/0.84
10/0,23
S10
3D GS
1. 1,4/0.82
9/0.21
Discussion 3D Gaussian number in GS-RelocNet. In S1 S6 of Table 7 we experiment with different numbers of 3D Gaussians. On one hand, the results with 512, 1024, and 16384 Gaussians are inferior to those with 4096 Gaussians, indicating that fewer Gaussians are insufficient for learning the scene features, while too many Gaussians increase the learning difficulty. On the other hand, the results with 2048, 4096, and 8192 Gaussians are comparable, suggesting that these configurations are sufficient for adequately learning the scene.
Subsequently, a common concern may lie in the large scene with 4096 Gaussians. To address this, framework may adopt coarse-to-fine strategy. In the coarse stage, uniformly sample 3D
24

---

Gaussians across the entire scene and select those with high confidence scores, iterating this process as needed. In the refinement stage, we focus on Gaussians in proximity to the high-confidence selections. establishing 2D-3D correspondences between image pixels and these 3D Gaussians, followed by PnP to estimate the camera pose.
Discussion of 3D Gaussian sampling strategy. For the sampling strategy, we uniformly sample 3D Gaussians within grid cells ensure a sufficient number of correspondences for robust pose estimation. Within each grid cell, Gaussians are randomly selected for training. Although we use 4096 Gaussians per training iteration, multiple iterations allow most Gaussians be utilized. To evaluate the robustness of this strategy, we conducted experiments using random sampling instead of uniform grid based sampling. The results show mean pose errors of 0. 7.76cm/0.26 in the scene-dependent setting on the 7-Scenes dataset and 7cm/0.18* on the Cambridge Landmarks dataset. These results indicate minimal performance degradation, demonstrating the robustness of our sampling strategy.
Discussion of usage of 3D Gaussians. In S7 of Table 7, we replace 3D Gaussians with the point cloud of the scene. The results clearly show a significant decrease in localization accuracy compared to using 3D Gaussians. We believe this is due to the fact that 3D Gaussians retain more texture and illumination information, while point clouds only capture geometric details.
Discussion of Spherical Harmonics (SH) in 3D GS. To evaluate the contribution of SH in 3D GS, we exclude SH and perform experiments on the 7 Scenes and Cambridge Landmarks datasets without using SH as the input of GS- RelooNee. The sene-independent results reveal a consistent increase in localization error. On 7 Scenes, the error rises from 1.4/0.82 to 1.7/0.93 (t 0.3/0.11), and on Cambridge Landmarks, it increases from 9/0.21 to 10 10/0.26 (t 1/0.05). This observed increase in error across both datasets confirms the effectiveness of SH in enhancing the performance of 3D GS for localization tasks.
Discussion of DINO v2 features. In GS-RelocNet, we also DINO v2 features as an additional input. To validate its effect, we conduct experiments without DINO v2 features. The results are 0.73/ 73.0 (f 0.02/0.02 with reeee-dependent setting), 1.4/0.84 (t 0.0/0.02 with veeee-nneeeendent setting) on 7 Scenes and 10/0.24 (1 1/0.03) on Cambridge Landmarks with scene-indepe secee-indeenneent setting. We can see that the additional feature plays a slightly positive role localization improvement. In the scene-dependent setting on 7 Scenes, it is even negligible. As an explanation, this is because that ScanNet contains large samples for training, making GS-RelocNet can learn the additional features itself.
A.7 More Detailed Studies of Refinement
Discussion of rendering method. In S8 - S10 of Table we present results using rendered images from both the 3D model and the NeRF network. The results demonstrate that the 3D GS based refinement yields the most significant improvements. When using rendered images from the 3D model, the results are comparable to those without refinement, likely due to the domain gap between rendered and real views. By incorporating NeRF and 3D GS, the lighting conditions are also considered, reducing this gap.
Discussion of cross-attention layers. To explore the potential of aross-attention layers, we conducted two ablation studies in the pose refinement stage. First, we integrated a cross-attention layer into the bidirectional feature fusion module of the refinement network. This yielded mean pose errors of 0.73cm/0.22 2 on the 7 Scenes dataset and 7cm/0.17 on the Cambridge Landmarks dataset in the scene-dependent setting, which are comparable to our original results. Second, we replaced the refinement network with Dust3R ViT variant, incorporating cross-attention. The dependet errors are 0. 0.76cm/0.27 on 7 Scenes and 9cm/0.21 on Cambridge Landmarks, indicating slightly reduced accuracy. We attribute the limited impact of cross-attention layers to the small input resolution of the refinement network (128 x 128), which constrains their effectiveness.
Discussion of environment conditions. Our current implementation does not include specific adaptations for environmental changes, such as variations in lighting or weather. Upon reviewing other 3D GS based methods, including GS-CPR and STDLoc, we found no explicit mention of specialized designs addressing such conditions. To mitigate the impact of environmental variations, we propose exploring 3D GS variants designed for enhanced robustness to environmental changes, as demonstrated recent works /Zhang et al. (2024); ; Kulhanek et al. (2024). These approaches could be integrated into our framework to improve performance under diverse conditions.
25

---

Discussion of RANSAC. confidence. To validate this, we also make the experiments with the traditional RANSAC using inlier number. The scene-independent results are 1.6/0.86 (f 0.1/0.04) and 10/0.24 (f 1/0.03) on Cambridge Landmarks. The results show slight accuracy improvement with confidence in RANSAC.
To predict the initial pose, we use the PnP with RANSAC using predicted
26


Reviews:
[{"review_id": "aEnu6hzhOs", "reviewer": "Reviewer_sxyi", "summary": "The paper tackles the task visual localization which consists in estimating the 6DoF pose of a query image within a known environment. It is a crucial task for AR/MR and autonomous-driving systems. One popular way of solving pose estimation is regressing the 3D position (in world coordinates) of each pixel within an image and estimating the pose through RANSAC+PnP. However this requires fitting the regression network on each scene independently and the underlying 3D point cloud representation lacks dense texture information. The authors hence leverage 3DGS representations which jointly encodes scene geometry and texture by introducing a scene independent visual localization pipeline. It consists of GS-RelocNet which establishes pixel to 3D Gaussians correspondences (used with a PnP+RANSAC loop to estimate an initial pose) and a refinement network that regresses residual coordinate maps between the query image and the image rendered from the initial pose (used to update the initial pose by estimating the relative pose with PnP+RANSAC). Both modules use fusion mechanisms to share information between different modalities in their two branches. The resulting pipeline is evaluated on two real world standard relocalization datasets.", "questions": "1) Do you have additional results to evaluate the generalization capabilities of GS-RelocNet?\n\n2) Which underlying 3DGS model is used? Which hyperparameters in the 3DGS model? How are the point clouds used to initialize the 3DGS models obtained?\nThere are some illumination variations in Cambridge Landmark, is this explicitly handled somewhere in the pipeline?\n\n3) For a query image, could you break down the runtime between the different steps of your pipeline?\n\n4) How many descriptors Ni per image or what is the patch embedding size?\n\nAdditional references for 3DGS visual localization:  \nGennady Sidorov, Malik Mohrat, Ksenia Lebedeva, Ruslan Rakhimov, and Sergey Kolyubin. “GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splatting for Improved Visual Localization.”  \nPietrantoni, Maxime, Gabriela Csurka, and Torsten Sattler. \"Gaussian Splatting Feature Fields for (Privacy-Preserving) Visual Localization.\" In CVPR 2025  \nHongjia Zhai, Xiyu Zhang, Boming Zhao, Hai Li, Yijia He, Zhaopeng Cui, Hujun Bao, and Guofeng Zhang. “SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality.” In TVCG 2025", "limitations": "Limitations not addressed", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n1) Conceptually, learning to regress point to 3D Gaussian correspondences is a good intermediate solution between feature matching and scene coordinate regression. GS-RelocNet combines two powerful backbones for image and point clouds through a feature fusion module which allows it to predict such correspondences with high accuracy. It is further supported by an ablation study.  \n2) 3DGS scene independent visual localization is a very relevant research direction and GS-RelocNet generalizes to unseen indoor scenes during training yielding very accurate pose estimation. \n3) The relocalization pipeline achieves state of the art results against rendering-based baselines and regression baselines on two standard relocalization datasets. \n4) The feedforward regression nature of the method makes the inference very efficient and allows the relocalization localization pipeline to almost run at real time. \n\n### Weaknesses\n1) Given that the scene-independent nature of the pipeline is advertised as the main contribution of the paper I find experiments around generalization to be too limited. Notably, the scene independent version is evaluated only in indoor environments which inherently facilitates generalization as room layouts and the semantic content of rooms are relatively similar between scenes. Furthermore, 7-Scenes is not a challenging dataset for visual localization given the small scale of the scenes and the dense distribution of training cameras.\n\n2) l.466 it is mentioned that for 7-Scenes, “ground truth poses were obtained using the KinectFusion system”. However, it seems like your pipeline (and some other methods in Tab. 1) is evaluated against the SfM pseudo ground truth poses from [1] while the rest of the methods in the table report results evaluated against the original Dslam pseudo ground truth poses. It is important that all methods within the table are evaluated against the same pseudo ground truth as results can significantly vary between both ground truths. Could you clarify which pseudo ground truth was used and if necessary evaluate your method against both ground truths.     \n\n[1] On the Limits of Pseudo Ground Truth in Visual Camera Re-Localisation. Eric Brachmann, Martin Humenberger, Carsten Rother, Torsten Sattler\n\n3) Matching based structure based methods ought to at least be mentioned in the introduction as, overall, they remain the visual localization approach with the best robustness and accuracy on real world scenes.    \n\n[2] Paul-Edouard Sarlin, Cesar Cadena, Roland Siegwart, and Marcin Dymczyk. From Coarse to Fine: Robust Hierarchical Localization at Large Scale. In CVPR, 2019.\n\n[3] Martin Humenberger, Yohann Cabon, Nicolas Guerin, Julien Morat, Jérôme Revaud, Philippe Rerole, Noé Pion, César Roberto de Souza, Vincent Leroy and Gabriela Csurka. Robust Image Retrieval-based Visual Localization using Kapture.\n\n4) There is a total lack of information regarding the underlying 3DGS model. \nSimilarly there is no information regarding RANSAC hyperparameters (number of iterations, thresholds) which can heavily influence the runtime.\nFor reproducibility purposes, this information should be added in the supplementary material.", "was_revised": true, "final_justification": "My concerns have been addressed through the rebuttal. Given the novelty of the approach and the state of the art localization accuracy I see no reason to reject the paper, I have upgraded my score accordingly.", "rebuttal": "**Responses to Reviewer sxyi**\n\nWe express our sincere gratitude for your insightful suggestions and valuable feedback, which have significantly enhanced the quality of our manuscript.\n\n**Weaness 1 & Question 1**. Generalization capabilities of GS-RelocNet. \n\n**Responses**. The reviewer raises a critical concern regarding the generalization capabilities of GS-RelocNet, an essential aspect of relocalization tasks. We appreciate this insightful comment. To address this, we conducted additional experiments as recommended, evaluating GS-RelocNet on two challenging indoor dynamic datasets: TUM RGB-D and Bonn. Notably, our training data from ScanNet consist exclusively of static scenes, while the test sequences include dynamic elements, posing a significant challenge for generalization.\n\nIn the TUM RGB-D test sequences, dynamic scenarios involving two individuals walking around a table increase the complexity of localization. Similarly, the Bonn dataset features highly dynamic sequences, such as individuals manipulating boxes or interacting with balloons. These experiments demonstrate the robustness of GS-RelocNet, which achieves substantial improvements in localization accuracy compared to existing SLAM systems.\n\nTable 1. RMSE of ATE [cm] results in four dynamic scenes of TUM RGB-D.\n\n| | fr3_walking_xyz | fr3_walking_static | fr3_walking_rpy | fr3_walking_half |  Mean |\n| :------: | :------: |:------: |:------: |:------: |:------: |\n| ORB-SLAM2 [1] | 45.9 | 9.3 | 65.8 | 32.8 | 38.5 |\n| DynaSLAM [2]  | 1.5 | 0.6 | 3.5 | 2.5 | 2.0 |\n| DS-SLAM [3]   | 2.5 | 0.8  | 44.4 | 3.0  | 12.7 |\n| LC-CRF SLAM [4] | 1.6 | 1.1 | 4.6 | 2.8 | 2.5 |\n| DGM-VINS [5] | 3.6 | 1.3 |7.1 | 3.3 | 3.8 |\n| Ours | **1.1** | **0.4** | **2.2** | **2.0**| **1.4**|\n\n**RMSE of ATE on dynamic TUM RGB-D**.\nTable 1 presents the Root Mean Square Error (RMSE) of Absolute Trajectory Error (ATE) for four dynamic scenes, compared against ORB-SLAM2, DynaSLAM, DS-SLAM, LC-CRF SLAM, and DGM-VINS. Our approach consistently outperforms these state-of-the-art SLAM systems in the RMSE of ATE metric, demonstrating superior localization accuracy in dynamic environments.\n\n**RMSE of ATE on dynamic Bonn**.\nWe evaluated our framework on the Bonn dataset across 20 test scenes, consistent with LC-CRF SLAM, and compared it with ReFusion [6], MaskFusion [7], and LC-CRF SLAM. The mean RMSE of ATE results are 23.8cm (ReFusion), 25.1cm (MaskFusion), 6.8cm (LC-CRF SLAM), and 4.3cm (Ours). Our framework achieves the lowest mean RMSE of ATE, highlighting its exceptional localization accuracy in highly dynamic scenes involving objects such as manipulated boxes or balloons.\n\n**Generalization on Cambridge Landmarks**.\nTo assess generalization, we trained GS-RelocNet on the MegaDepth dataset [8]  and tested it on the Cambridge Landmarks dataset in a scene-independent setting. The mean pose error across four scenes is 9cm/0.21°, surpassing the performance of DUSt3R (10cm/0.26°) and Reloc3R (38cm/0.49°). These results underscore the robustness and generalization capability of GS-RelocNet in diverse, unseen environments.\n\n**Reference**\n\n[1] R. Mur-Artal and J. D. Tardos, “Orb-slam2: An open-source slam ´\nsystem for monocular, stereo, and rgb-d cameras,” IEEE Transactions\non Robotics, vol. 33, no. 5, pp. 1255–1262, 2017.\n\n[2] B. Bescos, J. M. Facil, J. Civera, and J. Neira, “Dynaslam: Tracking, ´\nmapping, and inpainting in dynamic scenes,” IEEE Robotics and Au\u0002tomation Letters, vol. 3, no. 4, pp. 4076–4083, 2018\n\n[3] C. Yu, Z. Liu, X.-J. Liu, F. Xie, Y. Yang, Q. Wei, and Q. Fei, “Ds-slam:\nA semantic visual slam towards dynamic environments,” in IEEE/RSJ\nInternational Conference on Intelligent Robots and Systems, 2018, pp.\n1168–1174.\n\n[4] Z.-J. Du, S.-S. Huang, T.-J. Mu, Q. Zhao, R. R. Martin, and K. Xu,\n“Accurate dynamic slam using crf-based long-term consistency,” IEEE\nTransactions on Visualization and Computer Graphics, vol. 28, no. 4,\npp. 1745–1757, 2020\n\n[5] B. Song, X. Yuan, Z. Ying, B. Yang, Y. Song, and F. Zhou, “Dgm\u0002vins: Visual-inertial slam for complex dynamic environments with\njoint geometry feature extraction and multiple object tracking,” IEEE\nTransactions on Instrumentation and Measurement, vol. 72, pp. 1–11,\n2023.\n\n[6] E. Palazzolo, J. Behley, P. Lottes, P. Giguere, and C. Stachniss, “Re\u0002fusion: 3d reconstruction in dynamic environments for rgb-d cameras\nexploiting residuals,” in IEEE/RSJ International Conference on Intelli\u0002gent Robots and Systems, 2019, pp. 7855–7862.\n\n[7] M. Runz, M. Buffier, and L. Agapito, “Maskfusion: Real-time recogni\u0002tion, tracking and reconstruction of multiple moving objects,” in IEEE\nInternational Symposium on Mixed and Augmented Reality, 2018, pp.\n10–20.\n\n[8] Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos.  CVPR 2018.\n\n**Weaness 2**. Ground truth of 7-Scenes dataset. \n\n**Responses**. We appreciate the reviewer’s insightful comment regarding the ground truth poses for the 7-Scenes dataset. As noted, the dataset provides two types of ground truth: the original poses from KinectFusion and the SfM pseudo ground truth. We apologize for the lack of clarity in our initial submission. In the revised manuscript, we explicitly specify the ground truth used for each method in Table 1 of the paper. Specifically, ACE, GLACE, GS-CPR, STDLoc and ours utilize the SfM pseudo ground truth, while other methods, leverage the original KinectFusion ground truth.\n\nTo further address this concern, we conducted additional experiments using the original ground truth. Our method achieves mean pose errors of 1.21 cm / 0.61° in the scene-dependent setting and 1.44 cm / 0.82° in the scene-independent setting across the seven scenes. These results demonstrate superior accuracy compared to other methods evaluated with the original ground truth, highlighting the robustness of our approach.\n\n**Weaness 3**. Inclusion of match-based methods and references. \n\n**Responses**. We greatly appreciate the reviewer’s valuable suggestion. In response, we will incorporate a discussion of match-based methods into the introduction of the revised manuscript. Additionally, the referenced works have been included to provide a comprehensive background and context for our approach.\n\n**Weaness 4 & Question 2**. Implementation details of the 3DGS model and PnP with RANSAC. \n\n**Responses**. We sincerely appreciate the reviewer’s insightful comment. To construct the 3DGS model, we first utilize COLMAP to generate an initial point cloud using ground truth poses. Subsequently, we employ the original 3DGS model with its default configuration settings.\n\n--iterations: 30000. Total number of training iterations. \n\n--position\\_lr\\_init: 0.00016. The initial learning rate of the Gaussian position. \n\n--position\\_lr\\_final: 0.0000016. The final learning rate of the Gaussian position. \n\n--position\\_lr\\_delay\\_mult: 0.01. The delay multiplier before the learning rate decay begins. \n\n--position\\_lr\\_max\\_steps: 30000. The total number of steps for the learning rate decay. \n\n--feature\\_lr: 0.0025. Learning rate of spherical harmonic function coefficients. \n\n--opacity\\_lr: 0.05. Learning rate of opacity. \n\n--scaling\\_lr: 0.005. Scaling learning rate. \n\n--rotation\\_lr: 0.001. Learning rate of rotation.\n\n--densify\\_from\\_iter: 500. Densification begins from which iteration. \n\n--densify\\_until\\_iter: 15000. Densification ends at which iteration. \n\n--densification\\_interval: 100. Perform densification and pruning checks every few iterations. \n\n--opacity\\_prune\\_threshold: 0.005. Opacity pruning threshold. \n\n--densify\\_grad\\_threshold: 0.0002. The gradient threshold for densifying the Gaussian sphere. \n\nThe PnP with RANSAC uses OpenCV implementation with following parameters. \n\n--iterationsCount: 100. The number of RANSAC iterations. \n\n--reprojectionError: 8. Threshold for reprojection error. \n\n--confidence: 0.99. Degree of confidence.  \n\n--flags:SOLVEPNP\\_ITERATIVE. PnP solver algorithm. \n\nRegarding the code availability, we clarify in the submission checklist that the code is not publicly available during the review stage. Upon publication of the paper, the core code will be made publicly accessible to facilitate reproducibility.\n\n**Question 3**. Detailed running time of pipeline components. \n\n**Responses**. Our method comprises five key steps, including confidence regression, initial pose estimation using PnP with RANSAC, view rendering, residual coordinate regression, and pose refinement using PnP with RANSAC. We evaluated the computational efficiency of these steps across all test frames in the 7-Scenes and Cambridge Landmarks datasets. The average running times per frame are as follows: 39 ms for confidence regression, 4 ms for initial PnP with RANSAC, 9 ms for view rendering, 8 ms for residual coordinate regression, and 5 ms for PnP in pose refinement.\n\n**Question 4**. How many descriptors Ni per image or what is the patch embedding size? \n\n**Responses**. We thank the reviewer for this insightful question. The number of descriptors per image, $N_i$ per image is 4096 ($64 \\times 64$). \nThe patch embedding size of both GS-RelocNet and pose refinement network is $64 \\times 64$.\n\n**Limitation discussion**. A primary limitation of our framework is its reliance on a high-quality 3D GS model of the target scene. When the 3D GS model is of suboptimal quality, localization performance may degrade, leading to failures or significant errors."}, {"review_id": "PKVFaq2amh", "reviewer": "Reviewer_TpKv", "summary": "The paper proposes a camera localization method based on 3D Gaussian splatting.\n\nIn the first stage, the method generates 2D–3D correspondences. A neural network, GS-RelocNet, takes as input the RGB image and the 3D Gaussians, and outputs a confidence matrix. 2D–3D correspondences are derived from this matrix, and the pose is then estimated using PnP+RANSAC. Within GS-RelocNet, a unidirectional feature fusion module is proposed. GS-RelocNet is trained to regress a ground-truth confidence matrix, where each entry encodes the Mahalanobis distance between pixel coordinates and the reprojected 3D points.\n\n\nThe second stage is a refinement step. A refinement network takes the real RGB image and a rendered RGB image and outputs a residual coordinate map, following Wang & Qi. Here, a bidirectional feature fusion module is proposed. The network is trained using a coordinate loss map and a pose loss.\n\nThe approach is evaluated on 7-Scenes, Cambridge Landmarks, and an additional dataset (12-Scenes, in the supplementary material).\nIn the scene-dependent setting, the results are comparable to the state-of-the-art STDLoc (CVPR 2025).\nIn the scene-independent setting, the proposed method is the first to use Gaussian splatting and significantly outperforms existing methods on 7-Scenes.\nMoreover, the method is considerably faster than its competitors: 65 ms versus 143 ms for STDLoc.", "questions": "1. **Ablation study** -  Could you include an architecture using cross-attention layers in the ablation study, or explain why this was not considered appropriate? \n2. **Technical details** - Since the code will not be released, could you provide the missing details (e.g., tensor dimensions) to make re-implementation easier?\n3. **Auxiliary loss** - For the auxiliary loss, how did you backpropagate through the predicted camera pose produced by the RANSAC module?\n4. **Computation time** - Could you provide a breakdown of the computation time for each module (GS-RelocNet, PnP+RANSAC, and refinement iterations)?", "limitations": "Although the authors indicate in the checklist that limitations are discussed, I was unable to find any such discussion in the main paper or the supplementary material.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths\n1. The paper proposes the first scene-independent camera localization method based on 3D Gaussian splatting.\n2. The method is computationally efficient.\n3. The outperforms state-of-the-art methods on 7-Scenes in the scene-independent setting.\n4. An extensive ablation study supports the architectural and design choices.\n\n## Weaknesses\n1. Only a single NeurIPS paper (Zhu, 2024) is cited, which raises concerns about whether the paper is fully within the scope of NeurIPS.\n\n2. The authors do not plan to release their code and provide no justification. Reproducing the results may be challenging, as some implementation details are missing. For example, in the description of the Unidirectional Feature Fusion module, it appears that RGB features are reshaped to apply the residual connection, but the input and output dimensions are not specified.\n\n3. Two main contributions are the unidirectional and bidirectional feature fusion modules, but it is unclear why classical cross-attention layers were not considered instead. I would have expected a comparison with more standard cross-attention architectures (e.g., Dust3R’s ViT variant) in the ablation study.", "was_revised": true, "final_justification": "The authors have addressed most of my initial concerns (Ablation study, Technical details, Auxiliary loss and Computation time), and I have accordingly increased my rating.", "rebuttal": "**Responses to Reviewer TpKv**\n\nWe express our sincere gratitude for your insightful suggestions and valuable feedback, which have significantly enhanced the quality of our manuscript.\n\n**Weaness 1**. NeurIPS paper citation. \n\n**Responses**. Thanks for the helpful comment. \nWe add the following papers in our paper.\n\n[1] Zhu J, Yan S, Wang L, et al. LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment[J]. NIPS 2024.\n\n[2] Xu Y, Jiang H, Xiao Z, et al. Dg-slam: Robust dynamic gaussian splatting slam with hybrid pose optimization. NIPS 2024.\n\n[3] Hu J, Mao M, Bao H, et al. CP-SLAM: Collaborative neural point-based SLAM system. NIPS 2023.\n\n[4] Wang Y, Yan Y, Shi D, et al. NeRF-IBVS: visual servo based on nerf for visual localization and navigation. NIPS 2023.\n\n**Weaness 2 & Question 2**. Technical details. \n\n**Responses**. We appreciate the reviewer’s insightful comment and apologize for the lack of clarity in describing the unidirectional feature fusion module. The module is designed to maintain the same output size as the input RGB feature. Let the input RGB feature have dimensions $H_u \\times W_u \\times D_u$, and the model feature have dimensions $N_m \\times D_m$. The fusion process proceeds as follows. \n\n1.The RGB feature is reshaped to $N_m \\times (H_u \\times W_u \\times D_u / N_m)$, followed by a 1D convolution to transform it to $N_m \\times D_m$, aligning with the model feature’s dimensions.\n\n2.Both features undergo self-attention operations and are subsequently combined through element-wise addition.\n\n3.The combined feature is reshaped to $H_u \\times W_u \\times (N_m \\times D_m / H_u / W_u)$. A Swin Transformer block is then applied to restore the feature dimensions to $H_u \\times W_u \\times D_u$, matching the input RGB feature. The input RGB feature is added to this output to produce the final merged feature.\n\nWe acknowledge an error in the bottom left part of Figure 1, where a redundant line depicting the addition of the input RGB feature was included; this will be corrected in the revised manuscript. We apologize for this oversight.\n\nFor details on the 3DGS and RANSAC implementations, please refer to **Weaness 4 & Question 2** of **Reviewer sxyi**.\n\nAs stated in the submission checklist, the code is not publicly available during the review stage. **Upon publication, the core code will be released to ensure reproducibility.**\n\n**Weaness 3 & Question 1**. Ablation study of cross-attention layers. \n\n**Responses**. We appreciate the reviewer’s insightful comment regarding the efficacy of cross-attention layers, which have demonstrated strong performance in prior work. Our decision to exclude cross-attention layers in the unidirectional feature fusion module stems from the need to preserve the independence of the 3DGS branch. This design ensures that the point cloud features remain unaffected by RGB features during inference, thereby maintaining computational efficiency. Incorporating cross-attention layers would allow 3DGS features to influence RGB features, significantly increasing the framework’s runtime.\n\nTo explore the potential of cross-attention layers, we conducted two ablation studies in the pose refinement stage. First, we integrated a cross-attention layer into the bidirectional feature fusion module of the refinement network. This yielded mean pose errors of 0.73 cm / 0.24° in the scene-dependent setting on the 7-Scenes dataset and 7cm/ 0.17 on the Cambridge Landmarks dataset, comparable to our original results. Second, we replaced the refinement network with Dust3R’s ViT variant, incorporating cross-attention. This resulted in mean pose errors of 0.76 cm / 0.27° on 7-Scenes and 9cm/ 0.21 on Cambridge Landmarks, indicating slightly reduced accuracy. We attribute the limited impact of cross-attention layers to the small input resolution of the refinement network ($128 \\times 128$), which constrains their effectiveness.\n\n**Question 3**. Auxiliary loss. \n\n**Responses**.  We appreciate the reviewer’s query regarding the auxiliary loss. In our approach, the auxiliary loss directly predicts the 6-DoF pose (3-DoF translation and 3-DoF rotation) rather than estimating the pose via PnP from coordinates, similar to the methodology in PoseNet [1]. This loss is designed to enhance the geometric awareness of the pose refinement network, improving its ability to capture spatial relationships effectively.\n\n[1] Kendall, Alex and Cipolla, Roberto. Geometric loss functions for camera pose regression with deep learning. CVPR 2017\n\n**Question 4**. Computation time. \n\n**Responses**. Our method comprises five key steps, including confidence regression, initial pose estimation using PnP with RANSAC, view rendering, residual coordinate regression, and pose refinement using PnP with RANSAC. We evaluated the computational efficiency of these steps across all test frames in the 7-Scenes and Cambridge Landmarks datasets. The average running times per frame are as follows: 39 ms for confidence regression, 4 ms for initial PnP with RANSAC, 9 ms for view rendering, 8 ms for residual coordinate regression, and 5 ms for PnP in pose refinement.\n\n**Limitation discussion**. A primary limitation of our framework is its reliance on a high-quality 3D GS model of the target scene. When the 3D GS model is of suboptimal quality, localization performance may degrade, leading to failures or significant errors."}, {"review_id": "hyECNKTG92", "reviewer": "Reviewer_EzZn", "summary": "This paper proposes a novel scene-independent camera relocalization framework based on 3D GS, comprising two stages: initial pose estimation via 2D-3D correspondences between image pixels and 3D Gaussians, and pose refinement using rendered images from 3D GS. The framework introduces GS-RelocNet to establish correspondences and integrates a unidirectional 2D-3D feature fusion module and a bidirectional image feature fusion module. Experiments on 7 Scenes and Cambridge Landmarks demonstrate state-of-the-art performance, particularly in scene-independent relocalization, with significant improvements over existing methods.", "questions": "1. It is suggested to evaluate the generalization performance of the proposed method across multiple test datasets used for scene-independent methods, similar to how methods like DUSt3R and VGGT are evaluated.\n\n2. It is suggested to briefly explain the connections of each method module in the caption of Figure 1 to enhance readability.\n\n3. There is a contradiction between \"Scene-independent\" and \"Relocalization\" in the title, as Relocalization inherently implies scene-dependent settings. It is suggested to replace \"Relocalization\" with \"Localization\".", "limitations": "The experimental scenes in the paper are relatively simple and structured. It is suggested to enhance the method's performance in large-scale scenes with extreme appearance variations, which are more significant for real-world applications.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The first scene-independent relocalization framework using 3D GS.\n\n2. The unidirectional and bidirectional fusion modules enhance feature sharing, validated by ablation studies showing substantial performance gains.\n\n3. Runs at 15.4 FPS, outperforming other 3D GS-based methods, making it suitable for real-world applications.\n\nWeaknesses\n1. The paper emphasizes scene-independent methods. However, the paper only includes one benchmark on 7Scenes under the scene-independent setting. Therefore, more experiments (such as Cambridge Landmarks) under scene-independent conditions should be included.\n\n2. The evaluated datasets are relatively simple and structured, where the current method already demonstrates excellent performance. It is suggested to conduct evaluations on more challenging datasets.\n\n2. The Introduction is somewhat confusing. It is suggested to refine it to clearly express the logical links among the challenges, motivation, and methodology.", "was_revised": true, "final_justification": "The authors have addressed my concerns. It is impressive that the proposed method achieves better results than DUSt3R despite using significantly less training data. Therefore, I am increasing my score to 4.", "rebuttal": "**Responses to Reviewer EzZn**\n\nWe express our sincere gratitude for your insightful suggestions and valuable feedback, which have significantly enhanced the quality of our manuscript.\n\n**Weanesses 1,2 & Question 1**. Generalization capabilities. \n\n**Responses**. Thanks for the helpful comment. \n To show the generalization capabilities of GS-RelocNet, we make more experiments as suggested. Specifically, we conducted additional experiments as recommended, evaluating GS-RelocNet on two challenging indoor dynamic datasets: TUM RGB-D and Bonn. Notably, our training data from ScanNet consist exclusively of static scenes, while the test sequences include dynamic elements, posing a significant challenge for generalization.\n\nIn the TUM RGB-D test sequences, dynamic scenarios involving two individuals walking around a table increase the complexity of localization. Similarly, the Bonn dataset features highly dynamic sequences, such as individuals manipulating boxes or interacting with balloons. These experiments demonstrate the robustness of GS-RelocNet, which achieves substantial improvements in localization accuracy compared to existing SLAM systems.\n\nTable 1. RMSE of ATE [cm] results in four dynamic scenes of TUM RGB-D.\n\n| | fr3_walking_xyz | fr3_walking_static | fr3_walking_rpy | fr3_walking_half |  Mean |\n| :------: | :------: |:------: |:------: |:------: |:------: |\n| ORB-SLAM2 [1] | 45.9 | 9.3 | 65.8 | 32.8 | 38.5 |\n| DynaSLAM [2]  | 1.5 | 0.6 | 3.5 | 2.5 | 2.0 |\n| DS-SLAM [3]   | 2.5 | 0.8  | 44.4 | 3.0  | 12.7 |\n| LC-CRF SLAM [4] | 1.6 | 1.1 | 4.6 | 2.8 | 2.5 |\n| DGM-VINS [5] | 3.6 | 1.3 |7.1 | 3.3 | 3.8 |\n| Ours | **1.1** | **0.4** | **2.2** | **2.0**| **1.4**|\n\n**RMSE of ATE on dynamic TUM RGB-D**.\nTable 1 presents the Root Mean Square Error (RMSE) of Absolute Trajectory Error (ATE) for four dynamic scenes, compared against ORB-SLAM2, DynaSLAM, DS-SLAM, LC-CRF SLAM, and DGM-VINS. Our approach consistently outperforms these state-of-the-art SLAM systems in the RMSE of ATE metric, demonstrating superior localization accuracy in dynamic environments.\n\n**RMSE of ATE on dynamic Bonn**.\nWe evaluated our framework on the Bonn dataset across 20 test scenes, consistent with LC-CRF SLAM, and compared it with ReFusion [6], MaskFusion [7], and LC-CRF SLAM. The mean RMSE of ATE results are 23.8cm (ReFusion), 25.1cm (MaskFusion), 6.8cm (LC-CRF SLAM), and 4.3cm (Ours). Our framework achieves the lowest mean RMSE of ATE, highlighting its exceptional localization accuracy in highly dynamic scenes involving objects such as manipulated boxes or balloons.\n\n**Generalization on Cambridge Landmarks**.\nTo assess generalization, we trained GS-RelocNet on the MegaDepth dataset [8]  and tested it on the Cambridge Landmarks dataset in a scene-independent setting. The mean pose error across four scenes is 9cm/0.21°, surpassing the performance of DUSt3R (10cm/0.26°) and Reloc3R (38cm/0.49°). These results underscore the robustness and generalization capability of GS-RelocNet in diverse, unseen environments.\n\n**Reference**\n\n[1] R. Mur-Artal and J. D. Tardos, “Orb-slam2: An open-source slam ´\nsystem for monocular, stereo, and rgb-d cameras,” IEEE Transactions\non Robotics, vol. 33, no. 5, pp. 1255–1262, 2017.\n\n[2] B. Bescos, J. M. Facil, J. Civera, and J. Neira, “Dynaslam: Tracking, ´\nmapping, and inpainting in dynamic scenes,” IEEE Robotics and Au\u0002tomation Letters, vol. 3, no. 4, pp. 4076–4083, 2018\n\n[3] C. Yu, Z. Liu, X.-J. Liu, F. Xie, Y. Yang, Q. Wei, and Q. Fei, “Ds-slam:\nA semantic visual slam towards dynamic environments,” in IEEE/RSJ\nInternational Conference on Intelligent Robots and Systems, 2018, pp.\n1168–1174.\n\n[4] Z.-J. Du, S.-S. Huang, T.-J. Mu, Q. Zhao, R. R. Martin, and K. Xu,\n“Accurate dynamic slam using crf-based long-term consistency,” IEEE\nTransactions on Visualization and Computer Graphics, vol. 28, no. 4,\npp. 1745–1757, 2020\n\n[5] B. Song, X. Yuan, Z. Ying, B. Yang, Y. Song, and F. Zhou, “Dgm\u0002vins: Visual-inertial slam for complex dynamic environments with\njoint geometry feature extraction and multiple object tracking,” IEEE\nTransactions on Instrumentation and Measurement, vol. 72, pp. 1–11,\n2023.\n\n[6] E. Palazzolo, J. Behley, P. Lottes, P. Giguere, and C. Stachniss, “Re\u0002fusion: 3d reconstruction in dynamic environments for rgb-d cameras\nexploiting residuals,” in IEEE/RSJ International Conference on Intelli\u0002gent Robots and Systems, 2019, pp. 7855–7862.\n\n[7] M. Runz, M. Buffier, and L. Agapito, “Maskfusion: Real-time recogni\u0002tion, tracking and reconstruction of multiple moving objects,” in IEEE\nInternational Symposium on Mixed and Augmented Reality, 2018, pp.\n10–20.\n\n[8] Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos.  CVPR 2018.\n\n**Weaness 3**. Improvements to the introduction. \n\n**Responses**. We sincerely thank the reviewer for their valuable feedback. Below, we outline the proposed enhancements to the introduction, addressing challenges, motivation, and methodology.\n\n**Challenges**. The 3DGS model excels in novel view synthesis, offering new opportunities for visual localization pipelines. However, leveraging 3DGS for robust and accurate localization remains a significant challenge.\n\n**Motivation**. Current methods predominantly utilize 3D Gaussians for pose refinement, which heavily relies on the accuracy of initial camera pose estimation. When the initial pose estimation is inaccurate or fails, the refinement process becomes ineffective. To address this, our work proposes a novel framework that integrates 3DGS for both pose initialization and refinement, while achieving scene-independent relocalization to enhance robustness.\n\n**Methodology**. We introduce a novel relocalization framework that first establishes 2D-3D correspondences between image pixels and 3D Gaussians, followed by a refinement stage that predicts the relative pose between real and rendered views using 3DGS. A feature fusion module is incorporated in both stages to enhance correspondence regression. To the best of our knowledge, this is the first 3DGS-based, scene-independent relocalization framework, offering a robust solution for challenging localization tasks.\n\n**Question 2**. Connections between method modules in the caption of Figure 1. \n\n**Responses**. We sincerely appreciate the reviewer’s insightful comment. The revised caption for Figure 1 is presented below.\n\nArchitecture of GS-RelocNet. The GS-RelocNet framework integrates an RGB feature encoder, a 3D Gaussian feature encoder, an RGB descriptor decoder, a point descriptor decoder, and a confidence metric regression module. In the figure, arrows indicate the process flow, and numbers adjacent to each block denote the corresponding filter size.\n\n**Question 3**. \"Scene-independent\" and \"Relocalization\". \n\n**Responses**. We sincerely appreciate the reviewer’s insightful comment. Our method performs relocalization by regressing the camera pose within the coordinate space of a known 3DGS model, aligning with the standard definition of relocalization. The term \"scene-independent\" indicates that our framework can achieve robust relocalization in a target scene without requiring scene-specific pre-training, in contrast to most \"scene-dependent\" methods that necessitate prior training on the target scene.\n\nIn the revised version, we add these to express the terms more clearly."}, {"review_id": "adqNPbfYBe", "reviewer": "Reviewer_54m2", "summary": "This paper proposes a camera relocalization system coupled with unidirectional and bidirectional feature fusion module on 3DGS scenes. It extends the state of the art in neural rendering and pose estimation by allowing robust relocalization in unseen environments without scene-specific training. \nThe pipeline is structured in two stages: \n1. A coarse pose estimation module, GS-RelocNet, is used to obtain the confidence matrix between image pixels and randomly selected Gaussians. The confidence matrix is further processed with PnP with RANSAC algorithm to obtain the initial camera pose. GS-RelocNet is trained by comparing with the groundtruth confidence matrix. \n2. A rendered view under the predicted camera pose is processed together with the groundtruth rendering by a refinement network. This network outputs the residual coordinate map, also followed by a PnP method with RANSAC to predict the relative camera pose. Refined pose is calculated based on the original pose and relative pose. The refinement network is trained with coordinate map loss and an auxiliary loss for camera’s position and orientation.\n\nPerformance on existing datasets claim high localization accuracy and efficiency (~15 fps). The refinement network appears to improve significantly compared to scene-independent methods.", "questions": "1. The paper samples 3D Gaussian components uniformly within grid cells to construct correspondences. However, different 3D Gaussian components may have different scales and sizes, which in turn may influence their respective projected area on the image plane. This variation could affect the confidence scores within the confidence matrix and subsequently the final pose estimation. How sensitive is your method’s performance to the sampling strategy of 3D Gaussian components with different sizes?\n\n2. The pipeline utilizes rendered views from 3D Gaussian. However, rendered images may be sensitive to environment conditions, (e.g. lighting directions or intensities, artifacts from the original 3DGS scene). How robust is your method to these variations in rendered feature maps?\n\n3. This pipeline is trained on limited datasets. Under what conditions does the system fail (e.g., low-texture regions, reflective surfaces)?", "limitations": "The paper assumes the scene is static. The method’s ability to deal with moving objects, structural modifications, or other dynamic components — a common scenario in real-world applications — is currently unclear or unaddressed.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Scene-independent relocalization for 3DGS scenes: This paper achieves zero-shot generalization based solely on renderings and 3DGS features, unlike prior work 3DGS-Reloc[1], which relied on scene-specific LiDAR scans.\n2. Feature fusion architecture is well-motivated. Outperforms single-path approaches in GS-CPR [2].\n\nWeaknesses:\n1. In this paper, there are 4096 selected 3DGS for training and inference. For large-scale scenes, this resolution may reduce relocation accuracy.\n2. The method proposed in this paper may require higher gpu memory usage compared to other models.\n\n[1] Jiang P, Pandey G, Saripalli S. 3dgs-reloc: 3d gaussian splatting for map representation and visual relocalization[J]. arXiv preprint arXiv:2403.11367, 2024.\n\n[2] Liu C, Chen S, Bhalgat Y S, et al. GS-CPR: Efficient camera pose refinement via 3d gaussian splatting[C]//The Thirteenth International Conference on Learning Representations. 2025.", "was_revised": true, "final_justification": "The authors added further analysis and experimental result to address my concerns. Including these in the final version would enhance the quality of the submission.", "rebuttal": "**Responses to Reviewer 54m2**\n\nWe express our sincere gratitude for your insightful suggestions and valuable feedback, which have significantly enhanced the quality of our manuscript.\n\n**Weaness 1**. Large-scale scenes with 4096 Gaussians. \n\n**Responses**. We sincerely appreciate the reviewer’s insightful comment. To address large-scale scenes, our framework may adopt a coarse-to-fine strategy. In the coarse stage, we uniformly sample 3D Gaussians across the entire scene and select those with high confidence scores, iterating this process as needed. In the refinement stage, we focus on Gaussians in proximity to the high-confidence selections, establishing 2D-3D correspondences between image pixels and these 3D Gaussians, followed by PnP to estimate the camera pose.\n\n**Weaness 2**. GPU memory usage. \n\n**Responses**. We appreciate the reviewer’s insightful query regarding GPU memory usage. As the two referenced methods do not report GPU memory details, a direct comparison is not feasible. In our framework, model features for the 3DGS branch are pre-computed prior to relocalization, allowing us to execute only the RGB branch and the feature fusion module during inference. This approach significantly reduces GPU memory consumption. Additionally, we report the model parameters: GS-RelocNet comprises approximately 81M parameters, while the pose refinement network contains 12M parameters.\n\n**Question 1**. Different scales and sizes of 3D Gaussians.. \n\n**Responses**. We sincerely appreciate the reviewer’s insightful comment. Our relocalization framework establishes 2D-3D correspondences between image pixels and 3D Gaussians, followed by PnP to estimate the camera pose. For each 3D Gaussian, we employ confidence regression to identify its most closely matched pixel. The scale and size of 3D Gaussians are represented by their covariance matrix, which is incorporated into the loss function (Equation 1). This scale and size information enhances the accuracy of confidence regression.\n\nFor the sampling strategy, we uniformly sample 3D Gaussians within grid cells to ensure a sufficient number of correspondences for robust pose estimation. Within each grid cell, Gaussians are randomly selected for training. Although we use 4096 Gaussians per training iteration, multiple iterations allow most Gaussians to be utilized. To evaluate the robustness of this strategy, we conducted experiments using random sampling instead of uniform grid-based sampling. The results show mean pose errors of 0.76cm/ 0.26° in the scene-dependent setting on the 7-Scenes dataset and 7cm/ 0.18° on the Cambridge Landmarks dataset. These results indicate minimal performance degradation, demonstrating the robustness of our sampling strategy.\n\n**Question 2**. Environment conditions. \n\n**Responses**. We sincerely appreciate the reviewer’s insightful query regarding environmental conditions. Our current implementation does not include specific adaptations for environmental changes, such as variations in lighting or weather. Upon reviewing other 3DGS based methods, including GS-CPR and STDLoc, we found no explicit mention of specialized designs addressing such conditions. To mitigate the impact of environmental variations, we propose exploring 3DGS variants designed for enhanced robustness to environmental changes, as demonstrated in recent works [1, 2]. These approaches could be integrated into our framework to improve performance under diverse conditions.\n\n[1] Zhang D, Wang C, Wang W, et al. Gaussian in the wild: 3d gaussian splatting for unconstrained image collections. ECCV 2024.\n\n[2] Kulhanek J, Peng S, Kukelova Z, et al. Wildgaussians: 3d gaussian splatting in the wild. arXiv preprint arXiv:2407.08447, 2024.\n\n**Question 3**. Limited datasets. \n\n**Responses**. Thanks for the helpful comment. \n To further show the generalization capabilities of GS-RelocNet, we make more experiments as suggested. Specifically, we conducted additional experiments as recommended, evaluating GS-RelocNet on two challenging indoor dynamic datasets: TUM RGB-D and Bonn. Notably, our training data from ScanNet consist exclusively of static scenes, while the test sequences include dynamic elements, posing a significant challenge for generalization.\n\nIn the TUM RGB-D test sequences, dynamic scenarios involving two individuals walking around a table increase the complexity of localization. Similarly, the Bonn dataset features highly dynamic sequences, such as individuals manipulating boxes or interacting with balloons. These experiments demonstrate the robustness of GS-RelocNet, which achieves substantial improvements in localization accuracy compared to existing SLAM systems.\n\nTable 1. RMSE of ATE [cm] results in four dynamic scenes of TUM RGB-D.\n\n| | fr3_walking_xyz | fr3_walking_static | fr3_walking_rpy | fr3_walking_half |  Mean |\n| :------: | :------: |:------: |:------: |:------: |:------: |\n| ORB-SLAM2 [1] | 45.9 | 9.3 | 65.8 | 32.8 | 38.5 |\n| DynaSLAM [2]  | 1.5 | 0.6 | 3.5 | 2.5 | 2.0 |\n| DS-SLAM [3]   | 2.5 | 0.8  | 44.4 | 3.0  | 12.7 |\n| LC-CRF SLAM [4] | 1.6 | 1.1 | 4.6 | 2.8 | 2.5 |\n| DGM-VINS [5] | 3.6 | 1.3 |7.1 | 3.3 | 3.8 |\n| Ours | **1.1** | **0.4** | **2.2** | **2.0**| **1.4**|\n\n**RMSE of ATE on dynamic TUM RGB-D**.\nTable 1 presents the Root Mean Square Error (RMSE) of Absolute Trajectory Error (ATE) for four dynamic scenes, compared against ORB-SLAM2, DynaSLAM, DS-SLAM, LC-CRF SLAM, and DGM-VINS. Our approach consistently outperforms these state-of-the-art SLAM systems in the RMSE of ATE metric, demonstrating superior localization accuracy in dynamic environments.\n\n**RMSE of ATE on dynamic Bonn**.\nWe evaluated our framework on the Bonn dataset across 20 test scenes, consistent with LC-CRF SLAM, and compared it with ReFusion [6], MaskFusion [7], and LC-CRF SLAM. The mean RMSE of ATE results are 23.8cm (ReFusion), 25.1cm (MaskFusion), 6.8cm (LC-CRF SLAM), and 4.3cm (Ours). Our framework achieves the lowest mean RMSE of ATE, highlighting its exceptional localization accuracy in highly dynamic scenes involving objects such as manipulated boxes or balloons.\n\n**Generalization on Cambridge Landmarks**.\nTo assess generalization, we trained GS-RelocNet on the MegaDepth dataset [8]  and tested it on the Cambridge Landmarks dataset in a scene-independent setting. The mean pose error across four scenes is 9cm/0.21°, surpassing the performance of DUSt3R (10cm/0.26°) and Reloc3R (38cm/0.49°). These results underscore the robustness and generalization capability of GS-RelocNet in diverse, unseen environments.\n\n**Failure case discussion**. In the Stairs scene of the 7-Scenes dataset, our framework exhibits failures in regions with repetitive or low-texture patterns. These failures likely stem from the visual ambiguity caused by similar views across different positions, which confounds accurate camera pose estimation using solely RGB inputs. To address this limitation, incorporating temporal sequence information could enhance robustness by providing additional context for disambiguating such challenging regions.\n\n**Reference**\n\n[1] R. Mur-Artal and J. D. Tardos, “Orb-slam2: An open-source slam ´\nsystem for monocular, stereo, and rgb-d cameras,” IEEE Transactions\non Robotics, vol. 33, no. 5, pp. 1255–1262, 2017.\n\n[2] B. Bescos, J. M. Facil, J. Civera, and J. Neira, “Dynaslam: Tracking, ´\nmapping, and inpainting in dynamic scenes,” IEEE Robotics and Au\u0002tomation Letters, vol. 3, no. 4, pp. 4076–4083, 2018\n\n[3] C. Yu, Z. Liu, X.-J. Liu, F. Xie, Y. Yang, Q. Wei, and Q. Fei, “Ds-slam:\nA semantic visual slam towards dynamic environments,” in IEEE/RSJ\nInternational Conference on Intelligent Robots and Systems, 2018, pp.\n1168–1174.\n\n[4] Z.-J. Du, S.-S. Huang, T.-J. Mu, Q. Zhao, R. R. Martin, and K. Xu,\n“Accurate dynamic slam using crf-based long-term consistency,” IEEE\nTransactions on Visualization and Computer Graphics, vol. 28, no. 4,\npp. 1745–1757, 2020\n\n[5] B. Song, X. Yuan, Z. Ying, B. Yang, Y. Song, and F. Zhou, “Dgm\u0002vins: Visual-inertial slam for complex dynamic environments with\njoint geometry feature extraction and multiple object tracking,” IEEE\nTransactions on Instrumentation and Measurement, vol. 72, pp. 1–11,\n2023.\n\n[6] E. Palazzolo, J. Behley, P. Lottes, P. Giguere, and C. Stachniss, “Re\u0002fusion: 3d reconstruction in dynamic environments for rgb-d cameras\nexploiting residuals,” in IEEE/RSJ International Conference on Intelli\u0002gent Robots and Systems, 2019, pp. 7855–7862.\n\n[7] M. Runz, M. Buffier, and L. Agapito, “Maskfusion: Real-time recogni\u0002tion, tracking and reconstruction of multiple moving objects,” in IEEE\nInternational Symposium on Mixed and Augmented Reality, 2018, pp.\n10–20.\n\n[8] Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos.  CVPR 2018."}]

Author rebuttal:

Meta-review: This paper presents a novel two-stage, scene-independent visual localization framework using 3D Gaussian Splatting. It first establishes coarse 2D-3D correspondences for an initial pose estimate and then refines it using a rendered view, with both stages enhanced by specialized feature fusion modules. The proposed method demonstrate state-of-the-art accuracy on standard benchmarks. It is also highly efficient, operating at near real-time speeds.

After rebuttal, the final rates are 3 Borderline accepts and 1 Accept. The authors solved reviewers' concerns on generalization capabilities, computation overhead and ablation studies. The reviewers give positive comments and recognition. Given these positive reviews, the AC recommends acceptance and suggests to further improve the paper based on the reviewers' comments.
Decision: Accept (poster)