Graph-Guided Scene Reconstruction from Images with 3D Gaussian Splatting
Authors: Chong Cheng
Gaochao Song
Yiyang Yao
Qinzheng Zhou
Gangjian Zhang
Hao Wang
Venue: ICLR 2025 Poster (2025)
Keywords: 3D Gaussian Splatting
3D Reconstruction
NeRF
Large Scene Reconstruction
Graph
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: This paper investigates an open research challenge of reconstructing high-quality, large-scale 3D open scenes from images. It is observed existing methods have various limitations, such as requiring precise camera poses for input and dense viewpoints for supervision. 
To perform effective and efficient 3D scene reconstruction, we propose a novel graph-guided 3D scene reconstruction framework, GraphGS. Specifically, given a set of images captured by RGB cameras on a scene, we first design a spatial prior-based scene structure estimation method. This is then used to create a camera graph that includes information about the camera topology. Further, we propose to apply the graph-guided multi-view consistency constraint and adaptive sampling strategy to the 3D Gaussian Splatting optimization process. This greatly alleviates the issue of Gaussian points overfitting to specific sparse viewpoints and expedites the 3D reconstruction process. We demonstrate GraphGS achieves high-fidelity 3D reconstruction from images, which presents state-of-the-art performance through quantitative and qualitative evaluation across multiple datasets.

Published as a conference paper at ICLR 2025
GRAPH-GUIDED SCENE RECONSTRUCTION FROM IM- AGES WITH 3D GAUSSIAN SPLATTING
Chong Cheng"
Gaochao Song"
Yiyang Yaod
Qinzheng Zhou*
Gangjian Zhang'
Hao Wang"
(HZUST(GZ) "KKU
*SCUT
4UC Berkeley
ABSTRACT
This paper investigates an open research challenge of reconstructing high-quality, large 3D open scenes from images. It is observed existing methods have vari- ous limitations, such as requiring precise camera poses for input and dense view- points for supervision. To perform effective and efficient 3D scene reconstruction, we propose novel graph-guided 3D scene reconstruction framework, GraphGS. Specifically, given a set of images captured by RGB cameras on a scene, we first design a spatial prior-based scene structure estimation method. This then used to create camera graph that includes information about the camera topology. Further, we propose to apply the and adaptive sampling strategy to the 3D Gaussian Splatting optimization pro- cess. This greatly alleviates the issue of Gaussian points overfitting to specific sparse viewpoints and expedites the 3D reconstruction process. We demonstrate GraphGS achieves high-fidelity 3D reconstruction from images, which presents state-of-the-an art performance through quantitative and qualitative evaluation across multiple datasets. Project Page: /tttps://Sdagett world github. io/ graphgs/
multi-view consistency constraint
INTRODUCTION
3D scene reconstruction aims to transform 2D images into realistic 3D scenes. This technology has
many practical applications such as Augmented Reality (AR) and Virtual Reality (VR) (Cheng al.,
Guo et al. 2023] Mi & Xu 2023 Xiangli et al. 2022 Xu et al., 2023). The emergence of Neural Radiance Fields (NeRF) (Mildnnhall et 2021 and 3D Gaussian Splatting (3DGS) (Kerbl
2023
ct al. (023) enables differentiable novel-view synthesis and real-time rendering (Luiten et al. 2024
Wu et al. 20023 Yang al. 2023cp. Recent studies have applied NeRF and 3DGS to unbounded
environments such as street views and urban areas (Turki et al. 2022) Tancik et al., 2022 Wang
ct al. 2023c Lin et al. 2024 Yan et al. 2023), broadening the application scenario.
However, achieving high-qualing 3D scene reconstruction remains challenging task. is observed existing works require precise camera poses for input and dense viewpoints for supervision (Lin) ct al. 2024: Chen al., 2024|] Guo et al.) 2023). Notably, public benchmark datasets such those referenced in (Geiger et al. 2012: Turki et al. 2022 Sun et al. 2020) often experience pose inaceu- racies and sparse viewpoints for boundaries. These issues are typically attributed to motion-induced vibrations and limitations in the positioning equipment used in vehicles or drones. Moreover, if one would like to mobile devices such smartphones and DSLR cameras to capture images for 3D reconstruction, obtaining accurate camera poses and dense viewpoints can be challenging and troublesome.
Although tools like COLMAP have been developed to lower the entry barrier for 3D reconstruction by providing pose estimation, the heavy optimization costs and camera matching failures limit their practicality (Schönberger & Frahm 2016 Schönberger et al. 2016b). Additionally, methods that incorporate pose estimation strategies for object reconstruction, struggle with broader scene appli- cations ( (Yang et al. 2023b) Jain et al.) 2021 Wang et al.) (022a] et al. 2023a). The limited number of viewpoints and the vast number of images in outdoor scenes further complicate the issue.
"qqual contribution. "orresponning author.

---

Published as a conference paper at ICLR 2025
To address the challenges of reconstructing large open scenes from uncalibrated images, this pa- per proposes GraphGS, a framework specifically tailored for large 3D open scene reconstruction. GraphGS utilizes 3DGS combined with proposed spatial prior-based scene structure estimation to enhance the speed and accuracy of pose estimation, even for image collections comprising thou- sands of images. During the process of scene structure estimation, the matching relationship of cameras is also recorded, which is represented as camera graph. The camera graph provides useful topology information for scene cameras and benefits for gaussian optimization.
GraphGS innovatively proposes to apply the constructed camera graph to guide the optimization process of large 3DGS scenes, through multi-view consistency constraints and adaptive sampling strategy. This ensures a more accurate 3D Gaussian point distribution. Besides, our proposed ap- proach prevents Gaussian points from overfitting to given sparse viewpoints, thereby enhancing re- construction quality. Furthermore, our adaptive sampling strategy reduces the number of iterations for camera graph nodes, significantly accelerating the reconstruction of large open 3DGS scenes and reducing training time by nearly 50%.
The main contributions of this paper are summarized as follows:
We introduce 3DGS in conjunction with spatial prior-based structure estimation method to efficiently and accurately estimate structures from uncalibrated images.
Through our proposed camera graph-guided 3D Gaussian optimization, GraphGS not only improves the reconstruction quality, but also greatly accelerates the reconstruction process.
With GraphGS framework, our method achieves state-of-the--e (SOTA) performance in several large benchmarks without using ground truth camera poses.
2
RELATED WORK
2.1 SCENE RECONSTRUCTION
The advent of NeRF (Mildenhall et al.) 202T) has ushered in golden age for 3D scene construction.
2022 Reiser cct al. 2023) and generalization YYu et al. 202Ib; Wang et al. 2021a; Chen et al.
Numerous studies have improved its efficiency (Hedman al. 2021 Müller et al.
2021 Mip-
NeRF (Baronn et al. 2021) and ZiR-NeRF (Barron al. 2023) have tackled aliasing issues, while InstantNGP (Müller et al. 2022) integrates grid pyramid technologies to optimize sub-volumes. UC-NeRF (Cheng et al. 2023) targets outdoor scenes, enhancing image consistency through color correction and pose refinement. StreetSurf (Guo et al., 2023) and EmerNeRF (Yang et al. 2023a) introduce novel approaches for multi-view reconstructions through disentanglement and self-guided learning. Concurrently, PVG (Chen et al. (024) utilizes 3DGS (Kerbl et al. (023) to adnance scene reconstruction with techniques like time-dependent transparency and scene- flow smoothing.
Furthermore, to extend reconstruction techniques to larger-scale scenes, methods like Block- NeRF (Tancik et al., 2022). Mega-NeRF (Turki et al. 2022), and Switch-NeRF (Mi & Xu 2023) strategy. Mega-NeRF clustered pixels based on 3D sampling dis- tances, Block-NeRt organized images into street blocks, and Switch-NeRF utilized a sparse network for large scene synthesis. These methods improved scalability and flexibility but faced limitations in real-time rendering of large outdoor environments. VastGaussian (Lin et al.) (202)) incorporated 3DGS to enhance detail presentation and rendering speed in large scenes. While these methods have advanced scene reconstruction, they typically rely on precise camera poses and initial data like
employed a siivid--nnd-onngeee
LiDAR, which be difficult to obtain in real-world applications, especially in expansive outdoor settings.
2.2 POSE OPTIMIZATION
To address issues with pose accuracy, many studies seek to bypass the slow and occasionally impre- cise COLMAP process by concurrently optimizing camera pose and scene representation using the original MLP-based NeRF (Wang et al. 2021b). such as GARF (Chng et al. 2022) and BARF (Lin al. 2024) focuses refining camera poses and 3D scenes
et al. 2021. TeantTTensRF(Cheeem using decomposed low-rank tensors. These methods have been proven effective in recovering ob- ject structures and poses from imperfect unknown camera positions, although their application
2

---

Published as a conference paper at ICLR 2025
Spatial Prior-based Scene
Graph Guided Gaussian Optimization
Marched
Unmatched
Adaptive Sampling
Mutt-stow
consistoncy
Image Set
3D Gaussians
3D Gaussians Scene
Camera Graph
Figure 1: Framework of the GraphGS method for efficient large 3D scene reconstruction. The process begins with spatial prior-based structure estimation, followed by sttree-based efficient or- ganization of initialization points. The camera graph is obtained at the end of structure estimation, which contains topology information scene camera. The information in camera graph will be further used for the following gaussian optimization.
broader scene reconstruction remains challenging. In 3DGS, COLMAP (Schönberger & Frahm 2016 Schönberger et al. 2016b) is utilized for pose reconstruction and generating sparse initial points Structure-from frotMMMooooo--ooiioon (SfM) (Schönberger & Frahm 2016). However, COLMAP's dense matching time increases exponentially with the number of images, and its success rate is limited, which poses significant challenges for outdoor scene reconstruction.
Building the discussions above, this paper is dedicated to proposing low-pose-requiremenn rapid,
3D reconstruction method suitable for both general and large scenes.
METHOD
As shown Fig. we firstly design spatial prior-based scene structure estimation to improve ficiency of SiM pipeline with the input of thousands of images. During the process, we record the matching information of cameras and form a camera graph. The graph contains topology informa- tion of scene cameras, which is suitable for guiding gaussian optimization to improve reconstruction quality and efficiency. In addition, to accelerate the training process and decreases GPU memory consumption, octree point initialization strategy is introduced to reduce initialization points.
3.1 SPATIAL PRIOR-BASER STRUCTURE ESTIMATION
In the reconstruction of 3DGS scenes from thousands to tens of thousands of images, rapidly and accurately obtaining camera poses represents significant challenge. Traditional Structure from Mo- tion (SfM) pipelineCCOLMAP (Schönberger & Frahm 2016;; Schönberger et al., 2016b)). typically require days to accurately process such extensive datasets. We locate this problem exhaustive matcher of COLMAP pipeline, which takes all possible image pairs for feature matching and causes time bottleneck. To address this issue, have developed an innovative spatial analysis framework for estimating scene structure. Initially, acquire approximate camera poses from pre-trained fast relative pose estimation models. Subsequently, utilizing concentric nearest neighbor pairing (Sec. 3.1.I and quadrant filter (Sec. B.1.2p. we filter and prioritize image pairs that are erucial for accurate camera pose estimation. This approach dramatically reduces computational load, facilitating swift and precise reconstructions that transform processes from days-long endeavors tasks completed within hours. Additionally, we have constructed a camera graph to guide the optimization process 3DGS, further enhancing the efficiency and precision of the reconstruction.
3.1.1 CONCENTRIC NEAREST NEIGHBOR PAIRING
Compared to selecting all possible image pairs, opting for sparser set can significantly accelerate the process of scene structure estimation. However, improper selections may lead to failures in matching cameras with the scene during the estimation phase. Our principle is, the distribution of selected image pairs should reflect both local and global structure. To handle this issue, we propose concentric nearest neighbor pairing. For camera c, Cj from N cameras altogether, define s(i.j)

---

Published as a conference paper at ICLR 2025
Concentric Nearest Neighbor Pairing
2D case of Quadrant Filter
Quadrant Filter Algorithm
{a: ey}
g
R g
010100
Query State Tablo
h=2
#=1 {
exclide
depends
Match Exclude
Figure 2: Illustration of spatial prior-based structure estimation, including two parts: Concentric Nearest Neighbor Pairing (left) and Quadrant Filter (right). For Concentric Nearest Neighbor Pair- ing, we first select O 's nearest r' cameras for matching to guarantee stability of local bundle adjust- ment, then select cameras from every h camera based distance order, forming series of concentric circles. For quadrant filter of 2D case, camera 0 is posed at the center of the coordinate system, pointing towards the y-axis. For other cameras (we show 12 cameras), the relative position and orientation to C contains 4 x 4 = 16 states.
as the sorted order from nearest to farthest of camera ej relative to camera Cr then the set of all matched pairs Sau is given by:
Sall = U (s(1)
U s(t))
U S (i) connection
(1)
neighbor
consentric
1=1
s(1)
boighbor = [(C, ((C,,c))lee ([((j))))) rr N,j
5(0
{((,, e)10 & (s(i,)) - (%hh++++ < w. 1,,
concentric
=
5(9
connection = -
neighhor concentrc are illustrated in Fig. (left) Specifically, we traverse all of the cameras from to N (supposing the current camera is c;) and calculate which camera ej (from
where the process of ((()
s(1)
another loop) should form matching pair with c;. Then we have the following steps: (1) For o₁ we
sort other cameras based on their distance to C₁ and add c₂ nearest cameras as g(()) neighbor (2) We
add w cameras from every h + cameras based on the same sorted order as 5(() cuncentric" (3) We add
C--1 in the main loop to match a as sli) connection
This approach allows for smooth transition from local to global matching pairs, ensuring sparsity that enhances the efficiency of feature matching. Notice there are repetitive elements (e.g. camera pair (c, c3) equivalent (c3, (1)). eliminate them via HashSet at the end. Also, implement this method via K-DTree ( (Bentley) 1975)) for highly efficient storage and retrieval of camera distance relationships save pair selection time.
When treating cameras as a graph node, and adding edges for every matched pair in Sall. an undirected graph will be formed, which we refer to as camera graph. To guarantee the success of scene structure estimation, camera graph should be connected graph. However, this may not be satisfied when we only have c(() se())
Remark.
neighhor> concentric and the proof is presented in Appendix A g(()
To address this issue, we add additional connection and pairs in this set will not be affected by the following quadrant filter strategy.
3.1.2 QUADRANT FILTER
In reconstructing 3DGS scenes from a large number of images, selecting appropriate image pairs to accelerate the estimation of scene structure is essential. Given camera pairs which have little or
4

---

Published as a conference paper at ICLR 2025
no view intersection with each other, it is inefficient to mark them matching pairs. Besides, these pairs introduce noise for global refinement. In this section we introduce quadrant filter strategy to filter these noise pairs.
For one coarse camera C we denote its global position as p() = [2,, Us. 2;] and orientation as ( [v my orientation in 3D space, their relative position and relative orientation have 8 states respectively, totally with 8 x 64 states. This already provides enough information to filter noise pairs.
(+) (.) Broadly speaking, for arbitrary two camera G;, Cy with varying position and
To record the relative position of camera Gj to Ci we apply binary encoding as follows:
B(
{sgn(ar m4), sgnlys yt), sgn(zz -z,)}
with the sign function sgn(x))
1, if 0, if S
sgn(x)
(3)
where Blis)) is three-digit binary number, representing 8 quadrants. For example, when =00,111, it represents binary number 011, corresponding the 3rd quadrant. (i.j) repre- sents the position of e; relative to meaning c; is posed on the origin of the 3D coordinate system, pointing towards z-axis.
To record the relative orientation, a direct way is to calculate rotation matrix to transform them in the same standard, and follow the same approach like recording relative position. However, this would be time-consuming since we have to solve system of linear equations for every potential camera pairs. To address this issue, we have proposition as follows:
by v their rel- alive orientation have states rexpectively, corresponding to quadrants 3D coordinate system. The quadrant of relative orientation can be directly calculated via one cross product and one inner product:
Proposition 1 Given two spacial vectors di = [v' .y vf d -
(i)
())
B (( (sgn(eeld2)dd sgm(ee) (d sgg(d"ddTT)
(4)
where Blidd) is three-digit binary number, representing quadrants. [d symmetric matrix of cross product:
represents the anti-
(i)
0/
(5)
and e, = [1,00, = 1,0 represent unit vector of x. y axis.
The full proof is provided in Appendix C
Proposition I provides an efficient way to record 8 states of relative orientation. After encoding both position and orientation, we concatenate them to get final pose binary encoding with 6-bit:
(.j)) = (i5)) + (6-5))
(6)
this enables us directly query predefined state table to filter the noise camera pairs. As shown in Fig. [ (right). 2D example, in this case, the relative position and relative orientation have states respectively (that is. 4 quadrants). For different camera e, relative to c, the e₁ (green) should be included in matching if it appears 56) of Equation I while the 02 (red) should excluded. For ca (yellow), it has slight view intersection with c.. for this state, we set strict/loose mode of the state table to handle like Cy to matching), our method can also filter more than 38% noise camera pairs for cameras with random poses. The detailed quadrant division of 3D coordinate system, full state table and analysis of strict/loose mode are all presented in Appendix
practice. Notice even in the loose mode (we include all the yellow cameras
5

---

Published as a conference paper at ICLR 2025
3.1.3 OCTREE POINT INITIALIZATION
The design of this module is based on the observation that, in the scene reconstruction process using 3DGS, not all initial points contribute equally to the ultimate quality of the reconstruction. By efficiently managing these points, computational efficiency is optimized without compromising reconstruction quality. Drawing inspiration from PlenOctrees (Yu et al.] 2021a), we employ an Octree (Meagher 1980) structure for spatial detail management. Nodes the octree are evaluated based on the Level of Detail (LOD) of the initial points, with those falling below a detail threshold T deemed minimally contributory to quality-heing being pruned to simplify the model.
3.2
GRAPH-GUIDED GAUSSIAN OPTIMIZATION
3.2.1 CAMERA GRAPH DEFINITION
Based the observation that viewpoint coverage in open scenes is relatively sparse compared to in- door scenes or object reconstruction, supervision from a single viewpoint can lead to Gaussian points overfitting near specific cameras. This results in inaccurate Gaussian distribution and learning predoninantly from a single image rather than the entire scene. To address these challenges and improve the efficiency, accuracy, and robustness of 3DGS reconstruction in open scenes, we have specifically designed weighted undirected camera graph based on structured estimation results. The construction of this graph adheres to the following rules: 1) Each node corresponds camera's is rotation matrix R and translation matrix T; 2) Edges are formed based on the camera pairs selected in Section 3.T We aim to use this meticulously designed graph structure to precisely reflect the spatial relationships between cameras, thereby effectively guiding the optimization process of 3DGS open scenes.
view
3.2.2 GRAPH-GUIDED MULTI-VIEW CONSISTENCY CONSTRAINT
We know that adjacent multi-view scenes with mini- mal directional differences should exhibit nearly identi- cal photometric values (Kloukiniotis al. 2022). Con- sequently, we designed edge weight (i, of camera graph to measure this directional differences as follows:
0
7 0
-k|lp(9-p00
(i. =
(7)
det
034
1 1 d(
0St.
8gc
where p00.p5 and d(0) ,(i) are refined camera position and orientation from rotation and translation matrix R T. Take Fig. 3 for example, supposing there are 3 edges (3, 4), (3, 6). (3, 7) connecting to camera node 3. When calculating the edge weights, Eq. will take relative dis- tance d36, d37. dss and relative orientation O36, O177, Os4 into consideration, while the edge with smaller distance and smaller angle obtains larger weight.
6
du dyr des
05r 030 0st
(.(3,4) 4) (3 7) w.(3,6) 6)
Figure 3: Example of edge weights.
Based on the edge weights, we designed a multi-view consistency photometric loss Loous enhance the robust- ness of 3DGS optimization process as follows:
Leens = A =1 -E JJ P
(8)
where Tp)p) represents the intensity or color pixel p in the image from camera The matrices R5; and T5; describe the rotation and translation from camera i to camera j, respectively, while K, and K, are the intrinsic matrices, and is an adjustment factor. To obtain camera j given camera i, we traverse all edges of camera i and select the camera with the maximum edge weight camera as target camera j.
Counterintuitively, while target camera j may provide rendered results that significantly worse than the ground truth images for supervision, they enhance the reconstruction outcome. We believe this may act similar to data augmentation method, helping prevent overfitting.

---

Published as a conference paper at ICLR 2025
Waymo
KITTI
Method
FPS
PSNN
SSIM f LPIPS
FPS 1
PSNN 1
SSIM
LPIPS
Mip-NeRR 360 (Barron al.
2022
0.042
22.42
0.698
0.471
0.053
20.68
0.650
0.480
S-NeRF (Xic aL. 2023
0.001
19.22
0.515
0.400
0.008
18.71
0.606
0.352
StreetSuri Guo al. 2023
0.097
23.78
0.822
0.401
0.037
22.48
0.763
0.304
Zip-NeRF Barton et al. 2023
0,500
26.21
0.815
0.389
0.610
21.41
0.665
0.470
UC-NeRF Cheng et al 2023
0,032
26.72
0.800
0.375
0.051
24.05
0.721
0.400
BARF [Lin et al. 2
0.041
9.07
0.235
1.021
0.071
10.68
0.250
0.990
SPARF (et
20233)
UP-NeRF (ct al.
20233
0.120
26.16
0.876
0.375
EmerNeRF Yang et al.
2023a
0.043
25.92
0.763
0.384
0.28
25.24
0.801
0.237
3DGS (Kerbi a.l 2023
63
25.08
0.822
0.319
125
19.54
0.776
0.224
PVG (Chen 2024
50
28.11
0.849
0.279
59
26.63
0.885
0.127
Ours
29.43
0.899
0.217
79
26.98
0.887
0.157
Table 1: Quantitative comparison of novel view synthesis results on the Waymo and KITTI. Our method demonstrates a competitive edge in rendering quality with higher FPS, PSNR, SSIM, and lower LPIPS scores, affirming its efficacy in synthesizing realistic views in comparison to the current state-of-the-an methods.
3.2.3 ADAPTIVE SAMPLING OPTIMIZATION
We propose an adaptive sampling strategy that dynamically adjusts sampling rates during optimiza- tion based node importance in the graph. For nodes with less overlap with other viewpoints, sampling frequency is reduced according to their weights. To assess the importance of each node within the graph, devise specific importance weights. We find that performing fewer iterations on certain peripheral nodes does not significantly degrade the overall quality of reconstruction and helps prevent overfitting due to insufficient supervision signals. The design of node weights primar- ily considers two criteria: 1) Degree centrality, which calculates the number of neighbors connected each camera node; 2) Betweenness centrality, assessing the importance of each node across all shortest paths, where nodes with higher centrality are considered more significant. Degree centrality focuses on the activity level of nodes, while betweenness centrality is concerned with the control node exerts or its role bridge. In our experiments, we found that using only betweenness centrality yielded better results, so we design node weight (i)(i) based on Betweenness centrality:
(i) Jzh
(1) = Σ
(9)
where oyk represents the number of shortest path between node j.k of camera graph. (1) repre- sents the number of shortest path passing node i between node j.k. After getting node weights, we set probability function P(i) for camera o; during iteration:
P(1) = w. (M)))Mx((,,,))
(10)
where the denominator represents the maximum value of all node weights. The function P(i) represents the probability of view camera participating in gaussian optimization. This design is noteworthy as significantly reduces the number of optimization iterations, which accelerates the reconstruction process. Despite this reduction, it effectively enhances the geometric representation of Gaussian points, substantially improving the overall quality of the reconstruction.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets and Metries, We evaluate our method on three datasets: Waymo (Sun al. 2020 KITTI (Geiger et al. 2012). and Mill-19, which includes large scenes like Buildings and Rubble Turki al. 2022). For Waymo, across 32 scenes, totaling about 600 images per scene block. For KITTI, each scene block contains approximately 100 images. Consistent with prior research, we select out of every eight images
images from three cameras (front, front-leff, front-right)
7

---

Published as a conference paper at ICLR 2025
(a) Waymo
(b)
StreetSurf
3DGS
Figure 4: Qualitative comparison of novel view synthesis results on the Waymo and KITTI, show- casing the Ground Truth alongside results from our GraphGS method, StreetSurf (Guo et al. 2023) PVG (Chen et al. 2024), and 3DGS (Kerbl et al. (0033) for comprehensive evaluation. Our approach yields closer fidelity lity to the Ground Truth, highlighting the effectiveness of our reconstruction method in various urban scene complexities.
Building
Rubble
Method
PSNR
SSIM t LPIPS
PSNR
SSIM f LPIPS
Mega-NeRF Tuurk et al.
2022
20.93
0.547
0.349
24.05
0.553
0.373
Switch-NeRF (Mi Xu 2023
21.54
0.579
0.294
24.31
0.562
0.329
3DGS (Kerhl ei al 20233
23.01
0.769
0.164
26.78
0,800
0.161
VastGaussian (Lin et al.
2024)
23.50
0.804
0.130
26.92
0.823
0.132
Ours
26.60
0.854
0.163
27.03
0.869
0.185
Table 2: Quantitative comparison of novel view synthesis on the Mill 19 large scene dataset. for testing, using the remainder for training. Our method, designed for scenarios lacking pose data, does not use ground truth (GT) poses from the datasets.
We compare our method on the Waymo and KITTI datasets against various NeRF-based methods (UC-NeRF (Cheng et al. 2023). Mip-NeRF 360 (Barron et al., 2022), Zip-NeRt (Barron et al, 2023). StreetSurf (Guo et al. 2003),, EmerNeRF (Yang et 20023a). SNeRF (Xie et 2023)) and methods based on 3DGS (3DGS (Kerbl et al. 2023), PVG (Chen et al. (202)). We specifically eval- uate against both pose optimization (UC-NeRF (Cheng al. 2023) ) and pose-free reconstruction methods (c.g.. BARF (Lin et al.) (2021). SPARF (ct al. 2023b), UP- Ne-N (et al. (023a)). For the large Mill-19 dataset, we compare with Mega-NeRF (Turki al. (022). Switch-NeRF (Mi Xu) 2023), and VastGaussian (Lin et al. 2024), segmenting scenes for 3DGS due to memory limits. For some methods, we use published results for comparisons.
Implementation. Experiments were conducted using an NVIDIA RTX 3090 GPU and an AMD EPYC 7542 CPU. For image sets lacking pose or sequence information, we use a pre-trained model (Wang et al. 2024) to estimate relative poses in about 0.01 seconds per pair. Although not highly precise, it effectively outlines the coarse distribution of the images. We then apply our scene struc- ture strategy to estimate scene structures, generate camera graphs, and calculate consistency and importance weights. Additionally, we optimize memory and training efficiency by using an octree to prune redundant points from approximately 300k to 100k.
For the guantitative comparison of our Spatial based Structure Estimation with Colmap, the Colmap setup included vocabulary tree containing 256K visual words, built using the Flickr100k dataset (available on the COLMAP project page). Further details and training specifics available in the Appendix.
8

---

Published as a conference paper at ICLR 2025
(a) Building
(b) Rubble
Ground Truth
Ours
MeFa-NeRF
Switch-NeRF
3DGS
Figure 5: Qualitative comparison of novel view synthesis in the Mill 19 large scene dataset (Turki
et al., 2022). showcasing the Ground Truth alongside the results from our method and other state- of-the-art methods including Mega-NeRF (Turki et al. 2022). Switch-NeRF (Mi & Xu 2023) and
3DGS (Kerbl et al. 2023).
Matching
BA Time
Strategy
PSNR
SSIM
LPIPS $
Time
COLMAP (Ex)
62 min
140 min
30.05
0.89
0.23
0.6k
COLMAP (Vo)
50
104 min
29.02
0.88
0.26
Ours
3 min
20 min
30.18
0.90
0.24
COLMAP (Ex)
>2
2k
COLMAP (Vo)
>24 h
Ours
12 min
194 min
26.60
0.85
0.16
Table 3: Quantitative comparison of our Spatial Prior-based Structure Estimation for 0.6k and 2k images. 0.6k, COLMAP-Exhaustive (Ex) and COLMAP-Vocabll
(Vo) are benchmarked. At the 2k scene, COLMAP fails consistently, with SIM processing times exceeding 24 hours, yielding no results. Our approach demonstrates significantly faster while maintaining comparable accuracy.
4.2 EXPERIMENTAL RESULTS
Waymo, KITTI and Mill-19. Table T and Figure show that our method outperforms others in both quantitative and qualitative measures, driven by precise structure estimation and graph-guided optimization. It accurately captures fine details such as tree branches without noticcable blurring, while maintaining superior shape, geometry, and color fidelity. Notably, BARF does not converge on two datasets, and SPARF, is suitable only for sparse image sets. UP-NeRF also struggles with convergence the KITTI dataset. For large scenes like Mill-19, presented in Table and Figure [5] our method continues to demonstrate clear edges and detailed clarity in elements like rubble, grass, and stones.
Spatial Prior-based Structure Estimation. We demonstrate the efficiency of our spatial prior- based method through Concentric Nearest Neighbor Pairing (CNNP and Quadrant Filter (QF 3.1.2). Table 3 shows our method increases matching speed 20x and reduces bundle adjustment time compared traditional approaches like exhaustive or vocabtree (Schönberger et al. 201 matching. For large datasets, where naive 3DGS methods using COLMAP are impractical due to long Structure from Motion times (typically >24 hours and always fails), our method completes structure estimation in just few hours and achieves higher-quality reconstructions by accurately determining camera poses and avoiding invalid camera pairs.
Method
PSNR 1
SSIM
LPIPS
w/o QF
29.14
0.89
0.25
w/o CNNP
24.33
0.82
0.33
w/o Structure Estimation
24.77
0.803
0.444
w/o Multi-view Consistency
29.42
0.834
0.297
GraphGGSOOrs)
30.36
0.891
0.267
Table 4: Quantitative Comparison of Ablation Experiments for Submodules.

---

Published as a conference paper at ICLR 2025
w/ Adaptive Sampling
w/o Adaptive Sampling
Adaptive Sempling
wio Adeptive Sempling
Figure 6: Ablation Experiments Adaptive Sampling Optimization.
This figure highlights the
impact excluding our adaptive sampling optimization.
Training Time
Method
Iterations
PSNR 1 SSIM LPIPS L
w/o Adaptive Sampling
30000
54 min
28.81
0.884
0.253
W Adaptive Sampling
16830
28 min
30.36
0.891
0.267
Table 5: Quantitative comparison of Ablation Experiments Adaptive Sampling Optimization.
4.3 ABLATION STUDY
We conducted ablation studies to assess the contributions of our method's s components to the recon- struction process. This included evaluations of structural estimation and its sub-modules CNNP and QF, as well as the effects of graph-guided multi-view consistency constraint and adaptive sampling optimization on reconstruction quality.
Table Table:Jonfirmss confirms the impact of each module on reconstruction quality. On the Waymo dataset, omit- ting the QF leads to reduced accuracy due to the inclusion of non-intersecting camera pairs in pose estimation, affecting camera pose precision. Similarly, excluding the CNNP module results in notable decline in quality because fails to accurately capture global poses through local matching. Integrating structure estimation significantly enhances reconstruction quality by precisely estimat- ing the global scene structure. Additionally, the inclusion of graph-guided multi-view consistency constraints further improves outcomes by maintaining consistency across multiple views.
Table demonstrates that the adaptive sampling optimization method significantly reduces opti- mization time and improves the quality of reconstruction. As illustrated in Figure [ this approach mitigates overfitting to specific viewpoints by optimizing the distribution of Gaussian points. This enhancement not only prevents the occlusion of other perspectives but also reduces severe distortions within the reconstructed geometry. For more ablation experiments, see Appendix
CONCLUSION
In conclusion, through our proposed structure estimation and graph-guided optimization methods, GraphGS can achieve high-quality, rapid reconstruction of large scenes from image sets without ground truth poses. This capability particularly meaningful for multimedia applications such as virtual reality, gaming, and the metaverse. GraphGS not only meets the growing demand for fast and reliable scene reconstruction but also provides a scalable and accessible solution.
The absence of camera data and the anisotropic nature of the method limit the comprehensive capture and reconstruction of reverse scenes. Additionally, the reliance on feature points may cause the matching strategy to fail randomly, particularly in scenes with numerous dy- namic objects. The effects on distant views and the blurring caused by large Gaussian points in certain shots still need improvement and resolution. For more discussion, see Appendix G
Linitations.
ACKNOWLEDDMENT
This research is supported by the National Natural Science Foundation of China (No. 62406267), Tencent Rhino-Bird Focused Research Program, Guangzhou-HKUUT gram (Grant No.2025A0313956), the Guangzhou Municipal Science and Technology Project (No. 2025404J4070,, the Guangzhou Municipal Education Project (No. 20243 2024312122) and Education Bureau of Guangzhou Municipality.
Joint Funding Pro-
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualle and
Pratul P. Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields, 2021.
Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In 2022 TEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5460-5469, 2022. doi: 1091CC///28 2022.
00539.
Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. ICCV, 2023.
Jon Louis Bentley. Multidimensional binary search trees used for associative searching. Communi- cations of the ACM, 18(9):509-517, 1975.
Eric Brachmann, Jamie Wynn, Shuai Chen, Tommaso Cavallari, Áron Monszpart, Daniyar Tur- mukhambetov, and Victor Adrian Prisacariu. Scene coordinate reconstruction: Posing of image collections via incremental learning of a relocalizer, 2024.
abs/
1404.14335
Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, and Hao Su. Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo. arXiv preprint vrXiv:2103.15595 2021.
Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, and Li Zhang. Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering, 2024.
Bo- Yu Cheng, Wei-Chen Chiu, and Yu-Lun Liu. Improving robustness for joint optimization of camera poses and decomposed low-rank tensorial radiance fields, 2024.
Kai Cheng, Xiaoxiao Long, Wei Yin, Jin Wang, Zhiqiang Wu, Yuexin Ma, Kaixuan Wang, Xiaozhi Chen, and Xuejin Chen. Uc-nerf: Neural radiance field for caler-cal multi-view cameras in autonomous driving, 2023.
Shin-Fang Chng. Sameera Ramasinghe, Jamie Sherrah, and Simon Lucey. Gaussian activated neural radiance fields for high fidelity reconstruction and pose estimation. The European Conference on Computer Vision: ECCV, 2022.
Injae Kim al. Up-nerf: Unconstrained pose-prior-free neural radiance fields, 2023a.
Prune Truong et al. Sparf: Neural radiance fields from sparse and noisy poses, 2023b.
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are ready for autonomous driving? the kitti vision benchmark suite. Conference on Computer Vision and Pattern Recognition (CVPR), 2012.
Jianfei Guo, Nianchen Deng, Xinyang Li, Yeqi Bai, Botian Shi, Chiyu Wang, Chenjing Ding, Dongliang Wang, and Yikang Li. Streetsurf: Extending multi-view implicit surface reconstruction to street views. arXiv preprint
2023.
Peter Hedman, Pratul P. Srinivasan, Ben Mildenhall, Jonathan T. Barron, and Paul Debevec. Baking neural radiance fields for real-time view synthesis. ICCV, 2021.
Ajay Jain, Matthew Tancik, and Pieter Abbeel. Putting nerf on a diet: Semantically consistent few-shot view synthesis, 2021.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering, 2023.
Andreas Kloukiniotis, Andreas Papandreou, Christos Anagnostopoulos, Aris Lalos, Petros Kap- salas, Duong- Van Nguyen, and Konstantinos Moustakas. Carlascenes: A synthetic dataset for odometry in autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 4520-4528, June 2022.
11

---

Published as a conference paper at ICLR 2025
Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey. Barf: Bundle-adjusting neural radiance fields, 2021.
Jiaqi Lin, Zhihao Li, Xiao Tang, Jianzhuang Liu, Shiyong Liu, Jiayue Liu, Yangdi Lu, Xiaofei Wu, Songeen Xu, Youliang Yan, and Wenming Yang. Vastgaussian: Vast 3d gaussians for large scene reconstruction, 2024.
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In 3DV, 2024.
Donald Meagher. Octree encoding: A new technique for the representation, manipulation and dis- play of arbitrary 3-d objects by computer, 10 1980.
Zhenxing Mi and Dan Xu. Switch-nerf: Learning scene decomposition with mixture of experts for large-scale neural radiance fields. In International Conference Learning Representations (ICLR), 2023. URL
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Commun. ACM, 65(1):99-106 106, dec 2021. ISSN 0001-0782. doi: 10.1145/3503250, URL https:://ol.
10. 10.1145/3503250
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions Graphics, pp. 1-15, Jul 2022. doi: 10.1 10.1445/528223333127,, URL httppp//oo......ooo//./111/////3
3530127
Linfei Pan, Dániel Baráth, Mare Pollefeys, and Johannes L. Schönberger. Global structure-from motion revisited, 2024. BtR.httss://arxt wwtp.rg/////....// 20219.
Christian Reiser, Richard Szeliski, Dor Verbin, Pratul P. Srinivasan, Ben Mildenhall, Andreas Geiger, Jonathan T. Barron, and Peter Hedman. Merf: Memory-efficient radiance fields for real- time view synthesis in unbounded scenes. SIGGRAPH, 2023.
Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-mottie revisited. In Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2016.
Johannes Lutz Schönberger, True Price, Torsten Sattler, Jan- Michael Frahm, and Mare Pollefeys. A vote-and-verify strategy for fast spatial verification in image retrieval. In Asian Conference on Computer Vision (ACCV), 201
Johannes Lutz Schönberger, Enliang Zheng, Mare Pollefeys, and Jan-Michael Frahm. Pixelwise view selection for unstructured multi-view stereo. In European Conference on Computer Vision (ECCV), 2016b.
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In 2020 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), Jun 2020. doi: 10. 10110/////4400002000000333 URL. http://dxx doi www.ooog//..... 1ttgg/0.110000 w1t1//1//////22 600 2020 00252
Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben Mildenhall, Pratul Srinivasan, Jonathan Barron, and Henrik Kretzschmar. Block-NeRF: Scalable large scene neural view synthesis. arXiv, 2022.
Haithem Turki, Deva Ramanan, and Mahadev Satyanarayanan. Mega-nerf: Scalable construction of large-scale nerfs for virtual fly-throughs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12922-12931, June 2022.
Guangcong Wang, Zhaoxi Chen, Chen Change Loy, and Ziwei Liu. Sparsenerf: Distilling depth ranking for few-shot novel view synthesis, 2023a.
12

---

Published as a conference paper at ICLR 2025
Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Visual geometry grounded deep structure from motion, 2023b. URL httpttt//xxxi...oot / 04563
Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srinivasan, Howard Zhou, Jonathan T. Bar- ron, Ricardo Martin-Brualla, Noah Snavely, and Thomas Funkhouser. Ibrnet: Learning multi- view image-based rendering. In CVPR, 2021a.
Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Ge- ometric 3d vision made easy. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20697-20709, 2024.
Zian Wang, Tianchang Shen, Jun Gao, Shengyu Huang, Jacob Munkberg, Jon Hasselgren, Zan Gojeic, Wenzheng Chen, and Sanja Fidler. Neural fields meet explicit geometric representations for inverse rendering of urban scenes. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2023c.
Zirui Wang, Shangzhe Wu, Weidi Xie, Min Chen, and Victor Adrian Prisacariu. NeRF--; Neural radiance fields without known camera parameters. arXiv preprint (tti::2002.000666 2021b.
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Wang Xinggang. 4d gaussian splatting for real-time dynamic scene rendering, arXiv preprint arXiv:2310.08522 2023.
Yuanbo Xiangli, Linning Xu, Xingang Pan, Nanxuan Zhao, Anyi Rao, Christian Theobalt, Bo Dai, and Dahua Lin. Bungeenerf: Progressive neural radiance field for extreme multi-scale scene rendering. In European conference on computer vision, pp. 102.122. Springer, 2022.
Ziyang Xie, Junge Zhang, Wenye Li, Feihu Zhang, and Li Zhang. S-nerf: Neural radiance fields for street views. Mar 2023.
Linning Xu, Yuanbo Xiangli, Sida Peng, Xingang Pan, Nanxuan Zhao, Christian Theobalt, Bo Dai, and Dahua Lin. Grid-guided rodd-guidd neural radiance fields for large urban scenes, 2023.
Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians for modeling dynamic urban scenes. 2023.
Jiawei Yang. Boris Ivanovic, Or Litany, Xinshuo Weng, Seung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, and Yue Wang. Emernerf: Emergent spatial-temporal scene decomposition via self supervisen, 2023a.
Jiawei Yang. Marco Pavone, and Yue Wang. Freenerf: Improving few-shot neural rendering with free frequency regularization, 2023b.
Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. De- formable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. arXiv preprint arXiv:2309.13101, 202cc.
Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. Plenoctrees for real-time rendering of neural radiance fields, 2021a.
Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. PixeINeRF: Neural radiance fields from one few images. In CVPR, 2021b.
13

---

Published as a conference paper at ICLR 2025
Figure 7: Counter example for special case of r.w, In the special case, 2, h = 3, W = 1. This means for every camera, we will only add its nearest camera and farthest camera as graph edge. We list one counter example with N 8. In the counter example, the red nodes and yellow nodes are disconnected, while all the nodes follow the matching algorithm.
A APPENDIX: PROOF OF GRAPH CONNECTIVITY
For Equation. if we remove s(i)
connection the Sall would be as follows:
N
Sau =
U
(S(())
neighhor U s(()
concenttri))
(1I)
==1
s(i neighhor = ((((,,))s((.))) Sr,j 1,2 N, +
s(i)
((q, c,)0 (s(i,)) - (k w.j = 1, N. +
concentric
In the scenario, the camera graph may not be connected graph. It would be fairly troublesome to give direct proof. However, when considering proof by contradiction, one counter example for one special case of r r,,, is enough.
Proof:
We choose the special case when 2, h - w which means for every camera node in graph, we only add edges to its nearest camera and farthest camera. In this scenario, one counter example is illustrated in Fig. 7 The graph nodes represent cameras while the position of graph nodes also represents the actual position of cameras. For example, camera node 1 connected camera 2 and camera since they are the nearest and farthest cameras camera 1 respectively. After the edge adding algorithm done, the camera 2, 7, 8 and 3, 4, 4.5, 6 form two different connected components, while the entire camera graph not a connected graph.
B APPENDIX: STATE TABLE OF QUADRANT FILTER
In this part, we introduce quadrant division and the state table for filtering noise camera pairs as described in Section 3.1.2 of the main paper.
Quadrant Division. As illustrated in Figure g adopt the coordinate system of OpenCV, which is right-handed system. Within this camera coordinate system, it is assumed that the camera is facing +2 axis. We adhere to this assumption when calculating the relative orientation of cameras.
State Table. For state table, we identify 64 potential configurations describing the relative relation- ship between two cameras. For the conciseness of representation, we only illustrate the successful matching pairs, shown in Table The "position quadrant" column lists all possible quadrants of the relative position between cameras o; and Cj. while the "orientation quadrant" column indicates the quadrant of relative orientation that enables successful matching. For instance, if c; is positioned in the right upper rear quadrant of c; (quadrant in Fig. g), only a facing orientation in quadrant 7 (relative orientation) satisfies the matching condition under strict mode. Practically, employ Loose Mode for vehicle data due to the elongated and narrow camera track. For other situations, we apply Striet Mode.
14

---

Published as a conference paper at ICLR 2025
2
4
7
B
5
Quadrant
y
1 $
Quadrant number
Figure 8: Illustration of quadrant division: When calculating the relative position of cameras and Cj. we assume that camera & is positioned at the origin. In determining the relative direction, we assume that the orientation of camera is facing the + axis.
Orientation Quadrant
Position Quadrant
Striet Mode
Loose Mode
1
7
2, 3, 6, 2, 3,
2
7, 8
4,
7,
3
5, 6
1, 3, 6,
4
6
2, 6,
5
3
2, 2,3,
6.
6
3,4
2, 3,
4,
6, 7,8
7
1,2
1,
5, 6,
8
2
2, 6, 7
Table 6: State Table Illustration.
Probability of filter under Strict/Loose Mode. the situation of random camera position and orientation, the strict and loose mode of state table have different probabilities to filter irrelevant camera pairs, which can be calculated theoretically to measure efficiency. For Strict mode, the probability of filter is
1 x
1
13
Patrict
x
x
R 81.3%
(12)
8
16
For Loose mode, the probability of filter is
2 1
6
P Pleuse
x
37.5%
(13)
8
16
It is notable even in loose mode, the probability of filter is still high number, meaning more than 1/3 noise camera pairs are filtered. This greatly reduced the burden of calculation in the following structure pipeline compared to the situation without quadrant filter.
C APPENDIX: PROOF OF PROPOSITION
In the Proposition of the main paper, we utilize the internal relations cross product and inner product within right-handed 3D coordinate system to calculate the relative orientation without the need to solve the linear system:
(t,)) d
{sgn(e, (2 (())), sgn(eyld (dTG)) sgn(d (
(14)
where [ represents the anti-symmetric matrix of cross product:
(8)
02
y
0
U
(15)
15

---

Published as a conference paper at ICLR 2025
Step 1: Translation
x d
(
Step 2: Truncation
3, 7
2,
3,7
2, 6
4, a
5
4, 8
1,5
[0 E [
102 oz
Step 3: Rotation
d6
3,7 7
2,
of d 200
4, 8
Quadrant
Figure 9: Imagined Transform Illustration: Throughout the entire process, the relative orientation of d() and d is maintained without additional calculation.
and c, 01, c, j0, 1, 0] represent unit vector of x; axis. However, its mathematical principles are omitted due to limited space. Here we will explain the rationality of our method.
Proof:
For cameras 6; and with absolute orientations (6) and d((), we their cross product d(+) xdU0 [ox by vx] to reflect the relative orientation to some extent. We envision transforming them into a standard coordinate system illustrated in Figure & through several steps:
Step 1: Translate them the origin of the standard coordinate, This process does not alter the cross product or their relative orientation.
Step 2: From an overhead view of the standard coordinate (where the y-axis points into the page), we note that the plane formed by d(() and d(ó) divides the x-2 plane into two parts. We then transform () and d(i) such that d( d(u) changes from [ og v [0: vx 0), while maintaining their relative orientation. This transformation feasible because the linear system we aim to solve is not full rank. After this transformation, the plane of d and d'i coincides with the 2-1 plane.
Step 3: We rotate ( and d'i) around the z-axis so that d(() aligns with the z-axis.
After the aforementioned steps, as depicted in Figure we have successfully translated d(() and d(i) from an arbitrary situation the standard coordinate with d(t) pointing toward the axis. In this
16

---

Published as a conference paper at ICLR 2025
3,
1,2
y
y
7, B
5, 6
1/
y
[12 by
plane of d
Quadrant
Figure 10: Four situations for the plane of did), which depends on the sign of vg ng
process, transformation matrix is required, while vx og and the relative orientation of (()) and (() are preserved. This sets the stage for further analysis.
We then consider all potential positions of d(5), As shown in Figure d()) lies on green plane that can rotate around the z-axis. Viewing the coordinate system from the front, where the z-axis points into the page, and due to the constraints the right-handed rule, the potential positions of d(d) are limited half of the green plane, contingent upon one of four scenarios based on [1 1yy 0). Thus, we can limit the relative orientation of d(i) to two the eight quadrants through u and v.
Finally, we simply calculate the inner product of (()) and d() to determine the relative orientation along the z-axis (either quadrant 2, 3, 6, 7 or 1, 5, 8). Based on the derivation above, the final B(135) can be rewritten in a more direct form without matrix:
(+)) B B
{sgn ( sgg( sg db0)}
(16)
The Equation 16 and Equation text for the consistency context.
14 are actually equivalent, while we keep Equation 14 in the main
D APPENDIX: ADDITIONAL EXPERIMENTAL RESULTS
Method
Time (min)
PSNR
SSIM
LPIPS
ACEZero
10
17.50
0.725
0.354
PixSfM
130
28.75
0.847
0.366
GLOMAP
28
27.65
0.824
0.388
COLMAP
154
29.14
0.89
0.250
Ours
23
30.36
0.891
0.267
Table 7: Quantitative Comparison of SfM on Waymo dataset.
As shown in Table D. our method demonstrates significant improvements over other state-of the-art art SfM systems in terms of processing time and accuracy metrics. Despite the emergence of numerous
17

---

Published as a conference paper at ICLR 2025
Initial Points Number
Training Time
Method
PSNR
SSIM 1
LPIPS
w/o Octree Point Initialization
245820
54 min
29.74
0.841
0.291
Octree Point
100000
36 min
29.75
0.843
0.292
Initialization
Table 8: Quantitative comparison of Ablation Experiments on Octree Point Initialization.
Ground Truth
PVG
Figure 11: Qualitative comparison of novel view synthesis with added random Gaussian noise to simulate imprecise pose data. Our method retains clarity and definition, while alternative approaches like PVG (Chen et al. 2024) and 3DGS (Wu al. 2023) show distortions under similar conditions.
effective Structure from Motion (SfM) methods, COLMAP continues to be widely recognized for its robustness and accuracy the community, thereby serving benchmark for comparisons among incremental SfM (Pan et al., [024]. Notably, VGGSIM (Wang et al.) 2023b) was excluded from detailed comparisons due to memory limitations when processing large image datasets. In contrast, ACEZero Brachmann al. (2024), although faster, faces convergence issues in larger datasets. Our method not only surpasses others in terms of results from novel view synthesis but also maintains faster processing speed.
E APPENDIX: ADDITIONAL ABLATION STUDIES
Impact Octree Initialization on reconstruction quality and training time, Quantitative results in Table 8 show that our octree point initialization strategy cuts training time in half without com- promising reconstruction quality. Further reductions in initial points did not significantly decrease training times, likely due to the densification and point operations.
Camera Disturbance. To simulate real-world conditions where camera poses are uncertain, we introduce disturbances by adding random noise from 0 to 0.3 radians camera orientations and positions in a selected Waymo scene. We assess the robustness of our methods by comparing them with 3DGS and PVG under these conditions.
Our method remains robust against disturbances in camera poses, common in real-world scenarios like road bumps. Figure 17 and Table show that traditional 3DGS methods degrade significantly
18

---

Published as a conference paper at ICLR 2025
Method
PSNR f
SSIM 1
LPIPS
3DGS (R#0.3 & T
12.676
0,510
0.613
PVG (R*0.3 & T*033)
12.695
0.512
0.615
Our (R$0.3 & T033)
28.53
0.886
0.221
Table Quantitative comparison of novel view synthesis results on Waymo dataset with additional noise. R*0.3 and T03 represent the addition of 0.3 units of rotation and translation noise, respec- tively.
Images
Method
Pairing Time
Matching Time
BA Time
PSNR
SSIM
LPIPS
0.6
Ours
0.2 min
3 min
20 min
30.18
0.90
0.24
0.6
Frusta
2 min
45 min
86 min
30.15
0.89
0.24
2
Ours
2.1 min
12 min
194 min
26.60
0.85
0.16
2
Frusta
20 min
>24
> h
Table 10: Comparation of our Concentric Nearest Neighbor Pairing (CNNP), Quadrant Fileter (QF)
and naive frusta method on Waymo (0.6 k) and Mill 19 (2 k) datasets.
under such disturbances, with heavy blurring that obscures maintains quality by calibrating camera poses using key camera pairs.
objects. In contrast, approach
Comparison with naive overlapping frusta method We conduct experiment with the naive frusta method and compare it to our Concentric Nearest Neighbor Pairing (CNNP), Quadrant Fileter (QF) methods when selecting matched pairs. We implement the frusta algorithm with the following steps:
The intrinsic matrix of camera is obtained to calculate the near-far plane.
The frusta cameras is transformed to world coordinate system.
We check the intersection of their bounding box to judge the view intersection.
As shown in Tab. and further analysis are presented as follows:
Universality.
Notably, our matching strategy only needs the position and orientation of coarse cameras, i.e., only the 3-rd, 4-nd colume extrinsic matrix. Moreover, we do not need the information of intrinsic matrix. This indicates the universality of our method.
Efficiency. As shown in QF section and Appendix C calculate the relative orienta- tion with only inner product and cross-product based on fully convincing mathematical proof. As shown column of pairing time, this way greatly improves our efficiency com- pared to frusta method, which contains time-consuming coordinate transform and intersec- tion checking process.
Reconstruction Quality. In terms of reconstruction quality, there no obvious difference between our method and frusta, since pose accuracy is well-solved in BA procedure. Our approach offers better compatibility in cases where initial poses are poor.
Reason why frusta failed. Though the naive frusta method sounds reasonable, however, in practice, especially in the driving scenario with long and narrow camera track, the frusta method cannot filter image pairs efficiently since all of the camera orientation are extremely similar.
F APPENDIX: EXPERIMENTAL DETAILS
Experiments were conducted using an NVIDIA RTX 3090 GPU and an AMD EPYC 7542 CPU. The Waymo includes 600 images from three viewpoints (left, center, right), covering 599 image
19

---

Published as a conference paper at ICLR 2025
pairs. The Kitti includes 100 images with 99 image pairs. The Mill 19 comprises 2000 images with 1999 pairs included.
For the CNNP configuration, the settings of r 5, 20, setting, approximately ~ 18k/Cou pairs will selected for 0.6k images, and 200k//20n pairs will be selected for 2k images. The setting candidates for each camera C₁. The configuration h = 20, w = 1 means that one camera chosen match for c; out of every 20 cameras based on proximity.
were employed. Under this
designates the five nearest cameras matching
To optimize multi-view consistency and balance the coefficients, we set the coefficient to 0.07, which was empirically found be optimal in our experiments. Additionally, adopted graph- guided optimization setup, where the sampling probability is determined by the weights of the graph nodes. We set the minimum sampling probability at 0.5 to ensure that nodes with lower weights are not overlooked.
G APPENDIX: DISCUSSION OF LIMITATIONS AND CHALLENGES.
Our research focuses on 3DGS reconstruction of outdoor, unbounded scenes, primarily facing two challenges: 1) Accurate pose estimation outdoors difficult due the unpredictability and com- plexity of outdoor environments; 2) Outdoor scenes generally have sparser camera coverage and less overlap between images compared to indoor settings, resulting in insufficient constraints during training. To address these challenges, we introduced two key modules: Spatial Prior-Based Struc- ture Estimation and a graph- guiph-guueddd Gaussian optimization strategy. These modules are designed to efficiently and accurately complete the reconstruction of outdoor scenes.
However, our method shows limited improvement in 3DGS reconstruction of objects. This is primar- ily because datasets in this category usually have precise ground truth poses, and the camera setups often rotate 360 degrees around the object, providing sufficient image overlap and constraints to aid convergence. This results in nearly equal importance weights for graph nodes, which diminishes the impact of our optimization.
Moreover, our approach relies the accuracy of rough spatial priors. If the initial camera pose distribution in the xy plane is inaccurate, it may lead to Bundle Adjustment (BA) failure. In such cases, our method might not effectively handle pose estimation errors.
20


Reviews:
[{"review_id": "Mt4lliac9X", "reviewer": "Reviewer_4dAT", "summary": "The paper presents a collection of practical optimizations to improve quality and efficiency of Gaussian Splatting reconstructions from image collections without poses. The optimizations relate to \n1. Efficient View-Pair Finding for Match-graph Construction for Structure from Motion\n2. Octree initialization of 3D points and Level-of-details based pruning\n3. Multi-view Consistency Loss in 3DGS optimization\n4. Match-graph / Camera-graph Importance based View Sampling for 3DGS optimization\n\nThe author show results indicating, \nOptimization 1 leads to faster SfM (Table 3) and quality improvements in GS (Table 4)\nOptimizations 2, 4 leads to faster GS optimization (Table 5, Table 7)\nOptimizations 3, 4 leads to quality improvements in GS (Table 4, Table 7)\n\nThe results are evaluated on scenes from Waymo, Kitti, and Mill-19 datasets with images in the range of ~600, ~100, and ~2000.", "questions": "- See questions in Weaknesses sections. \n- What does \"w/o structure estimation\" mean in Table 4. \n- Which datasets are used for results in Table 4, Table 5, Table 7. \n- Is Dust3r used for only pairwise pose estimation or is the step that yields globally aligned point maps and poses also used?\n- Definition of S_3^i in Eqn 1. is ambiguous, can you clarify what does this set include?\n\nMinor : \n- Abstract mentions that : \"This paper investigates ...  reconstructing high-quality, large-scale 3D open scenes from images.\" but the paper deals with scenes with 100 images, 600 images, and two scenes with 1500-2000 images. Typically large-scale in context of SfM and 3DGS refers to city-scale scenes with tens of thousands of images. \n- It should be clarified if I(p) in Eqn 8 means \"intensity or color at pixel p in the image\" refers to GT image or rendered image.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, I like that the authors propose very practical optimizations that bring both quality and run-time improvements. \n\n* Originality: The key original contribution of the paper is: Concentric Nearest Neighbor Pairing (CNNP) and Quadrant Filter (QF) organizing/pruning view-pairs in camera-graph. Other contributions are good practical applications of previously known ideas. \n* Clarity: The paper is written in a clear language, structured well, and shows experimental validation of the proposed ideas. \n* Significance: The paper introduces practical ideas for 3DGS from no-pose image collections.", "weaknesses": "The method proposed in the paper is forming camera-graph using Dust3r to find relative poses between image pairs and pruning pairs using the proposed CNNP and QF steps. The efficiency of structure estimation is estimated w.r.t default COLMAP pipeline (assuming incremental Structure from Motion). This is not a fair comparison and sufficient details are not provided, making it harder to assess the true benefit of the proposed improvements. \n\n- Paper mentions that Dust3r is used to estimate pairwise relative poses in 0.01 seconds. Can you provide a more detailed breakdown of the Dust3r usage, including whether the 0.01 seconds is per pair or total, what specific hardware was used, and how many total pairs were evaluated across the different datasets\n\n- COLMAP exhaustive and COLMAP vocab-tree matching time are provided. However, sufficient details on the experimental setup and compute resources are not provided. For example, for vocab-tree matching, which dataset is used to compute the vocab-tree, how many nearest neighbors are retrieved per image, in total how many pairs are evaluated? What compute resources are used for this matching? \n\nWithout these details, it is difficult to draw conclusions. \n\nAt a more basic level, Dust3r + CNNP + QF contributions are mainly to improve match-graph construction and BA runtime. A fair comparison of the improvements in these runtimes should be with other SoTA efficient SfM methods, not default COLMAP. \n\nDefault incremental SfM implemented with COLMAP is commonly used by radiance field papers to compute poses but by no means this is the most efficient pipeline. There is a vast literature on how to approximate match-graph construction going back a decade. There are well-established alternatives to incremental Structure from Motion with implementations in Open source SfM libraries such as OpenMVG, OpenSfM, Theia, and most recently GLOMAP that offer much better run-time behavior. \n\nGiven the authors use prior poses estimated from Dust3r, the comparison of CNNP and QF steps should be done with match-graph pruning methods that already use prior poses. A naive baseline to compare against would be to construct a camera-graph only from view pairs with overlapping frusta. Can you add this comparison to your evaluation, evaluating both quality and run-time?\n\nI like the practicality of proposed ideas but I don't think that they are contextualized and compared correctly. The other ideas such as LOD-based point pruning and view-importance based sampling are nice practical improvements which provide qualitative and runtime gains w.r.t. original 3DGS paper, however 3DGS provides a baseline not SoTA comparison. A number of methods have been proposed since the original paper to improve both, the quality and efficiency of 3DGS (2DGS, RadSplats, . A few relevant to the paper: Hierarchical 3D Gaussian Representation (Kerbl et al SIGGRAPH Asia 2024), Scaffold-GS (Lu et al CVPR 2024), Octree-GS (Ren et al).  \n\nThe authors can also provide results on MipNeRF360 dataset which is used more commonly in radiance field literature, this will make it easier to compare their results against contemporary 3DGS methods.\n\nAs is, the paper is an assortment of good practical improvements for a sparse recon + 3DGS reconstruction system, and I am positive that these insights can be valuable for practitioners in the field. However, these small contributions are scattered across the pipeline and none are evaluated as thoroughly as they should be with SoTA methods and good baselines respectively for each, making it difficult to place the value/significance of contributions in context of SoTA.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AoMpYFQM6I", "reviewer": "Reviewer_MKvT", "summary": "Focus on the large-scale Gaussian-based reconstruction pipeline, the authors propose a graph-guided framework, GraphGS, which leverages spatial priors for scene structure estimation to create a camera graph encoding camera topology. Using graph-guided multi-view consistency and an adaptive sampling strategy, GraphGS enhances the 3D Gaussian Splatting optimization, mitigating overfitting to sparse viewpoints and accelerating reconstruction. Quantitative and qualitative evaluations across multiple datasets demonstrate that GraphGS achieves state-of-the-art performance in 3D reconstruction.", "questions": "1. If the spatial prior-based structure relies on the initial pose estimation, wouldn’t inaccurate initial results lead to a poorly constructed graph?\n2. In Table 1, why doesn’t the FPS of the proposed method exceed that of the original 3D Gaussian Splatting (3DGS)? Intuitively, the pose estimation should contribute positively. Where is the additional computation time being spent?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. the paper is easy to understand.\n2. the results show the effectiveness of proposed pipeline.", "weaknesses": "1. The manuscript lacks a detailed explanation for each term in Equation (1), which would enhance clarity and understanding.\n2. Many of the authors' methods are designed to improve upon COLMAP. It would be beneficial to include experiments comparing the accuracy of initial values in GS, such as pose accuracy, to illustrate the improvements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m6cYxEncwd", "reviewer": "Reviewer_X4qU", "summary": "The authors propose a framework for large-scale 3D reconstruction using Gaussian splatting.  The authors suggest the construction of a prior graph-guided scene structure. This results in estimating a camera graph that encodes the camera topology. Then based on graph weights the employ an adaptive sampling strategy to the 3D Gaussian splatting optimization.", "questions": "What is the minimum quality of poses required by the proposed framework? \nDoes any of the proposed steps account for large errors in the pose estimation? \nWhy do the authors attempt to compete strongly with COLMAP? COLMAP was used as a method to obtain initial poses and the authors used wang et al. 2024 to obtain initial poses. The framework the authors propose can be used with poses computed by either method.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors present an elaborate pre-processing framework to effectively scale Gaussian splitting to large scenes. The ideas presented in the paper are exciting and well-presented for the most part. The concept of exploiting low-cost prior heuristics to allow the network to focus on the underlying task is interesting and the experimental evaluation demonstrates the effectiveness of the method.", "weaknesses": "1) minor: There are some spelling errors, please proofread the manuscript e.g ( line 135 formed -> form, line 138 initializaition -> initialization )\n\n2) Line 147 - 149: Please specify here which models you use to obtain camera poses. How approximate are these poses? What is the error tolerance? Could you please provide quantitative metrics on the initial pose quality compared to ground truth if available?\n\n 3) For the concentric nn pairing the authors use the symbol S multiple times. It would make sense to use CNNR as a symbol of the overall process output and use more meaningful names than S1 S2 and S3 for the various heuristics. Maybe names related to their role in CNNR computation. \n\n4) In the quadrant filer could you please specify whether the orientation is provided as a normal or any other form (euler angles)?\n\n5) The adaptive sampling section is not clear. \n    - line 322 primarily considers two criteria -> Are there more criteria than these two?\n    - Line 344 We design node weight wn(i) based on betweenness centrality -> So the node weight does not take into account the degree centrality? \n    - How is the view selection probability integrated into the 3DGS optimization? \n\n6) Lines 457-458: Could you please provide more information on how you obtained your initial poses and what is the size of the dataset, the hardware used, or any preprocessing stats that the relative pose estimation is done in 10ms?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)